Anomaly Detection

List of Operators ↓

This chapter explains how to use anomaly detection based on deep learning.

With anomaly detection we want to detect whether or not an image contains anomalies. An anomaly means something deviating from the norm, something unknown.

An anomaly detection model learns common features of images without anomalies. The trained model will infer, how likely an input image contains only learned features or if the image contains something different. Latter one is interpreted as an anomaly. This inference result is returned as a gray value image. The pixel values therein indicate how likely the corresponding pixels in the input image pixels show an anomaly.

General Workflow

In this paragraph, we describe the general workflow for an anomaly detection task based on deep learning.

Preprocess the data

This part is about how to preprocess your data.

1. The information content of your dataset needs to be converted. This is done by the procedure

   • read_dl_dataset_anomaly.

   It creates a dictionary DLDatasetDLDatasetDLDatasetDLDatasetDLDatasetdldataset which serves as a database and stores all necessary information about your data. For more information about the data and the way it is transferred, see the section “Data” below and the chapter Deep Learning / Model.

2. Split the dataset represented by the dictionary DLDatasetDLDatasetDLDatasetDLDatasetDLDatasetdldataset. This can be done using the procedure

   • split_dl_dataset.

3. The network imposes several requirements on the images. These requirements (for example the image size and gray value range) can be retrieved with

   • get_dl_model_paramget_dl_model_paramGetDlModelParamGetDlModelParamGetDlModelParamget_dl_model_param.

   For this you need to read the model first by using

   • read_dl_modelread_dl_modelReadDlModelReadDlModelReadDlModelread_dl_model.

4. Now you can preprocess your dataset. For this, you can use the procedure

   • preprocess_dl_dataset.

   In case of custom preprocessing, this procedure offers guidance on the implementation.

   To use this procedure, specify the preprocessing parameters as, e.g., the image size. Store all the parameter with their values in a dictionary DLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamdlpreprocess_param, for which you can use the procedure

   • create_dl_preprocess_param.

   We recommend to save this dictionary DLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamdlpreprocess_param in order to have access to the preprocessing parameter values later during the inference phase.

Training of the model

This part explains how to train a DL anomaly detection model.

1. Set the training parameters and store them in the dictionary TrainingParamTrainingParamTrainingParamTrainingParamtrainingParamtraining_param. This can be done using the procedure

   • create_dl_train_param.

2. Train the model. This can be done using the procedure

   • train_dl_model.

   The procedure expects:

   • the model handle DLModelHandleDLModelHandleDLModelHandleDLModelHandleDLModelHandledlmodel_handle

   • the dictionary DLDatasetDLDatasetDLDatasetDLDatasetDLDatasetdldataset containing the data information

   • the dictionary TrainParamTrainParamTrainParamTrainParamtrainParamtrain_param containing the training parameters

Evaluation of the trained model

In this part, we evaluate the anomaly detection model.

1. Set the model parameters which may influence the evaluation.

2. The evaluation can be done conveniently using the procedure

   • evaluate_dl_model.

   This procedure expects a dictionary GenParamEvalGenParamEvalGenParamEvalGenParamEvalgenParamEvalgen_param_eval with the evaluation parameters.

3. The dictionary EvaluationResultsEvaluationResultsEvaluationResultsEvaluationResultsevaluationResultsevaluation_results holds the desired evaluation measures.

Inference on new images

This part covers the application of a DL anomaly detection model. For a trained model, perform the following steps:

1. Request the requirements the model imposes on the images using the operator

   • get_dl_model_paramget_dl_model_paramGetDlModelParamGetDlModelParamGetDlModelParamget_dl_model_param

   or the procedure

   • create_dl_preprocess_param_from_model.

2. Set the model parameter described in the section “Model Parameters” below, using the operator

   • set_dl_model_paramset_dl_model_paramSetDlModelParamSetDlModelParamSetDlModelParamset_dl_model_param.

3. Generate a data dictionary DLSampleDLSampleDLSampleDLSampleDLSampledlsample for each image. This can be done using the procedure

   • gen_dl_samples_from_images.

4. Every image has to be preprocessed the same way as for the training. For this, you can use the procedure

   • preprocess_dl_samples.

   When you saved the dictionary DLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamdlpreprocess_param during the preprocessing step, you can directly use it as input to specify all parameter values.

5. Apply the model using the operator

   • apply_dl_modelapply_dl_modelApplyDlModelApplyDlModelApplyDlModelapply_dl_model.

6. Retrieve the results from the dictionary 'DLResult'"DLResult""DLResult""DLResult""DLResult""DLResult".

Data

We distinguish between data used for training, evaluation, and inference on new images.

As a basic concept, the model handles data by dictionaries, meaning it receives the input data from a dictionary DLSampleDLSampleDLSampleDLSampleDLSampledlsample and returns a dictionary DLResultDLResultDLResultDLResultDLResultdlresult and DLTrainResultDLTrainResultDLTrainResultDLTrainResultDLTrainResultdltrain_result, respectively. More information on the data handling can be found in the chapter Deep Learning / Model.

Classes

In anomaly detection there are exactly two classes:

• 'ok'"ok""ok""ok""ok""ok", meaning without anomaly, class ID 0.

• 'nok'"nok""nok""nok""nok""nok", meaning with anomaly, class ID 1 (on pixel values an ID >0, see the subsection “Data for evaluation” below).

These classes apply to the whole image as well as single pixels.

Data for training

This dataset consists only of images without anomalies and the corresponding information. They have to be provided in a way the model can process them. Concerning the image requirements, find more information in the section “Images” below.

The training data is used to train a model for your specific task. With the aid of this data the model can learn which features the images without anomalies have in common.

Data for evaluation

This dataset should include images without anomalies but it can also contain images with anomalies. Every image within this set needs a ground truth label image_label specifying the class of the image (see the section above). This indicates if the image shows an anomaly ('nok'"nok""nok""nok""nok""nok") or not ('ok'"ok""ok""ok""ok""ok").

Evaluating the model performance on finding anomalies can visually also be done on pixel level if an image anomaly_file_name is included in the DLSampleDLSampleDLSampleDLSampleDLSampledlsample dictionary. In this image anomaly_file_name every pixel indicates the class ID, thus if the corresponding pixel in the input image shows an anomaly (pixel value > 0) or not (pixel value equal to 0).

(1)	(2)

Scheme of anomaly_file_name. For visibility, gray values are used to represent numbers. (1) Input image. (2) The corresponding anomaly_file_name providing the class annotations, 0: 'ok'"ok""ok""ok""ok""ok" (white and light gray), 2: 'nok'"nok""nok""nok""nok""nok" (dark gray).

Images

The model poses requirements on the images, such as the dimensions, the gray value range, and the type. The specific values depend on the model itself. See the documentation of read_dl_modelread_dl_modelReadDlModelReadDlModelReadDlModelread_dl_model for the specific values of different models. For a read model they can be queried with get_dl_model_paramget_dl_model_paramGetDlModelParamGetDlModelParamGetDlModelParamget_dl_model_param. In order to fulfill these requirements, you may have to preprocess your images. Standard preprocessing of an entire sample, including the image, is implemented in preprocess_dl_samples. In case of custom preprocessing these procedure offers guidance on the implementation.

Model output

As training output, the operator train_dl_model_anomaly_datasettrain_dl_model_anomaly_datasetTrainDlModelAnomalyDatasetTrainDlModelAnomalyDatasetTrainDlModelAnomalyDatasettrain_dl_model_anomaly_dataset will return a dictionary DLTrainResultDLTrainResultDLTrainResultDLTrainResultDLTrainResultdltrain_result with the best obtained error received during training and the epoch in which this error was achieved.

As inference and evaluation output, the model will return a dictionary DLResultDLResultDLResultDLResultDLResultdlresult for every sample. For anomaly detection, this dictionary includes the following extra entries:

• anomaly_score: A score indicating how likely the entire image is to contain an anomaly. This score is based on the pixel scores given in anomaly_image.

• anomaly_image: An image, where the value of each pixel indicates how likely its corresponding pixel in the input image shows an anomaly (see the illustration below). The values are .

(1)	(2)

Scheme of anomaly_image. For visualization purpose, gray values are used to represent numbers. (1) The anomaly_file_name providing the class annotations, 0: 'ok'"ok""ok""ok""ok""ok" (white and light gray), 2: 'nok'"nok""nok""nok""nok""nok" (dark gray) (2) The corresponding anomaly_image.

Specific Parameters

For an anomaly detection model, the model parameters as well as the hyperparameters are set using set_dl_model_paramset_dl_model_paramSetDlModelParamSetDlModelParamSetDlModelParamset_dl_model_param. The model parameters are explained in more detail in get_dl_model_paramget_dl_model_paramGetDlModelParamGetDlModelParamGetDlModelParamget_dl_model_param. As the training is done utilizing the full dataset at once and not batch-wise, certain parameters as e.g., 'batch_size_multiplier'"batch_size_multiplier""batch_size_multiplier""batch_size_multiplier""batch_size_multiplier""batch_size_multiplier" have no influence.

The model returns scores but classifies neither pixel nor image as showing an anomaly or not. For this classification, thresholds need to be given, setting the minimum score for a pixel or image to be regarded as anomalous. You can estimate possible thresholds using the procedure compute_dl_anomaly_thresholds. Applying these thresholds can be done with the procedure threshold_dl_anomaly_results. As results the procedure adds the following (threshold depending) entries into the dictionary DLResultDLResultDLResultDLResultDLResultdlresult of a sample:

anomaly_class

The predicted class of the entire image (for the given threshold).

anomaly_class_id

ID of the predicted class of the entire image (for the given threshold).

anomaly_region

Region consisting of all the pixels that are regarded as showing an anomaly (for the given threshold, see the illustration below).

(1)	(2)

Scheme of anomaly_region. For visualization purpose, gray values are used to represent numbers. (1) The anomaly_image with the obtained pixel scores. (2) The corresponding anomaly_region.

List of Operators

train_dl_model_anomaly_datasetTrainDlModelAnomalyDatasettrain_dl_model_anomaly_datasetTrainDlModelAnomalyDatasettrain_dl_model_anomaly_dataset

Train a deep learning model for anomaly detection.