Anomaly Detection

List of Operators ↓

This chapter explains how to use anomaly detection based on deep learning.

With anomaly detection we want to detect whether or not an image contains anomalies. An anomaly means something deviating from the norm, something unknown.

A possible example for anomaly detection: Every pixel of the input image gets assigned a value that indicates how likely the pixel is to be an anomaly. The worm is not part of the worm-free apples the model has seen during training and therefore its pixels get a much higher score.

An anomaly detection model learns common features of images without anomalies. The trained model will infer, how likely an input image contains only learned features or if the image contains something different. Latter one is interpreted as an anomaly. This inference result is returned as a gray value image. The pixel values therein indicate how likely the corresponding pixels in the input image pixels show an anomaly.

General Workflow

In this paragraph, we describe the general workflow for an anomaly detection task based on deep learning.

Preprocess the data

This part is about how to preprocess your data.

  1. The information content of your dataset needs to be converted. This is done by the procedure

    • read_dl_dataset_anomaly.

    It creates a dictionary DLDatasetDLDatasetDLDatasetDLDatasetDLDatasetdldataset which serves as a database and stores all necessary information about your data. For more information about the data and the way it is transferred, see the section “Data” below and the chapter Deep Learning / Model.

  2. Split the dataset represented by the dictionary DLDatasetDLDatasetDLDatasetDLDatasetDLDatasetdldataset. This can be done using the procedure

    • split_dl_dataset.

  3. The network imposes several requirements on the images. These requirements (for example the image size and gray value range) can be retrieved with

    For this you need to read the model first by using

  4. Now you can preprocess your dataset. For this, you can use the procedure

    • preprocess_dl_dataset.

    In case of custom preprocessing, this procedure offers guidance on the implementation.

    To use this procedure, specify the preprocessing parameters as, e.g., the image size. Store all the parameter with their values in a dictionary DLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamdlpreprocess_param, for which you can use the procedure

    • create_dl_preprocess_param.

    We recommend to save this dictionary DLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamdlpreprocess_param in order to have access to the preprocessing parameter values later during the inference phase.

Training of the model

This part explains how to train a DL anomaly detection model.

  1. Set the training parameters and store them in the dictionary TrainingParamTrainingParamTrainingParamTrainingParamtrainingParamtraining_param. This can be done using the procedure

    • create_dl_train_param.

  2. Train the model. This can be done using the procedure

    • train_dl_model.

    The procedure expects:

    • the model handle DLModelHandleDLModelHandleDLModelHandleDLModelHandleDLModelHandledlmodel_handle

    • the dictionary DLDatasetDLDatasetDLDatasetDLDatasetDLDatasetdldataset containing the data information

    • the dictionary TrainParamTrainParamTrainParamTrainParamtrainParamtrain_param containing the training parameters

Evaluation of the trained model

In this part, we evaluate the anomaly detection model.

  1. Set the model parameters which may influence the evaluation.

  2. The evaluation can be done conveniently using the procedure

    • evaluate_dl_model.

    This procedure expects a dictionary GenParamEvalGenParamEvalGenParamEvalGenParamEvalgenParamEvalgen_param_eval with the evaluation parameters.

  3. The dictionary EvaluationResultsEvaluationResultsEvaluationResultsEvaluationResultsevaluationResultsevaluation_results holds the desired evaluation measures.

Inference on new images

This part covers the application of a DL anomaly detection model. For a trained model, perform the following steps:

  1. Request the requirements the model imposes on the images using the operator

    or the procedure

    • create_dl_preprocess_param_from_model.

  2. Set the model parameter described in the section “Model Parameters” below, using the operator

  3. Generate a data dictionary DLSampleDLSampleDLSampleDLSampleDLSampledlsample for each image. This can be done using the procedure

    • gen_dl_samples_from_images.

  4. Every image has to be preprocessed the same way as for the training. For this, you can use the procedure

    • preprocess_dl_samples.

    When you saved the dictionary DLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamDLPreprocessParamdlpreprocess_param during the preprocessing step, you can directly use it as input to specify all parameter values.

  5. Apply the model using the operator

  6. Retrieve the results from the dictionary 'DLResult'"DLResult""DLResult""DLResult""DLResult""DLResult".


We distinguish between data used for training, evaluation, and inference on new images.

As a basic concept, the model handles data by dictionaries, meaning it receives the input data from a dictionary DLSampleDLSampleDLSampleDLSampleDLSampledlsample and returns a dictionary DLResultDLResultDLResultDLResultDLResultdlresult and DLTrainResultDLTrainResultDLTrainResultDLTrainResultDLTrainResultdltrain_result, respectively. More information on the data handling can be found in the chapter Deep Learning / Model.


In anomaly detection there are exactly two classes:

These classes apply to the whole image as well as single pixels.

Data for training

This dataset consists only of images without anomalies and the corresponding information. They have to be provided in a way the model can process them. Concerning the image requirements, find more information in the section “Images” below.

The training data is used to train a model for your specific task. With the aid of this data the model can learn which features the images without anomalies have in common.

Data for evaluation

This dataset should include images without anomalies but it can also contain images with anomalies. Every image within this set needs a ground truth label image_label specifying the class of the image (see the section above). This indicates if the image shows an anomaly ('nok'"nok""nok""nok""nok""nok") or not ('ok'"ok""ok""ok""ok""ok").

Evaluating the model performance on finding anomalies can visually also be done on pixel level if an image anomaly_file_name is included in the DLSampleDLSampleDLSampleDLSampleDLSampledlsample dictionary. In this image anomaly_file_name every pixel indicates the class ID, thus if the corresponding pixel in the input image shows an anomaly (pixel value > 0) or not (pixel value equal to 0).

image/svg+xml image/svg+xml
(1) (2)
Scheme of anomaly_file_name. For visibility, gray values are used to represent numbers. (1) Input image. (2) The corresponding anomaly_file_name providing the class annotations, 0: 'ok'"ok""ok""ok""ok""ok" (white and light gray), 2: 'nok'"nok""nok""nok""nok""nok" (dark gray).

The model poses requirements on the images, such as the dimensions, the gray value range, and the type. The specific values depend on the model itself. See the documentation of read_dl_modelread_dl_modelReadDlModelReadDlModelReadDlModelread_dl_model for the specific values of different models. For a read model they can be queried with get_dl_model_paramget_dl_model_paramGetDlModelParamGetDlModelParamGetDlModelParamget_dl_model_param. In order to fulfill these requirements, you may have to preprocess your images. Standard preprocessing of an entire sample, including the image, is implemented in preprocess_dl_samples. In case of custom preprocessing these procedure offers guidance on the implementation.

Model output

As training output, the operator train_dl_model_anomaly_datasettrain_dl_model_anomaly_datasetTrainDlModelAnomalyDatasetTrainDlModelAnomalyDatasetTrainDlModelAnomalyDatasettrain_dl_model_anomaly_dataset will return a dictionary DLTrainResultDLTrainResultDLTrainResultDLTrainResultDLTrainResultdltrain_result with the best obtained error received during training and the epoch in which this error was achieved.

As inference and evaluation output, the model will return a dictionary DLResultDLResultDLResultDLResultDLResultdlresult for every sample. For anomaly detection, this dictionary includes the following extra entries: