MVTec Software GmbH

Deep Learning

MVTec software products offer a large selection of operators, functions and methods that are either based on deep learning technologies, or allow customers to use deep learning technologies in their own applications.

Deep-Learning-Powered Technologies

Featured in HALCON and MERLIC

HALCON (free trial here) and MERLIC (free trial here) both include deep-learning-based OCR (check out this video to see it in action). Training of these new classifiers has been conducted with the help of deep learning technology. With these it is possible to achieve higher reading rates than with all previous classification methods.

Deep Learning in Your Application

Featured in HALCON

HALCON allows customers to use deep learning technologies in their own applications. Users can train their own classifier using CNNs (Convolutional Neural Networks). These can then be utilized to classify new data.

HALCON offers a seamlessly integrated, comprehensive set of deep learning functions for:

Semantic Segmentation
Object Detection
Anomaly Detection

Deep-learning-based image classification allows to easily assign images to trained classes. The low labeling effort enables particularly short set-up times, and applying the classifier to new data is especially fast.

With semantic segmentation, trained defect classes can be localized with pixel accuracy. This allows users to, e.g., solve inspection tasks, which previously could not be realized, or only with significant programming effort.

Object detection localizes trained object classes and identifies them with a surrounding rectangle (bounding box). Touching or partially overlapping objects are also separated, enabling object counting. HALCON also gives users the option to have these rectangles aligned according to the orientation of the object, resulting in a more precise detection, as rectangles then match the shape of the object more closely.

Anomaly detection significantly facilitates the automated surface inspection for, e.g., detection and segmentation of defects. The technology is able to unerringly and independently localize deviations, i.e., defects of any type, on subsequent images. You only need a low number of high quality images for training because defects of varying appearance can be detected without any previous knowledge or any preceding labeling efforts. Training a new network can mostly be done in a matter of seconds, allowing users to perform many iterations to fine-tune their application without sacrificing a lot of precious time. Additionally the inference has been accelerated with the current version.

The training for these four deep learning technologies can be performed on CPU. This greatly increases customers' flexibility in implementing deep learning, because training can now be performed directly on the production line, making it possible to adjust the application to changing external conditions "on the fly". The inference can be performed on GPUs, on x86 CPUs and on Arm® processors.


Labeling of data

Screenshot: MVTec Deep Learning Tool
A screenshot of the MVTec Deep Learning Tool

With the Deep Learning Tool, MVTec provides a solution for labeling training data required for HALCON's object detection and classification. The resulting HALCON dictionary can be seamlessly integrated in HDevelop to train the network.

To help you label data for HALCON's other deep learning technologies, we provide various HDevelop helper scripts.

Customers who already have their data labeled (with any tool) can get in touch with us to have their training data set converted into a format to be easily integrated into HDevelop.

Training user-specific CNNs

training the network by labeling images
CNN training with labeled image data

With HALCON, users are able to train their own classifier using pretrained CNNs (Convolutional Neural Networks) included in HALCON. These networks have been highly optimized for industrial applications and are based on hundreds of thousands of images. After training the CNN, it can be used for classifying new data.

Furthermore, previously created 3rd party networks, which have been exported into the ONNX (Open Neural Network Exchange) format can be used in HALCON.

Training is done by providing a sufficient amount of properly labeled training images. The software then analyzes these images and automatically learns which features can be used to identify the given classes. This is a big advantage compared to traditional classification methods, where these features had to be "handcrafted" by the user – a complex and cumbersome undertaking that requires skilled engineers with programming and vision knowledge.

Using the trained networks

Defect classification with deep learning
Trained CNN classifying defects on a per-image basis

Once the network has learned to differentiate between the given classes, e.g., tell if an image shows a scratched, a contaminated or a good sample, the network can be put to work. This means, users can then apply the newly created CNN classifier to new image data which the classifier then matches to the classes it has learned during training – this is also called "inference".This inference can be executed on GPUs, as well as on CPUs.

Application Areas

When looking for real-world applications, CNNs can for example be used for defect classification (e.g., for circuit boards, bottle mouths or pills), or for object classification (e.g., identifying the species of a plant from one single image).

Developed with Key Customers from Various Industries

While developing HALCON's deep learning features, we worked closely together with key pilot customers from various industries. Below, you find a quick overview of the different challenges and how deep learning helped us solve them.

Plant identification

The task: Identify plants

Before deep learning

New plant types had to be manually programmed. Separate MLP classifiers were used for feature extraction for each country – a time-consuming process.

With deep learning

It took us only around two weeks to solve the same tasks The error rate could be halved.

Identify emtpy rack spaces with deep learning

The task: identify empty rack spaces

Before deep learning

An elaborate hardware setup with 3D sensors was needed.

With deep learning

The solution works with 2D image data which - thanks to cheaper image capturing and quicker classification - translated into cost savings. In addition, the error rate could be significantly reduced.

Mood image: detect defective pills with deep learning

The task: detect defects on pills

Before deep learning

New defect classes had to be programmed manually and on site – a time-consuming and costly process.

With deep learning

New defect types can be trained automatically, a huge time- and cost-saver because engineers no longer have to travel to the customer's plant.

Lands inspection with deep learning

The task: check lands for defects

Before deep learning

A relatively high error rate required a lot of manual inspection by workers, slowing down production and increasing costs.

With deep learning

Implementing a deep-learning-based defect detection led to a massive reduction of the error rate, which drastically reduced the need for manual inspection, speeding up production and scaling down labor costs.

Deep Learning Helper

Below you will find a few additional downloads to help you getting started with deep learning in HALCON.



HDevelop script for labeling your data for semantic segmentaton.

Note: To label data for object detection and classification, we recommend using the MVTec Deep Learning Tool.


(19 MB)

HDevelop example script that demonstrates how to create a DLDataset dictionary for object detection from existing labeled data


(3.4 MB)

Minimal version of the object detection example


(3 KB)

Minimal version of the semantic segmentation example


(3 KB)