Release Notes – Deep Learning Tool Early Adopter

This document provides the release notes for MVTec Deep Learning Tool 0.6 Early Adopter, as released on 2021-04-15.

The license of version 0.6 EA is valid until 2022-12-31.

Version 0.6 Early adopter

New Features

• It is now possible to create a new DLT project directly from an existing HALCON dictionary (HDICT) file, so that the project automatically uses the correct deep learning method.
• The current scroll position on the Gallery and Review pages is now maintained after deleting images.
• It is now possible to configure parametrized image augmentation for trainings.
• Next to the “Batch Size” setting, a button was added to determine the maximum possible batch size on a GPU for the current training parameters.
• After the training of a Classification project, it is now possible to add unlabeled images to the project and calculate the inference for these images. On the Confusion Matrix tab, the images can be inspected and directly labeled with the predicted or any other class.
• On the Gallery and Image pages, it is now possible to modify the current split by assigning images to it manually. A split used by a training that was already trained cannot be modified.
• It is now possible to set the random seed that initializes the random number generator, which is used for the training. Thereby, random processes during the training return reproducible results.
• If the label class of an image is changed in the image inspection panel close to the confusion matrix on the Evaluation page such that the image list is rearranged, the grid remains at the previous position now.
• For training and evaluation, it is now possible to select the GPU to be used out of a list of all available GPUs. Further, the name of the GPU or CPU is displayed.
• The quick filter for label class and split type is now automatically activated when the selection changes.
• During the training, after each epoch the new model is stored as best model if the top-1 error on the validation images is lower than the error of a prior training step. Now, the new model is also stored as best model if the error is equal to the prior error.
• If the required HALCON license for Deep Learning Tool has expired, now a clear error message is shown.
• On the Evaluation page, shortcuts have been added to improve the usability: The size of the thumbnails can be changed with Ctrl++ and Ctrl+−, the images can be selected with Ctrl+A and cleared with Esc, and the images can be relabeled with the label classes' shortcuts as on the other pages.

Resolved Issues and Improvements

• When duplicating a training of a modified project, an error occurred. This problem has been fixed.
• DLT could crash if the training was reset shortly after the first epoch. This problem has been fixed.
• When removing and adding images, sometimes the old preprocessed images were still displayed on the Evaluation page. This problem has been fixed.
• On the Evaluation page, stepping through the images near the confusion matrix did not work as expected: When an image was selected and the "go to next image" button was pressed, the evaluation summary for both images was displayed instead of the summary for the next image alone. This problem has been fixed.
• When using “Save as” for a project, the trainings were not copied to the new project. This problem has been fixed.
• A color inconsistency existed between text edit fields and spin boxes. This problem has been fixed.

Version 0.5

New Features

New Split Page

• The new Split page shows the distribution of images to the single classes within a split.
• It is now possible to create and manage several different data splits. Before a model can be trained, a split must be created and assigned to the training. Splits can be renamed, duplicated, and deleted.
• If a training is selected on the Training page or on the Evaluation page, the split that is used by the training is regarded as the active split. The active split can also be set on the new Split page. An active split is displayed on the Gallery page, used for filter operations over images, and used as the default split when a new training is created.

Other New Features

• If a dataset containing split information is imported from an HDICT file, the split information is now always inserted into the active split. This can overwrite the split type of the affected images. If there is no split in the project yet, a new split is created with its name set to the base name of the imported HDICT file. A split that is used by a training is not modified but duplicated before the imported split information is merged into it.
• To get reproducible data splits when generating splits automatically, it is now possible to set the value of the random seed.
• On the Gallery page, now a context menu can be opened for each image with the following options: view the current image in the file browser, delete the current image(s) from the file system, or copy the current image to a different location.
• On the Gallery page, it is now possible to expand the split type overlay on an image. Furthermore, the abbreviation for the split type is only one letter now.
• It is now possible to stop a training after the first epoch and thus mark it as finished.
• It is now possible to export the deep learning model with the option "optimized_for_inference".
• The training settings have been extended with the possibility to configure the weights of the classes that are used during training.
• Before resetting a training, now a confirmation dialog is shown to avoid overwriting a trained model accidentally.
• It is now possible to duplicate a training so that the new training with adjusted settings can be used without losing the first training.
• It is now possible to rename trainings.
• It is now possible to evaluate the model of a paused training.
• It is now possible to change the number of epochs and the learning rate strategy for paused trainings.
• A quick image filter in the filter bar now allows filtering images by any combination of used label classes. In addition, for classification projects there is also a quick filter that allows filtering images by split type of the image within the currently active split.
• On the Training page, now the option "Use deterministic algorithms" is available. If enabled, only deterministic algorithms are used on a GPU to enable reproducible results for each run on the same hardware. This corresponds to setting the system variable 'cudnn_deterministic' to 'true'.
• The End User License Agreement (EULA) has been updated.

Resolved Issues and Improvements

• On high-DPI displays, fonts may have been rendered incorrectly. This problem has been fixed.
• On the Image page, the label class selector of the selected region did not close when clicked. This problem has been fixed.
• If no dataset ('test', 'validation', 'train') is selected on the Evaluation page, the 'test' dataset is used but this was not shown. Now, when starting a training without dataset, 'test' is automatically selected. Furthermore, the parameters are now disabled while computing the evaluation.
• On the Evaluation page, the columns of the Class Overview table were not properly aligned. Further, the column containing the label class names was too small in some cases. These problems have been fixed.
• For non-quadratic images, the heatmap was not displayed properly. This problem has been fixed.
• In the confusion matrix, long class names in the footer row may have been clipped. This problem has been fixed.
• If the top-1 error on the validation and training images was always zero for a training, the corresponding plot was empty. This problem has been fixed.
• On Windows systems with display scaled to 150%, the text in the Labels table on the Image page was too small. This problem has been fixed.

Known Issues

• Running a training or inference could cause CPU memory leak and crashes on machines with CUDA 11.1 and the graphics card series RTX-2000 and RTX-3000. To avoid this issue, select "Use deterministic algorithms" during training configuration, which sets the system variable 'cudnn_deterministic' to 'true'.

Version 0.4.3 Early Adopter

New Features

• The image preprocessing is not performed as a separate step before the actual training anymore. Instead, both now start in parallel without any delay of the training.
• The icon set used by Deep Learning Tool has been updated to meet the new design language of MVTec.
• Deep Learning Tool now is based on HALCON 20.11. In particular:
  • Models are now generated for HALCON 20.11.
  • The new pretrained model MobileNetV2 is available.
  • Deep Learning Tool now supports CUDA 10 and 11.
• Deep Learning Tool is now installed using the MVTec Software Manager (SOM). While MVP, the installer used before, cannot install this version of DLT, you need to use MVP to remove any previous versions.
• Deep Learning Tool now supports assessing the results of a training. Among others, the new Evaluation page offers the following features:
  • The Overview tab shows information about general properties of the training, its accuracy, and quality measures per class. Further, users can decide which dataset to evaluate and adapt the evaluation settings to the hardware.
  • An interactive confusion matrix shows a convenient overview about the performance of the model.
  • It is possible to view the heatmap for the predicted class of all processed images.
  • The user can now attach individual text notes to each trained model. This functionality is available via a Comment area on the new Evaluation page. This comment is shown when hovering over the comment icon of the training items in the training list.
  • The estimated inference time per image can be calculated.
  • A report of the evaluation results can be exported as a single HTML page.
• On the Evaluation page, it is possible to view the heatmap for the predicted class of all processed images.
• If the statistics dialog is displayed and a filter is active, it is now possible to switch between statistics about all images and about filtered images only.
• The user can now attach individual text notes to each trained model. This functionality is available via a Comment area on the new Evaluation page. This comment is shown when hovering over the comment icon of the training items in the training list.
• It is now possible to open a project by dragging the DLT project file from the Windows Explorer and dropping it over the main window of the Deep Learning Tool.
• If a label is deleted while doing a review, the Review page now keeps the current position when returning to the page.
• The button for exporting a trained model is now placed in the appropriate training area within the list of trainings for every training.
• DLT now excludes unlabeled images from the data split.
• It is now possible to resume a training or an evaluation after an error has occurred.

Resolved Issues and Improvements

• It was possible to drag the width of the quick help panel all the way down to zero. In this case, it was not possible to change the width back to a non-zero value. This problem has been fixed.
• On some systems, there could be graphic artifacts due to bad OpenGL drivers. Now, DLT uses ANGLE (DirectX) by default. You can switch back to OpenGL with the environment variable QT_OPENGL=opengl.
• The buttons for navigating to the next and the previous image on the Image page did not react to every click. This problem has been fixed.
• Spin boxes in Deep Learning Tool showed the following erroneous behaviors: After entering a number and then pressing the + or - button, the entered number was ignored. Instead, the old number was increased or decreased. Further, spin boxes could remain empty or display an invalid value after entering an invalid value twice. These problems have been fixed. In addition, when entering an invalid value into a spin box, the value is now displayed in a different color.
• Popup dialogs, like the one for editing the label class or the one for editing the project description, were sometimes clipped at the lower border of Deep Learning Tool such that the buttons of the dialog were neither visible nor usable. This problem has been fixed.
• After importing a dataset with a split, the split was not visible on the Gallery page. This problem has been fixed.
• If several trainings were left in a paused state, the resources of the trainings were not freed. This could lead to out of compute device memory errors. This problem has been fixed.
• When resetting a training, now the training folder is deleted completely and then recreated with the basic parameter files.
• Deep Learning Tool was displayed with 200% scaling on screens configured for 150% scaling. This problem has been fixed.
• Project files stored on a NAS device could sometimes not be opened by Deep Learning Tool and had to be copied to a local device as a workaround. This problem has been fixed.

Version 0.4.2 Early Adopter

New Features

• Deep Learning Tools now offers the possibility to train models for classification projects.
  This includes creating and deleting trainings, and configuring the trainings and the models by setting different parameters. Further, you can start, pause, and stop training runs as well as export the generated models.
  Trainings can be performed on the CPU or, if supported, on the GPU.
  The progress of a training is shown on the Results tab. While the loss and the top-1 error are displayed as plots, further values are shown as numbers.
• The image dataset can now be split into subsets for training, validation, and testing. For this, the ratio of these subsets can be defined, e.g., 70 % training images, 15 % validation images, and 15 % test images. The split dataset can be used for training a model.
  The split is part of the export of a dataset. Furthermore, the Image and Gallery pages gained the functionality to display the split type to which an image belongs. It is also possible to filter by the split type.
• Class IDs in imported HALCON dictionaries now are preserved for the export. The class IDs for classes created in Deep Learning Tool are numbered consecutively.

Resolved Issues and Improvements

• The import of a HALCON dataset dictionary file could fail. This problem has been fixed.
• It is now possible to import HALCON dictionaries that do not contain the key 'samples' or that contain an empty 'samples' tuple.

Version 0.4 Early Adopter

New Features

• The Deep Learning Tool now offers the option to filter the set of images of a project that is worked on. Filters apply to the Gallery, Image, and Review pages, as well as the HDICT export and the statistics.
• The right navigation panel now also shows a miniature image. A rectangle indicates the current image part that is visible in the main window. Further, the navigation panel offers to adapt the zoom level.
• When reviewing labels in case of the object detection scenario with oriented rectangles, it is now possible to rotate the thumbnail view.
• When reviewing labels in case of the object detection scenario with oriented rectangles, it is now possible to adjust the orientation of the selected labels.
• It is now possible to switch between the main pages by using the keyboard shortcuts Alt+1, Alt+2, etc.
• The tab bar now contains a button to open the statistics window. The statistics window can still be opened by clicking on the progress item as well.
• While previous versions of Deep Learning Tool supported Windows 7 and later, version 0.4 EA supports Windows 10 only. The documentation has been adapted accordingly.

Resolved Issues and Improvements

• When the New Project dialog was opened in the default sized Deep Learning Tool, the dialog was not completely visible such that the Browse button was clipped. This problem has been fixed.
• When importing and exporting object detection datasets, the coordinate system was inconsistent with HALCON (shifted by 0.5 pixel). This has been fixed.
• The opacity of the crosslines for labeling could not be changed using the keyboard. This has been fixed. Furthermore, the minimum opacity has been increased to 15%.
• In big projects with many images, removing a widely used label class could take a very long time and Deep Learning Tool seemed to hang. This problem has been fixed. Further, now a status message and, while deleting, a wait cursor are displayed.
• On the Image page, zooming with a track ball (or any mouse with fine-grained zoom steps) did not work properly. This has been fixed.
• When creating a new oriented rectangle, DLT sometimes showed the class name of other labels. This has been fixed. Now, class names are hidden during creating and editing a label.
• In the "Edit User Preferences" dialog, the name of the "Zoom in when moving mouse wheel up" setting was misleading and has been changed to "Invert mouse scroll direction for zooming". By default, this setting is turned off.

Version 0.3.1

New Features

• The license of Deep Learing Tool 0.3 expires on Dec 31, 2020. With version 0.3.1, the license is extended until June 30, 2021.
  Apart from that, no changes have been introduced in this version.
• Deep Learning Tool is now installed using the MVTec Software Manager (SOM). While MVP, the installer used before, cannot install this version of DLT, you need to use MVP to remove any previous versions.

Version 0.3

New Features

• With the new Review page, it is now possible to review labeled images and objects. Particularly, the Review page offers the following functionality:
  • It is possible to change the label class of selected labels.
  • It is possible to delete selected label regions. In case of classification it is possible to delete images on the Review page.
  • Depending on the selection of items in the gallery view of the Review page, the following information is shown in the info panel:
    Single selection
    • Name of the image containing the region
    • Region size
    • Label class
    Multi selection
    • Name of the image containing the regions (if the image is the same for all selected regions)
    • Label class (if the class is the same for all selected regions)
  • The documentation now covers the Review page.
• Deep Learning Tool now also supports classification projects. In addition to assigning such labels to images, this feature includes the following functionality:
  • It is now possible to export labeled images to an HDICT file.
  • An existing HDICT file for classification can be imported. Label classes are created if missing, and the labels are imported. If an image is loaded but assigned to another class already, then the following logic is applied:
    • If the imported image has a label, it overrules any label loaded already.
    • If the imported image has no label, any possibly existing label is kept.
    If the HDICT file is of wrong type, then the import is cancelled and an error message is shown.
  • The documentation now covers the Classification scenario.
• The Deep Learning Tool can be started using defined INI files via command line options:
  reset_preferences: Reset persistent settings to default values.
  add_preferences: Start Deep Learning Tool with additional preferences from a file.
  load_preferences: Reset all persistent settings and start Deep Learning Tool with the preferences from a file.
  use_preferences: Start Deep Learning Tool with the preferences from the file and store all modified preferences in the file.
• On the Projects tab, a summary of the selected project is displayed, which also contains the last modification time of the project and the program version that was used to write the project. This information now always reflects the state of the project file in the file system instead of the state of the current project in memory. Hence, it is not changed for the currently open project until the project is saved.
  In addition, the modification time of a project was not correctly updated when its name or description was changed on the project summary panel without explicitly opening the project before. This problem has been fixed.
• The color picker area on the dialog for creating or editing a label class has been improved. Now it is easier to assign class colors that contrast well with the image contents.
• On the Label tab, it is now possible to copy, cut and paste regions using Ctrl+C, Ctrl+X and Ctrl+V.
• A progress bar now shows the percentage of labeled images.
• If a user tries to save a project file that was modified and saved by another user concurrently, now a warning is displayed saying that continuing to save the project would overwrite the changes made by a different user.
• Class names can now contain any character.
• The Label tab has been extended to select multiple labels at once. While holding the Shift key, draw a rectangle to select all labels within this area.
• A selected label can be deselected by pressing the Esc key.
• If an HDict is imported, an already existing label in an image is not inserted again if position and class are equal.
• If a task takes a long time (for example: loading a project that contains many images), the status bar now indicates that the task is running.
• If an imported HALCON dictionary contains additional data fields for the project or for images, these data fields will now be stored in the project file of the Deep Learning Tool and can be re-exported later. However, this extra data is neither interpreted nor modified. Therefore, the consistency of data that belongs to label classes or label regions is not guaranteed, especially if label classes or label regions are removed or newly created.
• The documentation now contains information about the selection of multiple labels.
• A percentage bar and a dialog now show statistical information about the share of images with labels and the amount of labels for each class.
• The behavior for selecting multiple images in the Gallery tab has been changed. Now, when clicking an image without modifier keys (Shift or Ctrl), the current selection is abandoned and only the image that was clicked is selected. To select multiple images, the Shift or the Ctrl key has to be hold while clicking images. Clicking the check mark icon on the top left of each image still adds the image to the selection.
• Deleting images has been simplified in accordance with the revised selection functionality. When images are about to be deleted, a confirmation dialog is shown.
• The Submit Feedback functionality now opens a form on the MVTec website instead of the email program.
• If one of the pop-up dialogs is opened for editing the project name, the project description, or the label class, the other parts of the application are now disabled as long as the dialog is open.

Resolved Issues and Improvements

• On the Project tab, the summary of projects with many classes had been cropped such that program and file version of the project could not be read. This problem has been fixed.
• When closing and reopening Deep Learning Tool on a screen with high-DPI scaling that was not the primary screen, DLT reopened with half the size. This problem has been fixed.
• On the Gallery tab, there should be a "Browse" button for images that cannot be found in the file system under the given path. Depending on the selected language and the thumbnail size, these buttons could have disappeared. For some languages, the buttons were not visible even with the default thumbnail size. This problem has been fixed.
• If the GUI language was changed, the documentation was still opened in the default (English) language. Now the documentation is opened in the chosen language if it is available.
• Sometimes the label class list could not be scrolled. This issue has been fixed.
• In some dialogs it could have happened that the characters of the text were randomly displaced with a kind of jitter. This problem has been fixed.
• It was not possible to open RGB color images with a width greater than 8192 pixels. This problem has been fixed. Now the maximum image size is 32768 x 32768 pixels.
• On the Image tab, the class name next to labels did not vanish on hover out. This problem has been fixed.
• The deletion of a great number of images from the project took a very long time. This problem has been fixed.
• After deleting images with label regions from the project, old labels of the removed images could re-appear when new images were added to the project. This problem has been fixed.
• The HDict exported by DLT contained arbitrary IDs for label classes. This has been fixed. Now, the IDs of the label classes are enumerated from zero.
• When a corrupted working copy was saved (for example, due to a crash while writing data), DLT has crashed on next startup. This has been fixed. Now, leftovers will be restored in the state before the last action.
• It was not possible to open project files in the Deep Learning Tool if the read-only flag of the file was set or if the user had no write permission to the file. This problem has been fixed. Now it is possible to open projects without write permission as long as the user has read permission. When modifying such projects, they must be saved under a different name.
• If a project contains images that cannot be found in the file system, the user can search for the images and correct their paths. This did not work if the user moved the images to the folders used in the project, that is if the paths should work now without being changed. In this case, an error message was displayed. This problem has been fixed. Now, the images are also reloaded if the paths were not changed.
• When an old project file that was written with a previous version of the Deep Learning Tool was opened, the file version displayed on the Project tab was wrong. This problem has been fixed.
• When importing an image folder, all images in the currently displayed folder were imported instead of the currently selected folder. This has been fixed.
• After opening the Help menu, the shortcut F1 for displaying the help page in the browser worked only every second time. The same problem occurred with the shortcut F11 for switching into the full screen mode after the menu was opened. These problems have been fixed.

Known Issues

• In very rare cases, Deep Learning Tool crashes during startup. This is probably caused by an incompatible hardware setup. If you encounter such issues, please contact MVTec.

Version 0.2

New Features

• The Deep Learning Tool now allows to create projects for object detection with arbitrarily oriented rectangles as label regions. After creating such a project it is possible to draw and edit oriented rectangles. Furthermore, it is now possible to import label data from HALCON dictionaries as well as to export the label data into a HALCON dictionary.
• The Deep Learning Tool is now able to retrieve an RSS news feed published by MVTec. The latest news will be displayed automatically at the startup of the application. To request the news manually, click "News" in the help menu.
• The Deep Learning Tool now supports multiple languages (depending on what is installed in a certain program folder). The language can be configured via user preferences. Currently, the Deep Learning Tool supports English, German, simplified Chinese and Japanese. It is possible to use a customized translation.
• The Gallery tab has been extended by a multi selection functionality. Now, it is possible to select multiple images and delete them in one step.
• The Deep Learning Tool now supports the loading of images in more file formats and pixel types. This includes mainly the image file formats TIFF, JPEG-2000, JPEG-XR, and HOBJ. The pixel type of the loaded images is no longer restricted to byte images.
• The Deep Learning Tool has been extended with a feedback functionality. To send feedback to MVTec via email, open the help menu and choose "Feedback".
• Deleting a label class may destroy a lot of work. Therefore, the user has to confirm this action now via a warning dialog. In addition, by deleting a label class all labels of that class are also deleted.
• The application now has an entry "MVTec Deep Learning Tool" in the Windows Start Menu.

Resolved Issues and Improvements

• When creating a new project, it is possible to enter the path to the project file. When instead of the project file an exiting folder was entered, a message box asked the user if the file should be replaced, which is unexpectedly. Further, when trying to save the project an error message was printed that the file could not be saved. This problem has been fixed. Now, when an existing folder is entered as project path, the project's name is used as file name, which is created in the given folder. In addition, the extension .dltp is automatically added, if it was not entered in the path.
• On the Label tab selecting all labels at once with the shortcut Ctrl+A did not work when the image had the keyboard focus. It worked only when the table "Labels" had the keyboard focus. This problem has been fixed. Now, on the whole Label tab Ctrl+A can be used for selecting all labels.
• The shortcut Ctrl+W did not work for closing the current project. This problem has been fixed.
• When several HALCON dictionaries with different labels for the same images were imported one after the other, only the labels of the last imported dictionary were displayed, all other labels seemed to be lost. This problem has been fixed. Now, all imported labels are kept in the project and are displayed correctly.
• The "Save Project" dialog appeared on loading projects that have already been opened. Now, the view is switched to the Gallery tab instead.
• The dialog for adding/editing label classes did not contain any error indication. This problem has been fixed.
• Zooming the thumbnails in the Gallery tab using Ctrl+mouse wheel did not work if the mouse cursor was positioned over a quick info (file path).
• When creating a new class, the class was not selected. Now, the focus is on the newly created class.
• The scrolling speed of the image list in the Gallery tab has been improved.
• For creating a new label class the text field was replaced by a simple + button. Clicking this button opens a popup dialog that allows to enter the label class name and to select a color for the class.
• After clicking "Save Project As..." the current UI state was lost. Now, the application stays in the same state (on the same image in Label tab / on the same tab) as before.