gen_dl_model_heatmap — Infer the sample and generate a heatmap.
gen_dl_model_heatmap infers the sample given
DLSample and generates a heatmap for the class specified by
DLSample can be a single dictionary or a tuple of dictionaries for
To do so, the operator uses the deep learning model given by
HeatmapMethod determines, which method is used
to calculate the heatmap.
The operator returns
DLResult with a result dictionary for every
DLResult will be newly created
and already existing parameters with this name will be overwritten.
A heatmap can be useful to analyze which parts of an image have a strong
influence for the inference into a certain class.
Shape and area of these parts and therewith the heatmaps may vary widely
for differing networks especially if their architectures differ.
There are different methods how a heatmap is calculated.
The following ones can be selected using
Gradient-weighted Class Activation Mapping (Grad-CAM). See reference cited below for further information.
The input parameter
TargetClasses determines the target class, the
class for which the heatmap is generated.
As value the class ID has to be set.
Alternatively, an empty tuple can be handed over. In this case, the heatmap
is calculated for the class with the highest confidence value, thus the
top inferred class.
TargetClasses only supports a single value or an
The following entries can be set in the dictionary
Specifies the convolution layer whose activation and gradient are
used for the heatmap.
The convolution layer can be specified using its
as returned using 'summary' in
As default, the last (deepest) convolution layer of the network with a
non-trivial activation (width and height not equal to 1) is used.
As a general rule, the activation of the deepest convolution layer
is most suitable for calculating the heatmap, therefore it is
recommended to keep the default layer.
In case the convolution layer (selected with
is fused with a directly following ReLU activation layer, this parameter can
be used to determine, from which of these layers the activation and gradient
will be used for the heatmap. The following values are supported:
'true': convolution layer
'false': ReLU layer
use_conv_only is set to 'true'.
Sets the scaling method for the heatmap. The following values are supported:
'scale_after_relu': Negative values of the heatmap are set to 0 and then all values are scaled within the range [0,1]. As a consequence, areas within the heatmap can attain very high values close to 1 although their contribution to the classification result might be small.
'scale_before_relu': All values of the heatmap are divided by the maximum absolute value and then negative values are set to 0. This leads to values for the heatmap in [0,1]. However, areas of less activation (and hence with a small contribution to the classification result) may not tend to attain values near 1.
'none': The heatmap values are not scaled.
scaling is set to 'scale_after_relu'.
Every output dictionary in
DLResult contains the inference results
as obtained using
Additionally it includes a nested dictionary under the key
method is the name of the
specified method as given in
In this nested dictionary the heatmap is saved under the key
classID is the
ID of the target class.
To run this operator on GPU by setting 'runtime' to
get_dl_model_param), cuDNN and cuBLAS are required.
For further details, please refer to the
paragraph “Requirements for Deep Learning and Deep-Learning-Based Methods”.
Alternatively, this operator
can also be run on CPU by setting 'runtime' to 'cpu'.
The heatmap should be used as a tool for visualizing and better
understanding classification results. It is not intended as a
gen_dl_model_heatmap currently only supports models with
Handle of a Deep learning model.
Dictionaries with the sample input data.
Method to be used for the heatmap calculation.
Default value: 'grad_cam'
ID of the target class.
Default value: 
Dictionary for generic parameters.
Default value: 
Dictionaries with the result data.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra: "Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization," 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp. 618-626. doi: 10.1109/ICCV.2017.74
Deep Learning Inference