create_dl_layer_loss_cross_entropy — Create a cross entropy loss layer.
create_dl_layer_loss_cross_entropy( : : DLLayerInput, DLLayerTarget, DLLayerWeights, LayerName, LossWeight, GenParamName, GenParamValue : DLLayerLossCrossEntropy)
The operator create_dl_layer_loss_cross_entropy creates a cross
entropy loss layer whose handle is returned in
DLLayerLossCrossEntropy.
This layer computes the pixel-level cross entropy loss between
DLLayerInput and DLLayerTarget, optionally using per-pixel
weights given by DLLayerWeights.
Cross entropy is commonly used to measure the similarity between two vectors.
Illustrative example, where we have a pixel-level classification problem with three classes.
The input vector for a single pixel is and gives the predicted probability for each of the three classes.
The target probability vector is with a probability of 1.0 for the true class and 0.0 else. Entropy is calculated by the dot product of these two vectors. Since the target vector has only one non-zero entry, it can be given by the index of the true class instead of a vector, in this case .
The cross entropy is then simply the value of the input vector at the target class index, hence . Using this simplification, the cross entropy loss function over an input image can be defined by where the input consists of one prediction vector for each pixel, the target and weight consist of one value and for each input pixel, is the number of pixels and is the sum over all weights.
This layer expects multiple layers as input:
DLLayerInput: Specifies the predicted class probabilities.
These must be given as log values (e.g., the output of a softmax layer in
training mode, see create_dl_layer_softmax).
DLLayerTarget: Specifies the target class probabilities.
These can be given as by the class index of the true class (hard label) or
by a vector of class probabilities for all classes (soft label).
In case of class index targets, the values must lie within
'[0,...,num_classes-1]'. Invalid class indices are ignored
internally and have no influence on the loss computation.
In case of class probability targets, the values must describe a valid
probability distribution over all classes. The probabilities must be
non-negative and sum to 1.
DLLayerWeights: Specifies optional per-pixel weights.
If an empty tuple [] is passed, each pixel has an implicit weight
of 1.0.
The parameter LayerName sets an individual layer name.
Note that if creating a model using create_dl_model each layer of
the created network must have a unique name.
The parameter LossWeight determines the scalar weight factor with
which the loss, calculated in this layer, is multiplied.
This parameter can be used to specify the contribution of the cross entropy
loss to the overall network loss in case multiple loss layers are used.
The following generic parameters GenParamName and the corresponding
values GenParamValue are supported:
Determines whether apply_dl_model will include the output of this
layer in the dictionary DLResultBatch even without specifying this
layer in Outputs ('true') or not ('false').
Default: 'false'
Certain parameters of layers created using this operator
create_dl_layer_loss_cross_entropy can be set and retrieved using
further operators.
The following tables give an overview, which parameters can be set
using set_dl_model_layer_param and which ones can be retrieved
using get_dl_model_layer_param or get_dl_layer_param.
Note, the operators set_dl_model_layer_param and
get_dl_model_layer_param require a model created by
create_dl_model.
| Layer Parameters | set |
get |
|---|---|---|
'input_layer' (DLLayerInput, DLLayerTarget, and/or DLLayerWeights) |
x
|
|
'loss_weight' (LossWeight) |
x |
x
|
'name' (LayerName) |
x |
x
|
'output_layer' (DLLayerLossCrossEntropy) |
x
|
|
| 'shape' | x
|
|
| 'type' | x
|
| Generic Layer Parameters | set |
get |
|---|---|---|
| 'is_inference_output' | x |
x
|
| 'num_trainable_params' | x
|
DLLayerInput (input_control) dl_layer → (handle)
Input layer.
DLLayerTarget (input_control) dl_layer → (handle)
Target layer.
DLLayerWeights (input_control) dl_layer → (handle)
Weights layer.
LayerName (input_control) string → (string)
Name of the output layer.
LossWeight (input_control) number → (real)
Overall loss weight if there are multiple losses in the network.
Default: 1.0
GenParamName (input_control) attribute.name(-array) → (string)
Generic input parameter names.
Default: []
List of values: 'is_inference_output'
GenParamValue (input_control) attribute.value(-array) → (string / integer / real)
Generic input parameter values.
Default: []
Suggested values: 'true', 'false'
DLLayerLossCrossEntropy (output_control) dl_layer → (handle)
Cross entropy loss layer.
Deep Learning Professional