create_dl_layer_convolutionT_create_dl_layer_convolutionCreateDlLayerConvolutionCreateDlLayerConvolutioncreate_dl_layer_convolution (Operator)

Name

create_dl_layer_convolutionT_create_dl_layer_convolutionCreateDlLayerConvolutionCreateDlLayerConvolutioncreate_dl_layer_convolution — Create a convolutional layer.

Signature

Description

The operator create_dl_layer_convolutioncreate_dl_layer_convolutionCreateDlLayerConvolutionCreateDlLayerConvolutionCreateDlLayerConvolutioncreate_dl_layer_convolution creates a convolutional layer with NumKernelNumKernelNumKernelNumKernelnumKernelnum_kernel kernels in GroupsGroupsGroupsGroupsgroupsgroups filter groups whose handle is returned in DLLayerConvolutionDLLayerConvolutionDLLayerConvolutionDLLayerConvolutionDLLayerConvolutiondllayer_convolution.

The parameter DLLayerInputDLLayerInputDLLayerInputDLLayerInputDLLayerInputdllayer_input determines the feeding input layer and expects the layer handle as value.

The parameter LayerNameLayerNameLayerNameLayerNamelayerNamelayer_name sets an individual layer name. Note that if creating a model using create_dl_modelcreate_dl_modelCreateDlModelCreateDlModelCreateDlModelcreate_dl_model each layer of the created network must have a unique name.

The parameter KernelSizeKernelSizeKernelSizeKernelSizekernelSizekernel_size specifies the filter kernel in the dimensions width and height.

The parameter DilationDilationDilationDilationdilationdilation specifies the factor of filter dilation in the dimensions width and height.

The parameter StrideStrideStrideStridestridestride specifies how the filter is shifted.

The values for KernelSizeKernelSizeKernelSizeKernelSizekernelSizekernel_size, DilationDilationDilationDilationdilationdilation, and StrideStrideStrideStridestridestride can be set as

• a single value which is used for both dimensions

• a tuple [width, height] and [column, row], respectively.

The parameter GroupsGroupsGroupsGroupsgroupsgroups specifies the number of filter groups.

The parameter NumKernelNumKernelNumKernelNumKernelnumKernelnum_kernel specifies the number of filter kernels. NumKernelNumKernelNumKernelNumKernelnumKernelnum_kernel must be a multiple of GroupsGroupsGroupsGroupsgroupsgroups.

The parameter PaddingPaddingPaddingPaddingpaddingpadding determines the padding, thus how many pixels with value 0 are appended on the border of the processed input image. Supported values are:

• 'half_kernel_size'"half_kernel_size""half_kernel_size""half_kernel_size""half_kernel_size""half_kernel_size": The number of appended pixels depends on the specified KernelSizeKernelSizeKernelSizeKernelSizekernelSizekernel_size. More precisely, it is calculated as , where for the padding on the left / right border the value of KernelSizeKernelSizeKernelSizeKernelSizekernelSizekernel_size in dimension width is regarded and for the padding on the upper / lower border the value of KernelSizeKernelSizeKernelSizeKernelSizekernelSizekernel_size in height.

• 'none'"none""none""none""none""none": No pixels are appended.

• Number of pixels: Specify the number of pixels appended on each border. To do so, the following tuple lengths are supported:

  • Single number: Padding in all four directions left/right/top/bottom.

  • Two numbers: Padding in left/right and top/bottom: [l/r, t/b].

  • Four numbers: Padding on left, right, top, bottom side: [l,r,t,b].

    Restriction: 'runtime'"runtime""runtime""runtime""runtime""runtime" 'gpu'"gpu""gpu""gpu""gpu""gpu" does not support asymmetric padding, i.e., the padding values for the left and right side must be equal, as well as the padding values for the top and bottom side.

  Restriction: The integer padding values must be smaller than the value set for KernelSizeKernelSizeKernelSizeKernelSizekernelSizekernel_size in the corresponding dimension.

The output dimensions of the convolution layer are given by Thereby we use the following values: : output width/height, : input width/height, : number of pixels added to the left/top of the input image, and : number of pixels added to the right/bottom of the input image.

The parameter ActivationActivationActivationActivationactivationactivation determines whether an activation is performed after the convolution in order to optimize the runtime performance. The following values are supported:

• 'relu'"relu""relu""relu""relu""relu": perform a ReLU activation after the convolution.

  It is possible to specify an upper bound to the ReLU operation (see create_dl_layer_activationcreate_dl_layer_activationCreateDlLayerActivationCreateDlLayerActivationCreateDlLayerActivationcreate_dl_layer_activation) via the generic parameter 'upper_bound'"upper_bound""upper_bound""upper_bound""upper_bound""upper_bound". Note: It is not possible to specify a leaky ReLU.

• 'none'"none""none""none""none""none": no activation operation is performed.

We refer to the “Solution Guide on Classification” for more general information about the convolution layer and the reference given below for more detailed information about the arithmetic of the layer.

The following generic parameters GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name and the corresponding values GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value are supported:

'bias_filler'"bias_filler""bias_filler""bias_filler""bias_filler""bias_filler":

See 'weight_filler'"weight_filler""weight_filler""weight_filler""weight_filler""weight_filler" for an explanation of the values.

List of values: 'xavier'"xavier""xavier""xavier""xavier""xavier", 'msra'"msra""msra""msra""msra""msra", 'const'"const""const""const""const""const".

Default: 'const'"const""const""const""const""const"

'bias_filler_variance_norm'"bias_filler_variance_norm""bias_filler_variance_norm""bias_filler_variance_norm""bias_filler_variance_norm""bias_filler_variance_norm":

See 'weight_filler_variance_norm'"weight_filler_variance_norm""weight_filler_variance_norm""weight_filler_variance_norm""weight_filler_variance_norm""weight_filler_variance_norm" for an explanation of the values.

List of values: 'norm_in'"norm_in""norm_in""norm_in""norm_in""norm_in", 'norm_out'"norm_out""norm_out""norm_out""norm_out""norm_out", 'norm_average'"norm_average""norm_average""norm_average""norm_average""norm_average".

Default: 'norm_out'"norm_out""norm_out""norm_out""norm_out""norm_out"

'bias_filler_const_val'"bias_filler_const_val""bias_filler_const_val""bias_filler_const_val""bias_filler_const_val""bias_filler_const_val":

Specifies the constant bias term initialization value if 'bias_filler'"bias_filler""bias_filler""bias_filler""bias_filler""bias_filler" has been set to 'const'"const""const""const""const""const".

Restriction: Ignored for other values of 'bias_filler'"bias_filler""bias_filler""bias_filler""bias_filler""bias_filler".

Default: 0

'bias_term'"bias_term""bias_term""bias_term""bias_term""bias_term":

Determines whether the created convolutional layer has a bias term ('true'"true""true""true""true""true") or not ('false'"false""false""false""false""false").

Default: 'true'"true""true""true""true""true"

'is_inference_output'"is_inference_output""is_inference_output""is_inference_output""is_inference_output""is_inference_output":

Determines whether apply_dl_modelapply_dl_modelApplyDlModelApplyDlModelApplyDlModelapply_dl_model will include the output of this layer in the dictionary DLResultBatchDLResultBatchDLResultBatchDLResultBatchDLResultBatchdlresult_batch even without specifying this layer in OutputsOutputsOutputsOutputsoutputsoutputs ('true'"true""true""true""true""true") or not ('false'"false""false""false""false""false").

Default: 'false'"false""false""false""false""false"

'learning_rate_multiplier'"learning_rate_multiplier""learning_rate_multiplier""learning_rate_multiplier""learning_rate_multiplier""learning_rate_multiplier":

Multiplier for the learning rate for this layer that is used during training. If 'learning_rate_multiplier'"learning_rate_multiplier""learning_rate_multiplier""learning_rate_multiplier""learning_rate_multiplier""learning_rate_multiplier" is set to 0.0, the layer is skipped during training.

Default: 1.0

'learning_rate_multiplier_bias'"learning_rate_multiplier_bias""learning_rate_multiplier_bias""learning_rate_multiplier_bias""learning_rate_multiplier_bias""learning_rate_multiplier_bias":

Multiplier for the learning rate of the bias term. The total bias learning rate is the product of 'learning_rate_multiplier_bias'"learning_rate_multiplier_bias""learning_rate_multiplier_bias""learning_rate_multiplier_bias""learning_rate_multiplier_bias""learning_rate_multiplier_bias" and 'learning_rate_multiplier'"learning_rate_multiplier""learning_rate_multiplier""learning_rate_multiplier""learning_rate_multiplier""learning_rate_multiplier".

Default: 1.0

'upper_bound'"upper_bound""upper_bound""upper_bound""upper_bound""upper_bound":

Float value, which defines the upper bound for ReLU. To unset the upper bound, set 'upper_bound'"upper_bound""upper_bound""upper_bound""upper_bound""upper_bound" to an empty tuple.

Default: []

'weight_filler'"weight_filler""weight_filler""weight_filler""weight_filler""weight_filler":

This parameter defines the mode how the weights are initialized. The following values are supported:

• 'const'"const""const""const""const""const": The weights are filled with constant values.

• 'msra'"msra""msra""msra""msra""msra": The weights are drawn from a Gaussian distribution.

• 'xavier'"xavier""xavier""xavier""xavier""xavier": The weights are drawn from a uniform distribution.

Default: 'xavier'"xavier""xavier""xavier""xavier""xavier"

'weight_filler_const_val'"weight_filler_const_val""weight_filler_const_val""weight_filler_const_val""weight_filler_const_val""weight_filler_const_val":

Specifies the constant weight initialization value.

Restriction: Only applied if 'weight_filler'"weight_filler""weight_filler""weight_filler""weight_filler""weight_filler" = 'const'"const""const""const""const""const".

Default: 0.5

'weight_filler_variance_norm'"weight_filler_variance_norm""weight_filler_variance_norm""weight_filler_variance_norm""weight_filler_variance_norm""weight_filler_variance_norm":

This parameter determines the value range for 'weight_filler'"weight_filler""weight_filler""weight_filler""weight_filler""weight_filler". The following values are supported:

• 'norm_average'"norm_average""norm_average""norm_average""norm_average""norm_average": the values are based on the average of the input and output size

• 'norm_in'"norm_in""norm_in""norm_in""norm_in""norm_in": the values are based on the input size

• 'norm_out'"norm_out""norm_out""norm_out""norm_out""norm_out": the values are based on the output size.

Default: 'norm_in'"norm_in""norm_in""norm_in""norm_in""norm_in"

Certain parameters of layers created using create_dl_layer_convolutioncreate_dl_layer_convolutionCreateDlLayerConvolutionCreateDlLayerConvolutionCreateDlLayerConvolutioncreate_dl_layer_convolution can be set and retrieved using further operators. The following tables give an overview, which parameters can be set using set_dl_model_layer_paramset_dl_model_layer_paramSetDlModelLayerParamSetDlModelLayerParamSetDlModelLayerParamset_dl_model_layer_param and which ones can be retrieved using get_dl_model_layer_paramget_dl_model_layer_paramGetDlModelLayerParamGetDlModelLayerParamGetDlModelLayerParamget_dl_model_layer_param or get_dl_layer_paramget_dl_layer_paramGetDlLayerParamGetDlLayerParamGetDlLayerParamget_dl_layer_param. Note, the operators set_dl_model_layer_paramset_dl_model_layer_paramSetDlModelLayerParamSetDlModelLayerParamSetDlModelLayerParamset_dl_model_layer_param and get_dl_model_layer_paramget_dl_model_layer_paramGetDlModelLayerParamGetDlModelLayerParamGetDlModelLayerParamget_dl_model_layer_param require a model created by create_dl_modelcreate_dl_modelCreateDlModelCreateDlModelCreateDlModelcreate_dl_model.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

References

V. Dumoulin, F. Visin: "A guide to convolution arithmetic for deep learning", 2018, http://arxiv.org/abs/1603.07285

Module

Deep Learning Training