Convolution and Deconvolution

The convolution and deconvolution primitives compute forward, backward, or weight update for a batched convolution or deconvolution operations on 1D, 2D, or 3D spatial data with bias.

The operations are defined by the following formulas. We show formulas only for 2D spatial data which are straightforward to generalize to cases of higher and lower dimensions. Variable names follow the standard Conventions.

Forward

Let \(\src\), \(\weights\) and \(\dst\) be \(N \times IC \times IH \times IW\), \(OC \times IC \times KH \times KW\), and \(N \times OC \times OH \times OW\) tensors respectively. Let \(\bias\) be a 1D tensor with \(OC\) elements.

Furthermore, let the remaining convolution parameters be:

Parameter

Depth

Height

Width

Comment

Padding: Front, top, and left

\(PD_L\)

\(PH_L\)

\(PW_L\)

In the API padding_l indicates the corresponding vector of paddings (_l in the name stands for left)

Padding: Back, bottom, and right

\(PD_R\)

\(PH_R\)

\(PW_R\)

In the API padding_r indicates the corresponding vector of paddings (_r in the name stands for right)

Stride

\(SD\)

\(SH\)

\(SW\)

Convolution without strides is defined by setting the stride parameters to 1

Dilation

\(DD\)

\(DH\)

\(DW\)

Non-dilated convolution is defined by setting the dilation parameters to 0

The following formulas show how oneDNN computes convolutions. They are broken down into several types to simplify the exposition, but in reality the convolution types can be combined.

To further simplify the formulas, we assume that \(\src(n, ic, ih, iw) = 0\) if \(ih < 0\), or \(ih \geq IH\), or \(iw < 0\), or \(iw \geq IW\).

Regular Convolution

\[\begin{split}\dst(n, oc, oh, ow) & = \bias(oc) \\ + \sum_{ic=0}^{IC-1} \sum_{kh=0}^{KH-1} \sum_{kw=0}^{KW-1} & \src(n, ic, oh', ow') \cdot \weights(oc, ic, kh, kw).\end{split}\]

Here:

  • \(oh' = oh \cdot SH + kh - PH_L\),

  • \(ow' = ow \cdot SW + kw - PW_L\),

  • \(OH = \left\lfloor{\frac{IH - KH + PH_L + PH_R}{SH}} \right\rfloor + 1\),

  • \(OW = \left\lfloor{\frac{IW - KW + PW_L + PW_R}{SW}} \right\rfloor + 1\).

Convolution with Groups

oneDNN adds a separate groups dimension to memory objects representing \(\weights\) tensors and represents weights as \(G \times OC_G \times IC_G \times KH \times KW\) 5D tensors for 2D convolutions with groups.

\[\begin{split}\dst(n, g \cdot OC_G + oc_g, oh, ow) & = \bias(g \cdot OC_G + oc_g) \\ + \sum_{ic_g=0}^{IC_G-1} \sum_{kh=0}^{KH-1} \sum_{kw=0}^{KW-1} & \src(n, g \cdot IC_G + ic_g, oh', ow') \cdot \weights(g, oc_g, ic_g, kh, kw),\end{split}\]

where

  • \(IC_G = \frac{IC}{G}\),

  • \(OC_G = \frac{OC}{G}\), and

  • \(oc_g \in [0, OC_G)\).

The case when \(OC_G = IC_G = 1\) is also known as a depthwise convolution.

Convolution with Dilation

\[\begin{split}\dst(n, oc, oh, ow) & = \bias(oc) + \\ + \sum_{ic=0}^{IC-1} \sum_{kh=0}^{KH-1} \sum_{kw=0}^{KW-1} & \src(n, ic, oh'', ow'') \cdot \weights(oc, ic, kh, kw).\end{split}\]

Here:

  • \(oh'' = oh \cdot SH + kh \cdot (DH + 1) - PH_L\),

  • \(ow'' = ow \cdot SW + kw \cdot (DW + 1) - PW_L\),

  • \(OH = \left\lfloor{\frac{IH - DKH + PH_L + PH_R}{SH}} \right\rfloor + 1,\) where \(DKH = 1 + (KH - 1) \cdot (DH + 1)\), and

  • \(OW = \left\lfloor{\frac{IW - DKW + PW_L + PW_R}{SW}} \right\rfloor + 1,\) where \(DKW = 1 + (KW - 1) \cdot (DW + 1)\).

Deconvolution (Transposed Convolution)

Deconvolutions (also called fractionally-strided convolutions or transposed convolutions) can be defined by swapping the forward and backward passes of a convolution. One way to put it is to note that the weights define a convolution, but whether it is a direct convolution or a transposed convolution is determined by how the forward and backward passes are computed.

Difference Between Forward Training and Forward Inference

There is no difference between the forward_training and forward_inference propagation kinds.

Backward

The backward propagation computes \(\diffsrc\) based on \(\diffdst\) and \(\weights\).

The weights update computes \(\diffweights\) and \(\diffbias\) based on \(\diffdst\) and \(\src\).

Note

The optimized memory formats \(\src\) and \(\weights\) might be different on forward propagation, backward propagation, and weights update.

Execution Arguments

When executed, the inputs and outputs should be mapped to an execution argument index as specified by the following table.

Primitive input/output

Execution argument index

\(\src\)

DNNL_ARG_SRC

\(\weights\)

DNNL_ARG_WEIGHTS

\(\bias\)

DNNL_ARG_BIAS

\(\dst\)

DNNL_ARG_DST

\(\diffsrc\)

DNNL_ARG_DIFF_SRC

\(\diffweights\)

DNNL_ARG_DIFF_WEIGHTS

\(\diffbias\)

DNNL_ARG_DIFF_BIAS

\(\diffdst\)

DNNL_ARG_DIFF_DST

Operation Details

N/A

Data Types Support

Convolution primitive supports the following combination of data types for source, destination, and weights memory objects.

Note

Here we abbreviate data types names for readability. For example, dnnl::memory::data_type::f32 is abbreviated to f32.

Propagation

Source

Weights

Destination

Bias

forward / backward

f32

f32

f32

f32

forward

f16

f16

f16

f16

forward

u8, s8

s8

u8, s8, s32, f32

u8, s8, s32, f32

forward

bf16

bf16

f32, bf16

f32, bf16

backward

f32, bf16

bf16

bf16

weights update

bf16

f32, bf16

bf16

f32, bf16

Data Representation

Like other CNN primitives, the convolution primitive expects the following tensors:

Spatial

Source / Destination

Weights

1D

\(N \times C \times W\)

\([G \times ] OC \times IC \times KW\)

2D

\(N \times C \times H \times W\)

\([G \times ] OC \times IC \times KH \times KW\)

3D

\(N \times C \times D \times H \times W\)

\([G \times ] OC \times IC \times KD \times KH \times KW\)

Memory format of data and weights memory objects is critical for convolution primitive performance. In the oneDNN programming model, convolution is one of the few primitives that support the placeholder memory format tag any and can define data and weight memory objects format based on the primitive parameters. When using any it is necessary to first create a convolution primitive descriptor and then query it for the actual data and weight memory objects formats.

While convolution primitives can be created with memory formats specified explicitly, the performance is likely to be suboptimal.

The table below shows the combinations for which plain memory formats the convolution primitive is optimized for.

Spatial
Convolution Type
Data /
Weights logical tensor
Implementation optimized
for memory formats

1D, 2D, 3D

any

optimized

1D

f32, bf16

NCW / OIW, GOIW

ncw (abc) / oiw (abc), goiw (abcd)

1D

f32, bf16

NCW / OIW, GOIW

nwc (acb) / wio (cba), wigo (dcab)

1D

int8

NCW / OIW

nwc (acb) / wio (cba)

2D

f32, bf16

NCHW / OIHW, GOIHW

nchw (abcd) / oihw (abcd), goihw (abcde)

2D

f32, bf16

NCHW / OIHW, GOIHW

nhwc (acdb) / hwio (cdba), hwigo (decab)

2D

int8

NCHW / OIHW, GOIHW

nhwc (acdb) / hwio (cdba), hwigo (decab)

3D

f32, bf16

NCDHW / OIDHW, GOIDHW

ncdhw (abcde) / oidhw (abcde), goidhw (abcdef)

3D

f32, bf16

NCDHW / OIDHW, GOIDHW

ndhwc (acdeb) / dhwio (cdeba), dhwigo (defcab)

3D

int8

NCDHW / OIDHW

ndhwc (acdeb) / dhwio (cdeba)

Post-ops and Attributes

Post-ops and attributes enable you to modify the behavior of the convolution primitive by applying the output scale to the result of the primitive and by chaining certain operations after the primitive. The following attributes and post-ops are supported:

Propagation

Type

Operation

Description

Restrictions

forward

attribute

Output scale

Scales the result of convolution by given scale factor(s)

int8 convolutions only

forward

post-op

Eltwise

Applies an elementwise operation to the result

forward

post-op

Sum

Adds the operation result to the destination tensor instead of overwriting it

The primitive supports dynamic quantization via run-time output scales. That means a user could configure attributes with output scales set to the DNNL_RUNTIME_F32_VAL wildcard value instead of the actual scales, if the scales are not known at the primitive descriptor creation stage. In this case, the user must provide the scales as an additional input memory object with argument DNNL_ARG_ATTR_OUTPUT_SCALES during the execution stage.

Note

The library does not prevent using post-ops in training, but note that not all post-ops are feasible for training usage. For instance, using ReLU with non-zero negative slope parameter as a post-op would not produce an additional output workspace that is required to compute backward propagation correctly. Hence, in this particular case one should use separate convolution and eltwise primitives for training.

The following post-ops chaining should be supported by the library:

Type of convolutions

Post-ops sequence supported

f32 and bf16 convolution

eltwise, sum, sum -> eltwise

int8 convolution

eltwise, sum, sum -> eltwise, eltwise -> sum

The attributes and post-ops take effect in the following sequence:

  • Output scale attribute,

  • Post-ops, in order they were attached.

The operations during attributes and post-ops applying are done in single precision floating point data type. The conversion to the actual destination data type happens just before the actual storing.

Example 1

Consider the following pseudo code:

attribute attr;
attr.set_output_scale(alpha);
attr.set_post_ops({
        { sum={scale=beta} },
        { eltwise={scale=gamma, type=tanh, alpha=ignore, beta=ignored }
    });

convolution_forward(src, weights, dst, attr)

The would lead to the following:

\[\dst(\overline{x}) = \gamma \cdot \tanh \left( \alpha \cdot conv(\src, \weights) + \beta \cdot \dst(\overline{x}) \right)\]

Example 2

The following pseudo code:

attribute attr;
attr.set_output_scale(alpha);
attr.set_post_ops({
        { eltwise={scale=gamma, type=relu, alpha=eta, beta=ignored }
        { sum={scale=beta} },
    });

convolution_forward(src, weights, dst, attr)

That would lead to the following:

\[\dst(\overline{x}) = \beta \cdot \dst(\overline{x}) + \gamma \cdot ReLU \left( \alpha \cdot conv(\src, \weights), \eta \right)\]

Algorithms

oneDNN implementations may implement convolution primitives using several different algorithms which can be chosen by the user.

API

struct dnnl::convolution_forward : public dnnl::primitive

Convolution forward propagation primitive.

Public Functions

convolution_forward()

Default constructor. Produces an empty object.

convolution_forward(const primitive_desc &pd)

Constructs a convolution forward propagation primitive.

Parameters

pd – Primitive descriptor for a convolution forward propagation primitive.

struct desc

Descriptor for a convolution forward propagation primitive.

Public Functions

desc(prop_kind aprop_kind, algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &weights_desc, const memory::desc &bias_desc, const memory::desc &dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a convolution forward propagation primitive with bias.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
desc(prop_kind aprop_kind, algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &weights_desc, const memory::desc &dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a convolution forward propagation primitive without bias.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
desc(prop_kind aprop_kind, algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &weights_desc, const memory::desc &bias_desc, const memory::desc &dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a dilated convolution forward propagation primitive with bias.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aprop_kind – Propagation kind. Possible values are dnnl::prop_kind::forward_training, and dnnl::prop_kind::forward_inference.

  • aalgorithm – Convolution algorithm. Possible values are dnnl::algorithm::convolution_direct, dnnl::algorithm::convolution_winograd, and dnnl::algorithm::convolution_auto.

  • src_desc – Source memory descriptor.

  • weights_desc – Weights memory descriptor.

  • bias_desc – Bias memory descriptor. Passing zero memory descriptor disables the bias term.

  • dst_desc – Destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • dilates – Dilations for each spatial dimension. A zero value means no dilation in the corresponding dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(prop_kind aprop_kind, algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &weights_desc, const memory::desc &dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a dilated convolution forward propagation primitive without bias.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
struct primitive_desc : public dnnl::primitive_desc

Primitive descriptor for a convolution forward propagation primitive.

Public Functions

primitive_desc()

Default constructor. Produces an empty object.

primitive_desc(const desc &adesc, const engine &aengine, bool allow_empty = false)

Constructs a primitive descriptor for a convolution forward propagation primitive.

Parameters
  • adesc – Descriptor for a convolution forward propagation primitive.

  • aengine – Engine to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

primitive_desc(const desc &adesc, const primitive_attr &attr, const engine &aengine, bool allow_empty = false)

Constructs a primitive descriptor for a convolution forward propagation primitive.

Parameters
  • adesc – Descriptor for a convolution forward propagation primitive.

  • aengine – Engine to use.

  • attr – Primitive attributes to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

memory::desc src_desc() const

Returns a source memory descriptor.

Returns

Source memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a source parameter.

memory::desc weights_desc() const

Returns a weights memory descriptor.

Returns

Weights memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a weights parameter.

memory::desc dst_desc() const

Returns a destination memory descriptor.

Returns

Destination memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a destination parameter.

memory::desc bias_desc() const

Returns the bias memory descriptor.

Returns

The bias memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a bias parameter.

struct dnnl::convolution_backward_data : public dnnl::primitive

Convolution backward propagation primitive.

Public Functions

convolution_backward_data()

Default constructor. Produces an empty object.

convolution_backward_data(const primitive_desc &pd)

Constructs a convolution backward propagation primitive.

Parameters

pd – Primitive descriptor for a convolution backward propagation primitive.

struct desc

Descriptor for a convolution backward propagation primitive.

Public Functions

desc(algorithm aalgorithm, const memory::desc &diff_src_desc, const memory::desc &weights_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a convolution backward propagation primitive.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Convolution algorithm. Possible values are dnnl::algorithm::convolution_direct, dnnl::algorithm::convolution_winograd, and dnnl::algorithm::convolution_auto.

  • diff_src_desc – Diff source memory descriptor.

  • weights_desc – Weights memory descriptor.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(algorithm aalgorithm, const memory::desc &diff_src_desc, const memory::desc &weights_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for dilated convolution backward propagation primitive.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Convolution algorithm. Possible values are dnnl::algorithm::convolution_direct, dnnl::algorithm::convolution_winograd, and dnnl::algorithm::convolution_auto.

  • diff_src_desc – Diff source memory descriptor.

  • weights_desc – Weights memory descriptor.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • dilates – Dilations for each spatial dimension. A zero value means no dilation in the corresponding dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

struct primitive_desc : public dnnl::primitive_desc

Primitive descriptor for a convolution backward propagation primitive.

Public Functions

primitive_desc()

Default constructor. Produces an empty object.

primitive_desc(const desc &adesc, const engine &aengine, const convolution_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for a convolution backward propagation primitive.

Parameters
  • adesc – Descriptor for a convolution backward propagation primitive.

  • aengine – Engine to perform the operation on.

  • hint_fwd_pd – Primitive descriptor for a convolution forward propagation primitive. It is used as a hint for deciding which memory format to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

primitive_desc(const desc &adesc, const primitive_attr &attr, const engine &aengine, const convolution_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for a convolution backward propagation primitive.

Parameters
  • adesc – Descriptor for a convolution backward propagation primitive.

  • aengine – Engine to perform the operation on.

  • attr – Primitive attributes to use.

  • hint_fwd_pd – Primitive descriptor for a convolution forward propagation primitive. It is used as a hint for deciding which memory format to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

memory::desc diff_src_desc() const

Returns a diff source memory descriptor.

Returns

Diff source memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a diff source memory with.

memory::desc weights_desc() const

Returns a weights memory descriptor.

Returns

Weights memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a weights parameter.

memory::desc diff_dst_desc() const

Returns a diff destination memory descriptor.

Returns

Diff destination memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a diff destination parameter.

struct dnnl::convolution_backward_weights : public dnnl::primitive

Convolution weights gradient primitive.

Public Functions

convolution_backward_weights()

Default constructor. Produces an empty object.

convolution_backward_weights(const primitive_desc &pd)

Constructs a convolution weights gradient primitive.

Parameters

pd – Primitive descriptor for a convolution weights gradient primitive.

struct desc

Descriptor for a convolution weights gradient primitive.

Public Functions

desc(algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &diff_weights_desc, const memory::desc &diff_bias_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a convolution weights gradient primitive with bias.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Convolution algorithm. Possible values are dnnl::algorithm::convolution_direct, dnnl::algorithm::convolution_winograd, and dnnl::algorithm::convolution_auto.

  • src_desc – Source memory descriptor.

  • diff_weights_desc – Diff weights memory descriptor.

  • diff_bias_desc – Diff bias memory descriptor. Passing zero memory descriptor disables the bias term.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &diff_weights_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a convolution weights gradient primitive without bias.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Convolution algorithm. Possible values are dnnl::algorithm::convolution_direct, dnnl::algorithm::convolution_winograd, and dnnl::algorithm::convolution_auto.

  • src_desc – Source memory descriptor.

  • diff_weights_desc – Diff weights memory descriptor.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &diff_weights_desc, const memory::desc &diff_bias_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a dilated convolution weights gradient primitive with bias.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Convolution algorithm. Possible values are dnnl::algorithm::convolution_direct, dnnl::algorithm::convolution_winograd, and dnnl::algorithm::convolution_auto.

  • src_desc – Source memory descriptor.

  • diff_weights_desc – Diff weights memory descriptor.

  • diff_bias_desc – Diff bias memory descriptor. Passing zero memory descriptor disables the bias term.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • dilates – Dilations for each spatial dimension. A zero value means no dilation in the corresponding dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &diff_weights_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a dilated convolution weights gradient primitive without bias.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Convolution algorithm. Possible values are dnnl::algorithm::convolution_direct, dnnl::algorithm::convolution_winograd, and dnnl::algorithm::convolution_auto.

  • src_desc – Source memory descriptor.

  • diff_weights_desc – Diff weights memory descriptor.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • dilates – Dilations for each spatial dimension. A zero value means no dilation in the corresponding dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

struct primitive_desc : public dnnl::primitive_desc

Primitive descriptor for a convolution weights gradient primitive.

Public Functions

primitive_desc()

Default constructor. Produces an empty object.

primitive_desc(const desc &adesc, const engine &aengine, const convolution_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for a convolution weights gradient primitive.

Parameters
  • adesc – Descriptor for a convolution weights gradient primitive.

  • aengine – Engine to use.

  • hint_fwd_pd – Primitive descriptor for a convolution forward propagation primitive. It is used as a hint for deciding which memory format to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

primitive_desc(const desc &adesc, const primitive_attr &attr, const engine &aengine, const convolution_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for a convolution weights gradient primitive.

Parameters
  • adesc – Descriptor for a convolution weights gradient primitive.

  • attr – Primitive attributes to use.

  • aengine – Engine to use.

  • hint_fwd_pd – Primitive descriptor for a convolution forward propagation primitive. It is used as a hint for deciding which memory format to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

memory::desc src_desc() const

Returns a source memory descriptor.

Returns

Source memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a source parameter.

memory::desc diff_weights_desc() const

Returns a diff weights memory descriptor.

Returns

Diff weights memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a diff weights parameter.

memory::desc diff_dst_desc() const

Returns a diff destination memory descriptor.

Returns

Diff destination memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a diff destination parameter.

memory::desc diff_bias_desc() const

Returns the diff bias memory descriptor.

Returns

The diff bias memory descriptor.

Returns

A zero memory descriptor of the primitive does not have a diff bias parameter.

struct dnnl::deconvolution_forward : public dnnl::primitive

Deconvolution forward propagation primitive.

Public Functions

deconvolution_forward()

Default constructor. Produces an empty object.

deconvolution_forward(const primitive_desc &pd)

Constructs a deconvolution forward propagation primitive.

Parameters

pd – Primitive descriptor for a deconvolution forward propagation primitive.

struct desc

Descriptor for a deconvolution forward propagation primitive.

Public Functions

desc(prop_kind aprop_kind, algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &weights_desc, const memory::desc &bias_desc, const memory::desc &dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a deconvolution forward propagation primitive with bias.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aprop_kind – Propagation kind. Possible values are dnnl::prop_kind::forward_training, and dnnl::prop_kind::forward_inference.

  • aalgorithm – Deconvolution algorithm: dnnl::algorithm::deconvolution_direct, and dnnl::algorithm::deconvolution_winograd.

  • src_desc – Source memory descriptor.

  • weights_desc – Weights memory descriptor.

  • bias_desc – Bias memory descriptor. Passing zero memory descriptor disables the bias term.

  • dst_desc – Destination memory descriptor.

  • strides – Vector of strides for spatial dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(prop_kind aprop_kind, algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &weights_desc, const memory::desc &dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a deconvolution forward propagation primitive without bias.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
desc(prop_kind aprop_kind, algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &weights_desc, const memory::desc &bias_desc, const memory::desc &dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a dilated deconvolution forward propagation primitive with bias.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aprop_kind – Propagation kind. Possible values are dnnl::prop_kind::forward_training, and dnnl::prop_kind::forward_inference.

  • aalgorithm – Deconvolution algorithm: dnnl::algorithm::deconvolution_direct, and dnnl::algorithm::deconvolution_winograd.

  • src_desc – Source memory descriptor.

  • weights_desc – Weights memory descriptor.

  • bias_desc – Bias memory descriptor. Passing zero memory descriptor disables the bias term.

  • dst_desc – Destination memory descriptor.

  • strides – Vector of strides for spatial dimension.

  • dilates – Dilations for each spatial dimension. A zero value means no dilation in the corresponding dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(prop_kind aprop_kind, algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &weights_desc, const memory::desc &dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a dilated deconvolution forward propagation primitive without bias.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aprop_kind – Propagation kind. Possible values are dnnl::prop_kind::forward_training, and dnnl::prop_kind::forward_inference.

  • aalgorithm – Deconvolution algorithm: dnnl::algorithm::deconvolution_direct, and dnnl::algorithm::deconvolution_winograd.

  • src_desc – Source memory descriptor.

  • weights_desc – Weights memory descriptor.

  • dst_desc – Destination memory descriptor.

  • strides – Vector of strides for spatial dimension.

  • dilates – Dilations for each spatial dimension. A zero value means no dilation in the corresponding dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

struct primitive_desc : public dnnl::primitive_desc

Primitive descriptor for a deconvolution forward propagation primitive.

Public Functions

primitive_desc()

Default constructor. Produces an empty object.

primitive_desc(const desc &adesc, const engine &aengine, bool allow_empty = false)

Constructs a primitive descriptor for a deconvolution forward propagation primitive.

Parameters
  • adesc – Descriptor for a deconvolution forward propagation primitive.

  • aengine – Engine to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

primitive_desc(const desc &adesc, const primitive_attr &attr, const engine &aengine, bool allow_empty = false)

Constructs a primitive descriptor for a deconvolution forward propagation primitive.

Parameters
  • adesc – Descriptor for a deconvolution forward propagation primitive.

  • aengine – Engine to use.

  • attr – Primitive attributes to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

memory::desc src_desc() const

Returns a source memory descriptor.

Returns

Source memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a source parameter.

memory::desc weights_desc() const

Returns a weights memory descriptor.

Returns

Weights memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a weights parameter.

memory::desc dst_desc() const

Returns a destination memory descriptor.

Returns

Destination memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a destination parameter.

memory::desc bias_desc() const

Returns the bias memory descriptor.

Returns

The bias memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a bias parameter.

struct dnnl::deconvolution_backward_data : public dnnl::primitive

Deconvolution backward propagation primitive.

Public Functions

deconvolution_backward_data()

Default constructor. Produces an empty object.

deconvolution_backward_data(const primitive_desc &pd)

Constructs a deconvolution backward propagation primitive.

Parameters

pd – Primitive descriptor for a deconvolution backward propagation primitive.

struct desc

Descriptor for a deconvolution backward propagation primitive.

Public Functions

desc(algorithm aalgorithm, const memory::desc &diff_src_desc, const memory::desc &weights_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a deconvolution backward propagation primitive.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Deconvolution algorithm (dnnl::algorithm::convolution_direct, dnnl::algorithm::convolution_winograd).

  • diff_src_desc – Diff source memory descriptor.

  • weights_desc – Weights memory descriptor.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(algorithm aalgorithm, const memory::desc &diff_src_desc, const memory::desc &weights_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a dilated deconvolution backward propagation primitive.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Deconvolution algorithm (dnnl::algorithm::convolution_direct, dnnl::algorithm::convolution_winograd).

  • diff_src_desc – Diff source memory descriptor.

  • weights_desc – Weights memory descriptor.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • dilates – Dilations for each spatial dimension. A zero value means no dilation in the corresponding dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

struct primitive_desc : public dnnl::primitive_desc

Primitive descriptor for a deconvolution backward propagation primitive.

Public Functions

primitive_desc()

Default constructor. Produces an empty object.

primitive_desc(const desc &adesc, const engine &aengine, const deconvolution_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for a deconvolution backward propagation primitive.

Parameters
  • adesc – Descriptor for a deconvolution backward propagation primitive.

  • aengine – Engine to use.

  • hint_fwd_pd – Primitive descriptor for a deconvolution forward propagation primitive. It is used as a hint for deciding which memory format to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

primitive_desc(const desc &adesc, const primitive_attr &attr, const engine &aengine, const deconvolution_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for a deconvolution backward propagation primitive.

Parameters
  • adesc – Descriptor for a deconvolution backward propagation primitive.

  • attr – Primitive attributes to use.

  • aengine – Engine to use.

  • hint_fwd_pd – Primitive descriptor for a deconvolution forward propagation primitive. It is used as a hint for deciding which memory format to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

memory::desc diff_src_desc() const

Returns a diff source memory descriptor.

Returns

Diff source memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a diff source memory with.

memory::desc weights_desc() const

Returns a weights memory descriptor.

Returns

Weights memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a weights parameter.

memory::desc diff_dst_desc() const

Returns a diff destination memory descriptor.

Returns

Diff destination memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a diff destination parameter.

struct dnnl::deconvolution_backward_weights : public dnnl::primitive

Deconvolution weights gradient primitive.

Public Functions

deconvolution_backward_weights()

Default constructor. Produces an empty object.

deconvolution_backward_weights(const primitive_desc &pd)

Constructs a deconvolution weights gradient primitive.

Parameters

pd – Primitive descriptor for a deconvolution weights gradient primitive.

struct desc

Descriptor for a deconvolution weights gradient primitive.

Public Functions

desc(algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &diff_weights_desc, const memory::desc &diff_bias_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a deconvolution weights gradient primitive with bias.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Deconvolution algorithm. Possible values are dnnl::algorithm::deconvolution_direct, and dnnl::algorithm::deconvolution_winograd.

  • src_desc – Source memory descriptor.

  • diff_weights_desc – Diff weights memory descriptor.

  • diff_bias_desc – Diff bias memory descriptor. Passing zero memory descriptor disables the bias term.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &diff_weights_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a deconvolution weights gradient primitive without bias.

Arrays strides, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Deconvolution algorithm. Possible values are dnnl::algorithm::deconvolution_direct, and dnnl::algorithm::deconvolution_winograd.

  • src_desc – Source memory descriptor.

  • diff_weights_desc – Diff weights memory descriptor.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &diff_weights_desc, const memory::desc &diff_bias_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a dilated deconvolution weights gradient primitive with bias.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Deconvolution algorithm. Possible values are dnnl::algorithm::deconvolution_direct, and dnnl::algorithm::deconvolution_winograd.

  • src_desc – Source memory descriptor.

  • diff_weights_desc – Diff weights memory descriptor.

  • diff_bias_desc – Diff bias memory descriptor. Passing zero memory descriptor disables the bias term.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • dilates – Dilations for each spatial dimension. A zero value means no dilation in the corresponding dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

desc(algorithm aalgorithm, const memory::desc &src_desc, const memory::desc &diff_weights_desc, const memory::desc &diff_dst_desc, const memory::dims &strides, const memory::dims &dilates, const memory::dims &padding_l, const memory::dims &padding_r)

Constructs a descriptor for a dilated deconvolution weights gradient primitive without bias.

Arrays strides, dilates, padding_l, and padding_r contain values for spatial dimensions only and hence must have the same number of elements as there are spatial dimensions. The order of values is the same as in the tensor: depth (for 3D tensors), height (for 3D and 2D tensors), and width.

Note

All the memory descriptors may be initialized with the dnnl::memory::format_tag::any value of format_tag.

Parameters
  • aalgorithm – Deconvolution algorithm. Possible values are dnnl::algorithm::deconvolution_direct, and dnnl::algorithm::deconvolution_winograd.

  • src_desc – Source memory descriptor.

  • diff_weights_desc – Diff weights memory descriptor.

  • diff_dst_desc – Diff destination memory descriptor.

  • strides – Strides for each spatial dimension.

  • dilates – Dilations for each spatial dimension. A zero value means no dilation in the corresponding dimension.

  • padding_l – Vector of padding values for low indices for each spatial dimension ([[front,] top,] left).

  • padding_r – Vector of padding values for high indices for each spatial dimension ([[back,] bottom,] right).

struct primitive_desc : public dnnl::primitive_desc

Primitive descriptor for a deconvolution weights gradient primitive.

Public Functions

primitive_desc()

Default constructor. Produces an empty object.

primitive_desc(const desc &adesc, const engine &aengine, const deconvolution_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for a deconvolution weights update primitive.

Parameters
  • adesc – Descriptor for a deconvolution weights gradient primitive.

  • aengine – Engine to use.

  • hint_fwd_pd – Primitive descriptor for a deconvolution forward propagation primitive. It is used as a hint for deciding which memory format to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

primitive_desc(const desc &adesc, const primitive_attr &attr, const engine &aengine, const deconvolution_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for a deconvolution weights update primitive.

Parameters
  • adesc – Descriptor for a deconvolution weights gradient primitive.

  • attr – Primitive attributes to use.

  • aengine – Engine to use.

  • hint_fwd_pd – Primitive descriptor for a deconvolution forward propagation primitive. It is used as a hint for deciding which memory format to use.

  • allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

memory::desc src_desc() const

Returns a source memory descriptor.

Returns

Source memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a source parameter.

memory::desc diff_weights_desc() const

Returns a diff weights memory descriptor.

Returns

Diff weights memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a diff weights parameter.

memory::desc diff_dst_desc() const

Returns a diff destination memory descriptor.

Returns

Diff destination memory descriptor.

Returns

A zero memory descriptor if the primitive does not have a diff destination parameter.

memory::desc diff_bias_desc() const

Returns the diff bias memory descriptor.

Returns

The diff bias memory descriptor.

Returns

A zero memory descriptor of the primitive does not have a diff bias parameter.