BatchNormTrainingBackward#

BatchNormTrainingBackward operation calculated the gradients of input tensors.

Operation Attributes#

Attribute

Name

Description

Value Type

Supported

Values

Required or

Optional

epsilon

A number to be added to the variance to avoid division by zero

f32

A positive f32 value

Required

data_format

Controls how to interpret the shape of src and dst.

string

NCX, NXC (default)

Optional

Execution Arguments#

The inputs and outputs must be provided according to the below index order when constructing an operation.

Inputs#

Index

Argument Name

Required or Optional

0

src

Required

1

diff_dst

Required

2

mean

Required

3

variance

Required

4

gamma (\(\sigma^2\))

Optional

Outputs#

Index

Argument Name

Required or Optional

0

diff_src

Required

1

diff_gamma

Optional

2

diff_beta

Optional

@note diff_gamma and diff_beta should be either both provided or neither provided. If neither provided, the input gamma will be ignored.

Supported Data Types#

BatchNormTrainingBackward operation supports the following data type combinations.

Src / Diff_dst / Dst

Mean / Variance / Gamma / Diff_gamma / Diff_beta

f32

f32

bf16

f32, bf16

f16

f32