.. SPDX-FileCopyrightText: 2019-2020 Intel Corporation
..
.. SPDX-License-Identifier: CC-BY-4.0
.. default-domain:: cpp
.. include:: ../../replacements.inc.rst
.. _post_ops-label:
########
Post-ops
########
*Post-ops* are operations that are appended after a primitive. They are
implemented using the :ref:`attributes-link` mechanism. If there are multiple
post-ops, they are executed in the order they have been appended as follow:
.. math::
\dst = po[n](po[n-1] (...(po[0](OP()))))
.. note::
Post-ops does not preserve intermediate data during
computation. This typically makes them suitable for inference only.
The post-ops are represented by |post_ops| which is copied once it
is attached to the attributes using |primitive_attr::set_post_ops|
function. The attributes then need to be passed to a primitive descriptor
creation function to take effect. Below is a simple sketch:
.. code:: cpp
dnnl::post_ops po; // default empty post-ops
assert(po.len() == 0); // no post-ops attached
po.append_SOMETHING(params); // append some particular post-op
po.append_SOMETHING_ELSE(other_params); // append one more post-op
// (!) Note that the order in which post-ops are appended matters!
assert(po.len() == 2);
dnnl::primitive_attr attr; // default attributes
attr.set_post_ops(po); // attach the post-ops to the attr
// any changes to po after this point don't affect the value stored in attr
primitive::primitive_desc op_pd(params, attr); // create a pd with the attr
.. note::
Different primitives may have different post-ops support. Moreover, the
support might also depend on the actual implementation of a primitive. So
robust code should be able to handle errors accordingly. See the
:ref:`attributes_error_handling-link`.
.. note::
Post-ops do not change memory format of the operation destination memory
object.
The post-op objects can be inspected using the |post_ops::kind|
function that takes an index of the post-op to inspect (that must be less than
the value returned by |post_ops::len|), and returns its kind.
******************
Supported Post-ops
******************
.. _post_ops_eltwise-label:
Eltwise Post-op
===============
The eltwise post-op is appended using |post_ops::append_eltwise|
function. The |post_ops::kind| returns
|primitive::kind::eltwise| for such a post-op.
The eltwise post-op replaces:
.. math::
\dst[:] = \operatorname{Op}(...)
with
.. math::
\dst[:] = scale \cdot \operatorname{eltwise}(\operatorname{Op}(...))
The intermediate result of the :math:`\operatorname{Op}(...)` is not
preserved.
The :math:`scale` factor is supported in :ref:`int8
` inference only. For all other cases the
scale must be `1.0` (default value).
The scale parameter is set to :math:`1.0` by default, and can be set
using the |primitive_attr::set_scales_mask| attribute for the argument
|DNNL_ARG_ATTR_MULTIPLE_POST_OP(po_index)|.
.. _post_ops_sum-label:
Sum Post-op
===========
The sum post-op accumulates the result of a primitive with the existing data
and is appended using |post_ops::append_sum| function. The
|post_ops::kind| returns |primitive::kind::sum| for such
a post-op.
Prior to accumulating the result, the existing value is multiplied by
scale. The :math:`scale` factor is supported in :ref:`int8
` inference only and should be used
only when the result and the existing data have different magnitudes.
For all other cases the scale must be `1.0` (default value).
The scale parameter is set to :math:`1.0` by default, and can be set
using the |primitive_attr::set_scales_mask| attribute for the argument
|DNNL_ARG_ATTR_MULTIPLE_POST_OP(po_index)|.
Additionally, the sum post-op can reinterpret the destination values as a
different data type of the same size. This may be used to, for example,
reinterpret 8-bit signed data as unsigned or vice versa (which requires that
values fall within a common range to work).
The sum post-op replaces
.. math::
\dst[:] = \operatorname{Op}(...)
with
.. math::
\dst[:] = scale \cdot as_data_type(\dst[:]) + \operatorname{Op}(...)
.. _post_ops_binary-label:
Binary post-ops
============================
The binary post-op replaces:
.. math::
\dst[:] = \operatorname{Op}(...)
with
.. math::
\dst[:] = \operatorname{binary}(\operatorname{Op}(...), scale[:] \cdot Source\_1[:])
The binary post-op supports the same algorithms and broadcast semantic
as the :ref:`binary primitive`.
Furthermore, the binary post-op scale parameter is set to :math:`1.0`
by default, and can be set using the |primitive_attr::set_scales_mask|
attribute for the argument |DNNL_ARG_ATTR_MULTIPLE_POST_OP(po_index)|
| |DNNL_ARG_SRC_1|. For example:
.. code:: cpp
primitive_attr attr;
post_ops p_ops;
p_ops.append_binary(algorithm::binary_add, summand_md);
attr.set_post_ops(p_ops);
attr.set_scales_mask(DNNL_ARG_ATTR_MULTIPLE_POST_OP(0) | DNNL_ARG_SRC_1,
/* mask */ 0);
Examples of Chained Post-ops
============================
Post-ops can be chained together by appending one after another. Note that the
order matters: the post-ops are executed in the order they have been appended.
.. _post_ops_sum_relu-label:
Sum -> ReLU
-----------
This pattern is pretty common for the CNN topologies of the ResNet family.
.. code:: cpp
dnnl::post_ops po;
po.append_sum();
po.append_eltwise(
/* algorithm = */ dnnl::algorithm::eltwise_relu,
/* neg slope = */ 0.f,
/* unused for ReLU */ 0.f);
dnnl::primitive_attr attr;
attr.set_post_ops(po);
convolution_forward::primitive_desc(conv_d, attr, engine);
This will lead to the following computations:
.. math::
\dst[:] = \operatorname{ReLU}(\dst[:] + \operatorname{conv}(\src[:], \weights[:])
***
API
***
.. doxygenstruct:: dnnl::post_ops
:project: oneDNN
:members:
.. vim: ts=3 sw=3 et spell spelllang=en