.. SPDX-FileCopyrightText: 2019-2020 Intel Corporation
..
.. SPDX-License-Identifier: CC-BY-4.0

.. default-domain:: cpp

.. include:: /elements/oneDNN/source/replacements.inc.rst

Local Response Normalization
----------------------------

The LRN primitive performs a forward or backward local response normalization
operation defined by the following formulas. Variable names follow the
standard :ref:`conventions-label`.

Forward
~~~~~~~

LRN `across channels <#dnnl_lrn_across_channels>`__:

.. math::

       \dst(n, c, h, w) =
           \left\{k + \frac{\alpha}{n_{l}}
               \sum\limits_{i=-(n_{l}-1)/2}^{(n_{l}+1)/2-1}
                   (\src(n, c+i, h, w))^2
           \right\}^{-\beta}
           \cdot
           \src(n, c, h, w),

LRN `within channel <#dnnl_lrn_within_channel>`__:

.. math::

       \dst(n, c, h, w) =
           \left\{k + \frac{\alpha}{n_{l}}
               \sum\limits_{i=-(n_{l}-1)/2}^{(n_{l}+1)/2-1}
               \sum\limits_{j=-(n_{l}-1)/2}^{(n_{l}+1)/2-1}
                   (\src(n, c, h+i, w+j))^2
           \right\}^{-\beta}
           \cdot
           \src(n, c, h, w),

where :math:`n_{l}` is the ``local_size``. Formulas are provided for 2D
spatial data case.

Backward
~~~~~~~~

The backward propagation computes :math:`\diffsrc(n, c, h, w)`, based on
:math:`\diffdst(n, c, h, w)` and :math:`\src(n, c, h, w)`.

Execution Arguments
===================

When executed, the inputs and outputs should be mapped to an execution
argument index as specified by the following table.

====================== ========================
Primitive input/output Execution argument index
====================== ========================
:math:`\src`           |DNNL_ARG_SRC|
:math:`\dst`           |DNNL_ARG_DST|
workspace              |DNNL_ARG_WORKSPACE|
:math:`\diffsrc`       |DNNL_ARG_DIFF_SRC|
:math:`\diffdst`       |DNNL_ARG_DIFF_DST|
====================== ========================

*****************
Operation Details
*****************

1. During training, LRN might or might not require a workspace on forward and
   backward passes. The behavior is implementation specific. Optimized
   implementations typically require a workspace and use it to save some
   intermediate results from the forward pass that accelerate computations on
   the backward pass. To check whether a workspace is required, query the LRN
   primitive descriptor for the workspace. Success indicates that the
   workspace is required and its description will be returned.

2. The memory format and data type for ``src`` and ``dst`` are assumed to be
   the same, and in the API are typically referred to as ``data`` (e.g., see
   ``data_desc`` in dnnl::lrn_forward::desc::desc()). The same holds for
   ``diff_src`` and ``diff_dst``. The corresponding memory descriptors are
   referred to as ``diff_data_desc``.

*****************
Data Type Support
*****************

The LRN primitive supports the following combinations of data types.

.. note::

   Here we abbreviate data types names for readability. For example, |_f32| is
   abbreviated to |f32|.

================== ====================
Propagation        Source / Destination
================== ====================
forward / backward |f32|, |bf16|
forward            |f16|
================== ====================

*******************
Data Representation
*******************

Source, Destination, and Their Gradients
========================================

Like most other primitives, the LRN primitive expects the following tensors:

======= =============================================
Spatial Source / Destination
======= =============================================
0D      :math:`N \times C`
1D      :math:`N \times C \times W`
2D      :math:`N \times C \times H \times W`
3D      :math:`N \times C \times D \times H \times W`
======= =============================================

The LRN primitive is optimized for the following memory formats:

======= ============== =====================================================
Spatial Logical tensor Implementations optimized for memory formats
======= ============== =====================================================
2D      NCHW           |nchw| (|abcd|), |nhwc| (|acdb|), *optimized*
======= ============== =====================================================

Here *optimized* means the format chosen by the preceding compute-intensive
primitive.

***********************
Post-ops and Attributes
***********************

The LRN primitive does not support any post-ops or attributes.

***
API
***

.. doxygenstruct:: dnnl::lrn_forward
   :project: oneDNN
   :members:

.. doxygenstruct:: dnnl::lrn_backward
   :project: oneDNN
   :members:


.. vim: ts=3 sw=3 et spell spelllang=en