| // |
| // This confidential and proprietary software may be used only as |
| // authorised by a licensing agreement from ARM Limited |
| // (C) COPYRIGHT 2020-2024 ARM Limited |
| // ALL RIGHTS RESERVED |
| // The entire notice above must be reproduced on all authorised |
| // copies and copies may only be made to the extent permitted |
| // by a licensing agreement from ARM Limited. |
| |
| |
| === Activation Functions |
| |
| ==== CLAMP |
| |
| Clamp to an arbitrary minimum and maximum value. |
| Maximum and minimum values are specified as values in the range of the input type. |
| No zero point subtraction is done to the values, thus to clamp to the zero point value, the zero point itself should be supplied as the minimum value. |
| |
| include::{generated}/operators/CLAMP.adoc[] |
| |
| [source,c++] |
| ---- |
| include::{pseudocode}/operators/CLAMP.tosac[lines=10..-1] |
| ---- |
| |
| ==== ERF |
| |
| // erf(z) = \frac{2}{\sqrt{\pi}} \int_{0}^{z} e^{-t^2} dt |
| Error function: |
| |
| .Calculation for the error function |
| image::erf.svg["Error function", align="center"] |
| |
| For quantized integer data types, the TABLE operator should be used instead with |
| the following definition. |
| |
| The ERF table has 513 entries each of 16-bit precision and covering the input range -4.0 to +4.0 in steps of 1/64. |
| |
| [source,c++] |
| ---- |
| include::{pseudocode}/operators/tables/ERF.tosac[lines=10..-1] |
| ---- |
| |
| include::{generated}/operators/ERF.adoc[] |
| |
| [source,c++] |
| ---- |
| include::{pseudocode}/operators/ERF.tosac[lines=10..-1] |
| ---- |
| |
| ==== SIGMOID |
| |
| Applies the sigmoid logistic function to each element of the input tensor. |
| |
| // sigmoid(x) = \frac{1}{1 + e^{-x}} |
| |
| .Calculation for the sigmoid function |
| image::sigmoid.svg["Sigmoid definition"] |
| |
| For quantized integer data types, the TABLE operator should be used instead. |
| Each implementation may choose an appropriate TABLE given the scale and zero point of the input data. |
| Eight or sixteen bit precision tables may be used based on the input tensor to the sigmoid function. |
| Below we give an example table generation for 16-bit sigmoid. |
| This sigmoid table has 513 entries each of 16-bit precision and covering the input range -16.0 to +16.0 in steps of 1/16. |
| |
| .Code for generating 16-bit sigmoid table |
| [source,c++] |
| ---- |
| include::{pseudocode}/operators/tables/SIGMOID.tosac[lines=10..-1] |
| ---- |
| |
| include::{generated}/operators/SIGMOID.adoc[] |
| |
| [source,c++] |
| ---- |
| include::{pseudocode}/operators/SIGMOID.tosac[lines=10..-1] |
| ---- |
| |
| ==== TANH |
| |
| Parameterized hyperbolic tangent. |
| // tanh(x) = \frac{1 - e^{-2x}}{1 + e^{-2x}} |
| |
| .Calculation for the sigmoid function |
| image::tanh.svg["Hyperbolic tangent definition"] |
| |
| For quantized integer data types, the TABLE operator should be used instead. |
| Each implementation may choose an appropriate TABLE given the scale and zero point of the input data. |
| Eight or sixteen bit precision tables may be used based on the input tensor to the sigmoid function. |
| Below we give an example table generation for 16-bit hyperbolic tangent. |
| This tanh_table has 513 entries each of 16-bit precision and covering the input range -8.0 to +8.0 in steps of 1/32. |
| |
| .Calculation of an example 16-bit tanh table |
| [source,c++] |
| ---- |
| include::{pseudocode}/operators/tables/TANH.tosac[lines=10..-1] |
| ---- |
| |
| include::{generated}/operators/TANH.adoc[] |
| |
| [source,c++] |
| ---- |
| include::{pseudocode}/operators/TANH.tosac[lines=10..-1] |
| ---- |