| /// Copyright (c) 2022-2023 ARM Limited and Contributors. All rights reserved. |
| /// |
| /// SPDX-License-Identifier: MIT |
| /// |
| |
| namespace armnn |
| { |
| /** |
| @page delegate TfLite Delegate |
| |
| |
| @section delegateintro About the delegate |
| 'armnnDelegate' is a library for accelerating certain TensorFlow Lite (TfLite) operators on Arm hardware. It can be |
| integrated in TfLite using its delegation mechanism. TfLite will then delegate the execution of operators supported by |
| Arm NN to Arm NN. |
| |
| The main difference to our @ref S6_tf_lite_parser is the amount of operators you can run with it. If none of the active |
| backends support an operation in your model you won't be able to execute it with our parser. In contrast to that, TfLite |
| only delegates operations to the armnnDelegate if it does support them and otherwise executes them itself. In other |
| words, every TfLite model can be executed and every operation in your model that we can accelerate will be accelerated. |
| That is the reason why the armnnDelegate is our recommended way to accelerate TfLite models. |
| |
| If you need help building the armnnDelegate, please take a look at our [build guide](delegate/BuildGuideNative.md). |
| |
| |
| @section delegatesupport Supported Operators |
| This reference guide provides a list of TensorFlow Lite operators the Arm NN SDK currently supports. |
| |
| @section delegatefullysupported Fully supported |
| |
| The Arm NN SDK TensorFlow Lite delegate currently supports the following operators: |
| |
| - ABS |
| |
| - ADD |
| |
| - ARG_MAX |
| |
| - ARG_MIN |
| |
| - AVERAGE_POOL_2D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE |
| |
| - AVERAGE_POOL_3D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, SIGN_BIT, TANH, NONE |
| |
| - BATCH_MATMUL |
| |
| - BATCH_TO_SPACE_ND |
| |
| - BROADCAST_TO |
| |
| - CAST |
| |
| - CEIL |
| |
| - CONCATENATION, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE |
| |
| - CONV_2D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE |
| |
| - CONV_3D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE |
| |
| - DEPTH_TO_SPACE |
| |
| - DEPTHWISE_CONV_2D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE |
| |
| - DEQUANTIZE |
| |
| - DIV |
| |
| - EQUAL |
| |
| - ELU |
| |
| - EXP |
| |
| - EXPAND_DIMS |
| |
| - FILL |
| |
| - FLOOR |
| |
| - FLOOR_DIV |
| |
| - FULLY_CONNECTED, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE |
| |
| - GATHER |
| |
| - GATHER_ND |
| |
| - GELU |
| |
| - GREATER |
| |
| - GREATER_EQUAL |
| |
| - HARD_SWISH |
| |
| - L2_NORMALIZATION |
| |
| - L2_POOL_2D |
| |
| - LESS |
| |
| - LESS_EQUAL |
| |
| - LOCAL_RESPONSE_NORMALIZATION |
| |
| - LOG |
| |
| - LOGICAL_AND |
| |
| - LOGICAL_NOT |
| |
| - LOGICAL_OR |
| |
| - LOGISTIC |
| |
| - LOG_SOFTMAX |
| |
| - LSTM |
| |
| - MAXIMUM |
| |
| - MAX_POOL_2D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE |
| |
| - MAX_POOL_3D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, SIGN_BIT, TANH, NONE |
| |
| - MEAN |
| |
| - MINIMUM |
| |
| - MIRROR_PAD |
| |
| - MUL |
| |
| - NEG |
| |
| - NOT_EQUAL |
| |
| - PACK |
| |
| - PAD |
| |
| - PADV2 |
| |
| - POW |
| |
| - PRELU |
| |
| - QUANTIZE |
| |
| - RANK |
| |
| - REDUCE_MAX |
| |
| - REDUCE_MIN |
| |
| - REDUCE_PROD |
| |
| - RELU |
| |
| - RELU6 |
| |
| - RELU_N1_TO_1 |
| |
| - RESHAPE |
| |
| - RESIZE_BILINEAR |
| |
| - RESIZE_NEAREST_NEIGHBOR |
| |
| - REVERSEV2 |
| |
| - RSQRT |
| |
| - SCATTERND |
| |
| - SHAPE |
| |
| - SIN |
| |
| - SOFTMAX |
| |
| - SPACE_TO_BATCH_ND |
| |
| - SPACE_TO_DEPTH |
| |
| - SPLIT |
| |
| - SPLIT_V |
| |
| - SQRT |
| |
| - SQUARE_DIFFERENCE |
| |
| - SQUEEZE |
| |
| - SLICE |
| |
| - STRIDED_SLICE |
| |
| - SUB |
| |
| - SUM |
| |
| - TANH |
| |
| - TILE |
| |
| - TRANSPOSE |
| |
| - TRANSPOSE_CONV |
| |
| - UNIDIRECTIONAL_SEQUENCE_LSTM |
| |
| - UNPACK |
| |
| More machine learning operators will be supported in future releases. |
| |
| @section delegateoptions Delegate Options |
| |
| The general list of runtime options are described in @subpage runtimeoptions |
| |
| @subsection opaquedelegateoptions Opaque Delegate Options |
| |
| In Opaque Delegate, delegate options are passed via ArmNNSettings which is a FlatBuffer of tflite::TFLiteSettings. |
| |
| Arm NN Settings | Possible Values | Description |
| :--------------- ------------- | :----- | :---------- |
| backends | ["GpuAcc"/"CpuAcc"] | A comma separated list without whitespaces of backends which should be used for execution. Falls back to next backend in list if previous does not provide support for operation. |
| fastmath | [true/false] | Allows the use of optimisation techniques e.g. Winograd that will reduce execution time with the possibility of a drop in accuracy. |
| additional_parameters | JSON string of additional Arm NN delegate options | JSON string of additional Arm NN delegate options. The general list of runtime options are described in [Runtime options](\ref runtimeoptions). |
| |
| **/ |
| } |