| // |
| // This confidential and proprietary software may be used only as |
| // authorised by a licensing agreement from ARM Limited |
| // (C) COPYRIGHT 2020-2021,2024 ARM Limited |
| // ALL RIGHTS RESERVED |
| // The entire notice above must be reproduced on all authorised |
| // copies and copies may only be made to the extent permitted |
| // by a licensing agreement from ARM Limited. |
| |
| == Operators |
| |
| === Operator Arguments |
| |
| Operators process input arguments to produce output arguments. |
| Their behavior can be configured using attribute arguments. |
| Arguments may have one of the following types: |
| |
| * `tensor_t<element_type>`, abbreviated `T<element_type>`, represents a tensor whose elements are of type `element_type` where `element_type` can be any of the data types supported in TOSA. |
| * `tensor_list_t` represents a list of tensors. When lists are homogeneous, i.e. contain tensors of the same type, their type is further qualified as follows: `tensor_list_t<T<element_type>>`. |
| * `tosa_graph_t` represents a TOSA graph (see <<operator-graphs>>). |
| |
| Arguments belong to one of three categories: Input, Output, or Attribute. The category to which an argument belongs further constrains its type: |
| |
| * An Input argument must be a tensor or a list of tensors used to provide the data read by the operation. |
| * An Output argument must be a tensor or a list of tensors into which the data produced by the operation is written. |
| * An Attribute argument is constant, i.e. its value is known at compilation time. It may have any data type supported by TOSA. |
| |
| [[operator-graphs]] |
| === Operator Graphs |
| |
| A TOSA graph is a collection of TOSA operators where: |
| |
| * The output of an operator in the graph may be connected to one or more inputs of other operators in the graph |
| * When an output is connected to an input the tensor list shapes must match |
| * The attributes of the operators are defined and considered part of the graph |
| * The attributes must be in the valid range permitted for the operator |
| * The tensor dimensions must be in the valid range permitted for the operator |
| |
| Some operators, such as control flow operators, take a graph of other operators as an attribute. The type `tosa_graph_t` will denote a graph of operators and the following functions define the tensor shape list for the graph input and outputs: |
| |
| [source,c++] |
| ---- |
| shape_list_t tosa_input_shape(tosa_graph_t graph); |
| shape_list_t tosa_output_shape(tosa_graph_t graph); |
| ---- |
| |
| Similarly the type tensor_list_t will be used for a list of tensors and the following function returns the shape of a tensor list: |
| [source,c++] |
| ---- |
| shape_list_t tensor_list_shape(tosa_list_t tensor_list); |
| ---- |
| |
| The following function denotes the execution of a TOSA graph within a TOSA context, |
| on an input tensor list to produce an output tensor list. A TOSA context, represented |
| by `tosa_context_t` provides the environment in which a TOSA graph is executed. |
| Any side-effects that result from the execution of a graph within a context are not |
| observable by graphs executing in a different context. Operators are executed in an |
| implementation-defined order that must be a topological ordering of the TOSA graph. |
| |
| [source,c++] |
| ---- |
| tosa_execute_graph(tosa_context_t context, tosa_graph_t graph, tosa_list_t input_list, tosa_list_t output_list, tosa_level_t level) { |
| ERROR_IF(tensor_list_shape(input_list) != tosa_input_shape(graph)); |
| ERROR_IF(tensor_list_shape(output_list) != tosa_output_shape(graph)); |
| |
| // Declare the global list for storing persistent variable tensors across multiple graphs |
| if (!variable_tensors) { |
| variable_tensors = list<tensor_t>(); |
| } else { // Clear the "seen flag" |
| for (tensor_t var_tensor in variable_tensors) { |
| var_tensor.seen = false; |
| } |
| } |
| |
| for_each(operator in graph order) { |
| ERROR_IF(operator input tensors do not meet requirement of operator Arguments inputs) |
| ERROR_IF(operator attributes do not meet requirement of operator Arguments attributes) |
| ERROR_IF(operator output tensors do not meet requirement of operator Arguments outputs) |
| ERROR_IF(operator data types do not meet requirement of operator Supported Data Types) |
| // Execute the operator as defined by the operation function pseduo-code |
| tosa_execute_operator(context, operator, level); |
| } |
| } |
| ---- |
| |
| include::tensor_ops.adoc[] |
| |
| include::activation_funcs.adoc[] |
| |
| include::ewise_binary.adoc[] |
| |
| include::ewise_unary.adoc[] |
| |
| include::ewise_ternary.adoc[] |
| |
| include::comparison.adoc[] |
| |
| include::reduction.adoc[] |
| |
| include::data_layout.adoc[] |
| |
| include::scatter_gather.adoc[] |
| |
| include::image.adoc[] |
| |
| include::type_conversion.adoc[] |
| |
| include::data_nodes.adoc[] |
| |
| include::custom.adoc[] |
| |
| include::control_flow.adoc[] |
| |
| include::variable.adoc[] |
| |
| include::shape.adoc[] |