Supported Ops

This file was automatically generated by Vela using the --supported-ops-report parameter.
Vela version: 3.4.0

This file complies with Gitiles Markdown syntax

Summary table of constraints for:

TFLite Summary Table

The table below contains TFLite operators that can be placed on the Ethos-U NPU.
If the constraints are not met, then that operator will be scheduled on the CPU instead.
For any other TFLite operator not listed, will be left untouched and scheduled on the CPU.
Please check the supported operator list for your chosen runtime for further information.

OperatorTFLite Constraints
ABSGeneric, Specific
ADDGeneric, Specific
AVERAGE_POOL_2DGeneric, Specific
CONCATENATIONGeneric, Specific
CONV_2DGeneric, Specific
DEPTHWISE_CONV_2DGeneric, Specific
EXPAND_DIMSGeneric, Specific
FULLY_CONNECTEDGeneric, Specific
HARD_SWISHGeneric, Specific
LEAKY_RELUGeneric, Specific
LOGISTICGeneric
MAXIMUMGeneric, Specific
MAX_POOL_2DGeneric, Specific
MEANGeneric, Specific
MINIMUMGeneric, Specific
MULGeneric, Specific
PACKGeneric
PADGeneric, Specific
QUANTIZEGeneric
RELUGeneric
RELU6Generic
RELU_N1_TO_1Generic
RESHAPEGeneric, Specific
RESIZE_BILINEARGeneric, Specific
SLICEGeneric
SOFTMAXGeneric, Specific
SPLITGeneric
SPLIT_VGeneric, Specific
SQUEEZEGeneric, Specific
STRIDED_SLICEGeneric, Specific
SUBGeneric, Specific
TANHGeneric
TRANSPOSE_CONVGeneric, Specific
UNPACKGeneric

TFLite Generic Constraints

This is a list of constraints that all NPU operators must satisfy in order to be scheduled on the NPU.

  • Input(s) and Output tensors must not be dynamic
  • Input(s) and Output tensors must have a defined shape
  • Output tensors cannot be scalar
  • Scalar Input tensors are only valid for op type: ADD, EXPAND_DIMS, MAXIMUM, MEAN, MINIMUM, MUL, SPLIT, SPLIT_V, SUB
  • Input(s) and Output tensors must not be greater than 4D
  • Input(s), Output and Weight tensors must have quantization parameters
  • Input(s), Output and Weight tensors with quantization scales must be finite
  • Input and Output tensors must have quantization scales that fit within float32 precision
  • Constant tensors should not have NoneType-values
  • Tensors must be of type: int16, int32, int8, uint8
  • Tensors which are int32 are only valid when op type is: ADD, MUL, SUB
  • Tensor dimensions must be in the range [1, 65535]
  • Per-axis quantization is only supported for the following op types: CONV_2D, DEPTHWISE_CONV_2D, TRANSPOSE_CONV
  • The fused activation function (if present) must be one of type: LOGISTIC, RELU, RELU6, RELU_N1_TO_1, TANH
  • If a fused activation function is present, the Output tensor must be one of type: int16, int8, uint8

TFLite ABS Constraints

This is a list of constraints that the ABS operator must satisfy in order to be scheduled on the NPU.

  • At least one Input's shape must match the OFM's shape
  • IFM and OFM data types must match
  • Batch size must be 1 for Input tensors with more than 2 dimensions

TFLite ADD Constraints

This is a list of constraints that the ADD operator must satisfy in order to be scheduled on the NPU.

  • At least one Input's shape must match the OFM's shape
  • Both Input data types must match
  • For IFM that are signed, OFM must also be signed
  • For IFM that are unsigned, OFM must either be the same type or int32
  • Batch size must be 1 for Input tensors with more than 2 dimensions
  • Broadcasting is only allowed for rank indices with dimension 1, from either IFM1 or IFM2

TFLite AVERAGE_POOL_2D Constraints

This is a list of constraints that the AVERAGE_POOL_2D operator must satisfy in order to be scheduled on the NPU.

  • Stride values for both width and height must be integer types
  • IFM and OFM data types must match
  • Kernel filter values for both width and height must be integer types
  • IFM Tensor batch size must be 1
  • Stride values for both width and height must be in the range [1, 3]
  • Kernel filter values for both width and height must be in the range [1, 8]
  • VALID padding: Kernel filter height must be in the range [1, 256]
  • VALID padding: Product of kernel filter width and height must be in the range [1, 65536]

TFLite CONCATENATION Constraints

This is a list of constraints that the CONCATENATION operator must satisfy in order to be scheduled on the NPU.

  • Axis attribute must exist
  • Axis attribute must be in the range [0, <ofm_dimensions>)
  • All Input dimensionalities must match OFM dimensionality
  • All Input dimensions must match OFM dimension in all axes except the one defined by the axis attribute

TFLite CONV_2D Constraints

This is a list of constraints that the CONV_2D operator must satisfy in order to be scheduled on the NPU.

  • Stride values for both width and height must be integer types
  • Dilation factor values for both width and height must be integer types
  • Stride values for both width and height must be in the range [1, 3]
  • Dilation factor values for both width and height must be in the range [1, 2]
  • Dilated kernel height must be in the range [1, 64]
  • Product of dilated kernel width and height must be in the range [1, 4096]
  • Weight tensor must be 8-bit
  • Weight tensor must be constant
  • The sum of the weights cannot exceed 8323072
  • Optional Bias tensor must be of type: int32, int64
  • Optional Bias tensor values must fit within 40-bits
  • IFM Tensor batch size must be 1

TFLite DEPTHWISE_CONV_2D Constraints

This is a list of constraints that the DEPTHWISE_CONV_2D operator must satisfy in order to be scheduled on the NPU.

  • Stride values for both width and height must be integer types
  • Dilation factor values for both width and height must be integer types
  • Stride values for both width and height must be in the range [1, 3]
  • Dilation factor values for both width and height must be in the range [1, 2]
  • Dilated kernel height must be in the range [1, 64]
  • Product of dilated kernel width and height must be in the range [1, 4096]
  • Weight tensor must be 8-bit
  • Weight tensor must be constant
  • The sum of the weights cannot exceed 8323072
  • Optional Bias tensor must be of type: int32, int64
  • Optional Bias tensor values must fit within 40-bits
  • IFM Tensor batch size must be 1
  • For depth multipliers > 1, IFM channels must be 1 and OFM channels must be equal to the depth multiplier

TFLite EXPAND_DIMS Constraints

This is a list of constraints that the EXPAND_DIMS operator must satisfy in order to be scheduled on the NPU.

  • Input and output quantisation must match.

TFLite FULLY_CONNECTED Constraints

This is a list of constraints that the FULLY_CONNECTED operator must satisfy in order to be scheduled on the NPU.

  • The output tensor(s) must have 2D shape
  • The IFM and OFM must have the same number of dimensions if keep_num_dims is set to true
  • Weight tensor must be 8-bit
  • Weight tensor must be constant
  • Optional Bias tensor must be of type: int32, int64
  • Optional Bias tensor values must fit within 40-bits

TFLite HARD_SWISH Constraints

This is a list of constraints that the HARD_SWISH operator must satisfy in order to be scheduled on the NPU.

  • IFM must be int8 or uint8
  • IFM and OFM data types must match

TFLite LEAKY_RELU Constraints

This is a list of constraints that the LEAKY_RELU operator must satisfy in order to be scheduled on the NPU.

  • At least one Input's shape must match the OFM's shape
  • IFM and OFM data types must match
  • Alpha only allowed to be negative if IFM is int8 or uint8
  • Batch size must be 1 for Input tensors with more than 2 dimensions

TFLite MAXIMUM Constraints

This is a list of constraints that the MAXIMUM operator must satisfy in order to be scheduled on the NPU.

  • At least one Input's shape must match the OFM's shape
  • IFM and OFM data types must match
  • Batch size must be 1 for Input tensors with more than 2 dimensions
  • Both Input quantization parameters must match OFM quantization parameters
  • Broadcasting is only allowed for rank indices with dimension 1, from either IFM1 or IFM2

TFLite MAX_POOL_2D Constraints

This is a list of constraints that the MAX_POOL_2D operator must satisfy in order to be scheduled on the NPU.

  • Stride values for both width and height must be integer types
  • IFM and OFM data types must match
  • Kernel filter values for both width and height must be integer types
  • IFM Tensor batch size must be 1
  • Stride values for both width and height must be in the range [1, 3]
  • Kernel filter height must be in the range [1, 256]
  • Product of kernel filter width and height must be in the range [1, 65536]

TFLite MEAN Constraints

This is a list of constraints that the MEAN operator must satisfy in order to be scheduled on the NPU.

  • IFM must be int8 or uint8
  • Input tensor must be at least 2D
  • Axis indices must correspond to height and width axes
  • IFM Tensor batch size must be 1
  • Product of height and width must be no greater than 65536
  • Product of height and width must be no greater than 4096 when:
    IFM and OFM have different scale or zero point; or
    'keep_dims' is True
  • Product of IFM height and width must be no greater than 256 when:
    The IFM shape has 4 dimensions; and
    The axis indices specify reduction across 2 dimensions; and
    The axis indices correspond to the width and height dimensions of the IFM; and
    'keep_dims' is True; and
    IFM datatype is int8
  • For single axis averages across the height dimension:
    IFM height must be no greater than 256 if the IFM and OFM scale and zero point match; otherwise
    IFM height must be no greater than 64 if the IFM and OFM scale or zero point do not match

TFLite MINIMUM Constraints

This is a list of constraints that the MINIMUM operator must satisfy in order to be scheduled on the NPU.

  • At least one Input's shape must match the OFM's shape
  • IFM and OFM data types must match
  • Batch size must be 1 for Input tensors with more than 2 dimensions
  • Both Input quantization parameters must match OFM quantization parameters
  • Broadcasting is only allowed for rank indices with dimension 1, from either IFM1 or IFM2

TFLite MUL Constraints

This is a list of constraints that the MUL operator must satisfy in order to be scheduled on the NPU.

  • At least one Input's shape must match the OFM's shape
  • Both Input data types must match
  • For IFM that are signed, OFM must also be signed
  • For IFM that are unsigned, OFM must either be the same type or int32
  • Batch size must be 1 for Input tensors with more than 2 dimensions
  • Broadcasting is only allowed for rank indices with dimension 1, from either IFM1 or IFM2

TFLite PAD Constraints

This is a list of constraints that the PAD operator must satisfy in order to be scheduled on the NPU.

  • Number of input tensors must be exactly 2
  • The padding tensor must be constant
  • The padding tensor must have the shape [3,2] or [4,2]
  • The pad tensor can only pad width and height
  • Pad tensor must be of type: int32, int64

TFLite RESHAPE Constraints

This is a list of constraints that the RESHAPE operator must satisfy in order to be scheduled on the NPU.

  • Input and output quantisation must match.
  • Shape must be constant

TFLite RESIZE_BILINEAR Constraints

This is a list of constraints that the RESIZE_BILINEAR operator must satisfy in order to be scheduled on the NPU.

  • The width and height of the IFM and OFM must match one of the following criteria:
    IFM W and H must both be 1
    IFM must match OFM
    OFM W and H must be equal and 2/4/8x IFM -1, if align_corners is True
    OFM W and H must be equal and 2/4/8x IFM, if align_corners is False
  • half_pixel_centers are not supported

TFLite SOFTMAX Constraints

This is a list of constraints that the SOFTMAX operator must satisfy in order to be scheduled on the NPU.

  • IFM and OFM shapes must match
  • IFM and OFM data types must match
  • Beta value needs to be positive

TFLite SPLIT_V Constraints

This is a list of constraints that the SPLIT_V operator must satisfy in order to be scheduled on the NPU.

  • Only one size is allowed to be inferred

TFLite SQUEEZE Constraints

This is a list of constraints that the SQUEEZE operator must satisfy in order to be scheduled on the NPU.

  • Input and output quantisation must match.

TFLite STRIDED_SLICE Constraints

This is a list of constraints that the STRIDED_SLICE operator must satisfy in order to be scheduled on the NPU.

  • Exactly 4 Input tensors are required
  • Begin, End and Stride Input tensors must be constant
  • ellipsis_mask must be 0
  • new_axis_mask and shrink_axis_mask cannot both be set
  • Slice 'end' values must be greater than 'begin' values
  • All Strides values must be 1

TFLite SUB Constraints

This is a list of constraints that the SUB operator must satisfy in order to be scheduled on the NPU.

  • At least one Input's shape must match the OFM's shape
  • Both Input data types must match
  • For IFM that are signed, OFM must also be signed
  • For IFM that are unsigned, OFM must either be the same type or int32
  • Batch size must be 1 for Input tensors with more than 2 dimensions
  • Broadcasting is only allowed for rank indices with dimension 1, from either IFM1 or IFM2

TFLite TRANSPOSE_CONV Constraints

This is a list of constraints that the TRANSPOSE_CONV operator must satisfy in order to be scheduled on the NPU.

  • Stride values for both width and height must be integer types
  • Dilation factor values for both width and height must be integer types
  • Stride values for both width and height must be in the range [1, 3]
  • Dilation factor values for both width and height must be in the range [1, 2]
  • Dilated kernel height must be in the range [1, 64]
  • Product of dilated kernel width and height must be in the range [1, 4096]
  • Weight tensor must be 8-bit
  • Weight tensor must be constant
  • The sum of the weights cannot exceed 8323072
  • Optional Bias tensor must be of type: int32, int64
  • Optional Bias tensor values must fit within 40-bits
  • IFM Tensor batch size must be 1
  • Stride values for both width and height must be 2
  • SAME padding: OFM dimensions must equal IFM dimensions multiplied by stride
  • VALID padding: OFM dimensions must equal IFM dimensions multiplied by stride,
    minus difference between kernel size and stride