commit | 1c08afa0ed049edd486498e62bab94a4dc7924bc | [log] [tgz] |
---|---|---|
author | Rickard Bolin <rickard.bolin@arm.com> | Fri Jan 07 14:22:52 2022 +0000 |
committer | Rickard Bolin <rickard.bolin@arm.com> | Wed Jan 12 10:00:56 2022 +0000 |
tree | 451e53d3a8275d2ac774e6267ca96835d9189e9e | |
parent | bdb1d6e0fce5e52979f3a5742aaddd3a68b9a0f2 [diff] |
MLBEDSW-5534: Enet_640_640_int8 output diff The output diff is caused by not including the kernel dilation when calculating the bottom padding to be used on the last h_stripe. This only shows up when using dedicated_sram since shared_sram does not split into multiple h_stripes and thus uses the padding specified by the skirt instead. Signed-off-by: Rickard Bolin <rickard.bolin@arm.com> Change-Id: I7f643748b153004d65be2124c0ac6c9d21cd803f
This tool is used to compile a TensorFlow Lite for Microcontrollers neural network model into an optimised version that can run on an embedded system containing an Arm Ethos-U NPU.
In order to be accelerated by the Ethos-U NPU the network operators must be quantised to either 8-bit (unsigned or signed) or 16-bit (signed).
The optimised model will contain TensorFlow Lite Custom operators for those parts of the model that can be accelerated by the Ethos-U NPU. Parts of the model that cannot be accelerated are left unchanged and will instead run on the Cortex-M series CPU using an appropriate kernel (such as the Arm optimised CMSIS-NN kernels).
After compilation the optimised model can only be run on an Ethos-U NPU embedded system.
The tool will also generate performance estimates (EXPERIMENTAL) for the compiled model.
The tool has limited functionality for compiling a TOSA neural network (EXPERIMENTAL).
Vela runs on the Linux and Microsoft Windows 10 operating systems, see note in Installation section below.
The following should be installed prior to the installation of Vela:
And optionally:
Vela is available to install as a package from PyPi, or as source code from ML Platform. Both methods will automatically install all the required dependencies.
Note: For installing on Microsoft Windows 10 you need to have a C99 capable toolchain installed. The recommended and tested toolchain is Microsoft Visual C++ 14.2 Build Tools, see https://wiki.python.org/moin/WindowsCompilers
Install Vela from PyPi using the following command:
pip3 install ethos-u-vela
First obtain the source code by either downloading the desired TGZ file from:
https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ethos-u-vela
Or by cloning the git repository:
git clone https://review.mlplatform.org/ml/ethos-u/ethos-u-vela.git
Once you have the source code, Vela can be installed using the following command:
pip3 install .
Or, if you use pipenv
:
pipenv install .
If you plan to modify the Vela codebase then it is recommended to install Vela as an editable package to avoid the need to re-install after every modification. This is done by adding the -e
option to the above install commands like so:
pip3 install -e .
Or, if you use pipenv
:
pipenv install -e .
If you plan to contribute to the Vela project (highly encouraged!) then it is recommended to install Vela along with the pre-commit tools (see Vela Testing for more details).
Vela is run with an input .tflite
file passed on the command line. This file contains the neural network to be compiled. The tool then outputs an optimised version with a _vela.tflite
file prefix, along with the performance estimate (EXPERIMENTAL) CSV files, all to the output directory. It also prints a performance estimation summary back to the console, see Vela Performance Estimation Summary.
If you use the pipenv
virtual environment tool then first start by spawning a shell in the virtual environment:
pipenv shell
After which running Vela is the same regardless of whether you are in a virtual environment or not.
Example usage:
my_model.tflite
. The optimised version will be output to ./output/my_network_vela.tflite
.vela my_model.tflite
/path/to/my_model.tflite
and specify the output to go in the directory ./results_dir/
.vela --output-dir ./results_dir /path/to/my_model.tflite
vela --accelerator-config ethos-u65-512 my_model.tflite
vela --optimise Size my_model.tflite
vela --optimise Performance my_model.tflite
vela --optimise Performance --arena-cache-size 300000 my_model.tflite
My_Sys_Config
system configuration along with the My_Mem_Mode
memory mode from the vela_cfg.ini
configuration file.vela --config vela_cfg.ini --system-config My_Sys_Config --memory-mode My_Mem_Mode my_model.tflite
vela --help
When running the Vela compiler it may report a number of warning messages to the console. These should all be thoroughly reviewed as they will indicate decisions that the compiler has made in order to create the optimised network.
Some example networks that contain quantised operators which can be compiled by Vela to run on the Ethos-U NPU can be found at: https://tfhub.dev/s?deployment-format=lite&q=quantized
Please see Vela External APIs.
Please see Vela Contributions.
Please see Vela Debug Database.
Please see Vela CLI Options. This includes a description of the system configuration file format.
Please see Vela Performance Estimation Summary.
Please see Vela Releases.
Please see Vela Security.
Please see Vela Supported Operators for the list of operators supported in this release.
Please see Vela Testing.
Please see Vela Community Bug Reporting for a description of how to report bugs.
Additional useful information:
Vela is licensed under Apache License 2.0.