The ML embedded evaluation kit provides a range of ready to use machine learning (ML) applications for users to develop ML workloads running on the Arm® Ethos-U NPU and Arm® Cortex-M CPUs. You can also access metrics such as inference cycle count to estimate performance.
The Arm® Ethos-U NPU is a new class of ML processor, specifically designed to accelerate ML computation in constrained embedded and IoT devices.
Experiment with the included end-to-end software use cases and create your own ML applications for Cortex-M CPU and Ethos-U NPU.
|ML application||Description||Neural Network Model|
|Image classification||Recognize the presence of objects in a given image||Mobilenet V2|
|Keyword spotting(KWS)||Recognize the presence of a key word in a recording||MicroNet|
|Automated Speech Recognition(ASR)||Transcribe words in a recording||Wav2Letter|
|KWS and ASR||Utilise Cortex-M and Ethos-U to transcribe words in a recording after a keyword was spotted||MicroNet Wav2Letter|
|Anomaly Detection||Detecting abnormal behavior based on a sound recording of a machine||MicroNet|
|Visual Wake Word||Recognize if person is present in a given image||MicroNet|
|Noise Reduction||Remove noise from audio while keeping speech intact||RNNoise|
|Object detection||Detects and draws face bounding box in a given image||Yolo Fastest|
|Generic inference runner||Code block allowing you to develop your own use case for Ethos-U NPU||Your custom model|
This repository is for building and deploying Machine Learning (ML) applications targeted for Arm® Cortex®-M and Arm® Ethos™-U NPU. To run evaluations using this software, we suggest using:
Arm® Corstone™-300 MPS3 based Fixed Virtual Platform (FVP) offers a choice of the Arm® Ethos™-U55 NPU or Arm® Ethos™-U65 NPU software fast model in combination with the new Arm® Cortex™-M55 processor
Arm® Corstone™-300 and Corstone™-310 design implementations are publicly available on Download FPGA Images page, or as a Fixed Virtual Platform of the MPS3 development board.
To run ML applications on the Cortex-M and Ethos-U NPU:
First, verify that you have installed all of the required prerequisites.
Dockerfileis also available if you would like to build a Docker image.
Clone the Ethos-U evaluation kit repository:
git clone "https://review.mlplatform.org/ml/ethos-u/ml-embedded-evaluation-kit" cd ml-embedded-evaluation-kit
Pull all the external dependencies with the following command:
git submodule update --init
Next, run the
build_default Python script. It handles the downloading of the neural network models, compiling using Vela, and building the project using CMake.
python3.9 ./build_default.py --toolchain arm
Change directory to the generated cmake build folder which contains the
.axf file output in the
bin subdirectory. Launch the application by passing the
.axf to the FVP you downloaded when installing the prerequisites. Alternatively, from the root directory add
<cmake-build-your_config> to the path to the axf and use one of the following commands:
From auto-generated (or custom) build directory: <path_to_FVP>/FVP_Corstone_SSE-300_Ethos-U55 -a ./bin/ethos-u-kws.axf From root directory: <path_to_FVP>/FVP_Corstone_SSE-300_Ethos-U55 -a <cmake-build-your_config>/bin/ethos-u-kws.axf
A telnet window is launched through which you can interact with the application and obtain performance figures.
For more details, you can view the quick start guide.
Note: The default flow assumes Arm® Ethos™-U55 NPU usage, configured to use 128 Multiply-Accumulate units and is sharing SRAM with the Arm® Cortex®-M55.
Ml embedded evaluation kit supports:
Ethos™-U NPU Default MACs/cc Other MACs/cc supported Default Memory Mode Other Memory Modes supported Ethos™-U55 128 32, 64, 256 Shared_Sram Sram_Only Ethos™-U65 256 512 Dedicated_Sram Shared_Sram
For more information see Building.
See full documentation:
The ML use cases have common code such as initializing the Hardware Abstraction Layer (HAL)
The common application code can be run on native host machine (x86_64 or aarch64) or Arm Cortex-M architecture because of the HAL
Google® TensorFlow™ Lite for Microcontrollers inference engine is used to schedule the execution of neural network models
The Ethos-U NPU driver is integrated TensorFlow Lite for Microcontrollers
The provided set of common ML use-case functions will assist in implementing your application logic
The CMake build system will discover and automatically include new ML application code into the compilation workflow
A high level overview of the different components in the software, and the platforms supported out-of-the-box, is shown in the diagram below.
Note: The Ethos-U NPU software stack is described here.
For a more detailed description of the build graph with all major components, see Building.
There are source files in the repository that form the core of the Machine Leaning flow for all the use cases. These are exposed as APIs that the examples can use and even be combined to form chained use cases. The API sources are designed to be portable across platforms and provide functionality for preprocessing of data, running an inference, and postprocessing of results. These allow a common flow for all use cases with minor differences in how each of these blocks are instantiated.
As an independent CMake project, these APIs can be used by or integrated into other projects easily. We also produce CMSIS Packs with these sources, so they could be used in all tools/IDEs (for example, Arm® Development Studio and Keil® µVision®) that support the use of CMSIS Packs.
The ML embedded eval kit welcomes contributions. For more details on contributing to the eval kit please see the the contributors guide.
Please, if you want to start public discussion, raise any issues or questions related to this repository, use https://discuss.mlplatform.org/c/ml-embedded-evaluation-kit forum.
This product conforms to Arm's inclusive language policy and, to the best of our knowledge, does not contain any non-inclusive language. If you find something that concerns you, email email@example.com.
The ML Embedded applications samples are provided under the Apache 2.0 license, see License Apache 2.0.
Application input data sample files are provided under their original license:
|Automatic Speech Recognition Samples||Creative Commons Attribution 4.0 International Public License||http://www.openslr.org/12/|
|Image Classification Samples||Creative Commons Attribution 1.0||https://www.pexels.com|
|Keyword Spotting Samples||Creative Commons Attribution 4.0 International Public License||http://download.tensorflow.org/data/speech_commands_v0.02.tar.gz|
|Keyword Spotting and Automatic Speech Recognition Samples||Creative Commons Attribution 4.0 International Public License||http://download.tensorflow.org/data/speech_commands_v0.02.tar.gz|
|Visual Wake Word Samples||Creative Commons Attribution 1.0||https://www.pexels.com|
|Noise Reduction Samples||Creative Commons Attribution 4.0 International Public License||https://datashare.ed.ac.uk/handle/10283/2791/|
|Object Detection Samples||Creative Commons Attribution 1.0||https://www.pexels.com|