| commit | 2f0ddb67d8f9267ab600a8a26308cab32f9e16ac | [log] [tgz] |
|---|---|---|
| author | Eanna O Cathain <eanna.ocathain@arm.com> | Thu Mar 03 15:58:10 2022 +0000 |
| committer | TeresaARM <teresa.charlinreyes@arm.com> | Wed Apr 13 14:27:44 2022 +0000 |
| tree | 0eab15a96ebf1e21e60347804a08144380e53800 | |
| parent | bab8fa9a11cf3bfef4b72fb757b81575b6fd75f0 [diff] |
MLECO-2492 Add CPP OD example with TFLITE-ArmnnDelegate Signed-off-by: Dvir Markovich <dvir.markovich@arm.com> Change-Id: If412c15ba49abe8370a570260b0a8ed8de305b7c
Arm NN is a key component of the machine learning platform, which is part of the Linaro Machine Intelligence Initiative.
The Arm NN SDK is a set of open-source software and tools that enables machine learning workloads on power-efficient devices. It provides a bridge between existing neural network frameworks and power-efficient Cortex-A CPUs, Arm Mali GPUs and Arm Ethos NPUs.
Arm NN SDK utilizes the Compute Library to target programmable cores, such as Cortex-A CPUs and Mali GPUs, as efficiently as possible. To target Ethos NPUs the NPU-Driver is utilized. We also welcome new contributors to provide their own driver and backend. Note, Arm NN does not provide support for Cortex-M CPUs.
Arm NN support models created with TensorFlow Lite (TfLite) and ONNX. Arm NN analysis a given model and replaces the operations within it with implementations particularly designed for the hardware you want to execute it on. This results in a great boost of execution speed. How much faster your neural network can be executed depends on the operations it contains and the available hardware. Below you can see the speedup we've been experiencing in our experiments with a few common networks.
\image html PerformanceChart.png
Arm NN is written using portable C++14 and the build system uses CMake, therefore it is possible to build for a wide variety of target platforms, from a wide variety of host environments.
Arm NN has added some quick start guides that will help you to setup Arm NN and run models quickly. The quickest way to build Arm NN is to either use our Debian package or use the prebuilt binaries available in the Assets section of every Arm NN release. There is an installation guide available here which provides step by step instructions on how to install the Arm NN Core, the TfLite Parser and PyArmNN for Ubuntu 20.04. These guides can be used with the prebuilt binaries. At present we have added a quick start guide that will show you how to integrate the delegate into TfLite to run models using python. More guides will be added here in the future.
Depending on what kind of framework (Tensorflow Lite, ONNX) you've been using to create your model there are multiple software tools available within Arm NN that can serve your needs.
Generally, there is a parser available for each supported framework. ArmNN-Parsers are C++ libraries that you can integrate into your application to load, optimize and execute your model. Each parser allows you to run models from one framework. If you would like to run an ONNX model you can make use of the Onnx-Parser. There also is a parser available for TfLite models but the preferred way to execute TfLite models is using our TfLite-Delegate. We also provide python bindings for our parsers and the Arm NN core. We call the result PyArmNN. Therefore your application can be conveniently written in either C++ using the "original" Arm NN library or in Python using PyArmNN. You can find tutorials on how to setup and use our parsers in our doxygen documentation. The latest version can be found in the wiki section of this repository.
Arm NN's software toolkit comes with the TfLite Delegate which can be integrated into TfLite. TfLite will then delegate operations, that can be accelerated with Arm NN, to Arm NN. Every other operation will still be executed with the usual TfLite runtime. This is our recommended way to accelerate TfLite models. As with our parsers there are tutorials in our doxygen documentation that can be found in the wiki section.
If you would like to use Arm NN on Android you can follow this guide which explains how to build Arm NN using the AndroidNDK. But you might also want to take a look at another repository which implements a hardware abstraction layer (HAL) for Android. The repository is called Android-NN-Driver and when integrated into Android it will automatically run neural networks with Arm NN.
The section above introduces the most important components that Arm NN provides. You can find a complete list in our doxygen documentation. The latest version can be found in the wiki section of our github repository.
For FAQs and troubleshooting advice, see FAQ.md or take a look at previous github issues.
If you would like to get involved but don't know where to start, a good place to look is in our Github Issues.
Feature requests without a volunteer to implement them are closed, but have the 'Help wanted' label, these can be found here. Once you find a suitable Issue, feel free to re-open it and add a comment, so that other people know you are working on it and can help.
When the feature is implemented the 'Help wanted' label will be removed.
The Arm NN project welcomes contributions. For more details on contributing to Arm NN see the Contributing page on the MLPlatform.org website, or see the Contributor Guide.
Particularly if you'd like to implement your own backend next to our CPU, GPU and NPU backends there are guides for backend development: Backend development guide, Dynamic backend development guide
The armnn/tests directory contains tests used during Arm NN development. Many of them depend on third-party IP, model protobufs and image files not distributed with Arm NN. The dependencies of some of the tests are available freely on the Internet, for those who wish to experiment, but they won't run out of the box.
Arm NN is provided under the MIT license. See LICENSE for more information. Contributions to this project are accepted under the same license.
Individual files contain the following tag instead of the full license text.
SPDX-License-Identifier: MIT
This enables machine processing of license information based on the SPDX License Identifiers that are available here: http://spdx.org/licenses/
Third party tools used by Arm NN:
| Tool | License (SPDX ID) | Description | Version | Provenience |
|---|---|---|---|---|
| cxxopts | MIT | A lightweight C++ option parser library | SHA 12e496da3d486b87fa9df43edea65232ed852510 | https://github.com/jarro2783/cxxopts |
| doctest | MIT | Header-only C++ testing framework | 2.4.6 | https://github.com/onqtam/doctest |
| fmt | MIT | {fmt} is an open-source formatting library providing a fast and safe alternative to C stdio and C++ iostreams. | 7.0.1 | https://github.com/fmtlib/fmt |
| ghc | MIT | A header-only single-file std::filesystem compatible helper library | 1.3.2 | https://github.com/gulrak/filesystem |
| half | MIT | IEEE 754 conformant 16-bit half-precision floating point library | 1.12.0 | http://half.sourceforge.net |
| mapbox/variant | BSD | A header-only alternative to 'boost::variant' | 1.1.3 | https://github.com/mapbox/variant |
| stb | MIT | Image loader, resize and writer | 2.16 | https://github.com/nothings/stb |
Arm NN uses the following security related build flags in their code:
| Build flags |
|---|
| -Wall |
| -Wextra |
| -Wold-style-cast |
| -Wno-missing-braces |
| -Wconversion |
| -Wsign-conversion |
| -Werror |