Gitiles
Code Review Sign In
review.mlplatform.org / ml / armnn / 1dc83febfb76d6a770bdf3ba16c4034a970c2320 / . / 24.05 / parsers.html
blob: 0f0dcadbb543e3498ab3b16d05b277e1fce3dc97 [file] [log] [blame]
Nikhil Raj1dc83fe2024-05-16 09:47:51 +0100[diff] [blame^]1<!-- HTML header for doxygen 1.8.17-->
2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
3<html xmlns="http://www.w3.org/1999/xhtml">
4<head>
5<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
6<meta http-equiv="X-UA-Compatible" content="IE=9"/>
7<meta name="generator" content="Doxygen 1.8.17"/>
8<meta name="viewport" content="width=device-width, initial-scale=1"/>
9<title>Arm NN: Parsers</title>
10<link href="tabs.css" rel="stylesheet" type="text/css"/>
11<script type="text/javascript" src="jquery.js"></script>
12<script type="text/javascript" src="dynsections.js"></script>
13<link href="navtree.css" rel="stylesheet" type="text/css"/>
14<script type="text/javascript" src="resize.js"></script>
15<script type="text/javascript" src="navtreedata.js"></script>
16<script type="text/javascript" src="navtree.js"></script>
17<link href="search/search.css" rel="stylesheet" type="text/css"/>
18<script type="text/javascript" src="search/searchdata.js"></script>
19<script type="text/javascript" src="search/search.js"></script>
20<script type="text/x-mathjax-config">
21 MathJax.Hub.Config({
22 extensions: ["tex2jax.js"],
23 jax: ["input/TeX","output/HTML-CSS"],
24});
25</script>
26<script type="text/javascript" async="async" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js"></script>
27<link href="doxygen.css" rel="stylesheet" type="text/css" />
28<link href="customdoxygen.css" rel="stylesheet" type="text/css"/>
29</head>
30<body>
31<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
32<div id="titlearea">
33<table cellspacing="0" cellpadding="0">
34 <tbody>
35 <tr style="height: 56px;">
36 <img alt="ArmNN" src="Arm_NN_horizontal_blue.png" style="max-width: 15rem; margin-top: .5rem; margin-left 13px"/>
37 <td id="projectalign" style="padding-left: 0.9em;">
38 <div id="projectname">
39 &#160;<span id="projectnumber">24.05</span>
40 </div>
41 </td>
42 </tr>
43 </tbody>
44</table>
45</div>
46<!-- end header part -->
47<!-- Generated by Doxygen 1.8.17 -->
48<script type="text/javascript">
49/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
50var searchBox = new SearchBox("searchBox", "search",false,'Search');
51/* @license-end */
52</script>
53<script type="text/javascript" src="menudata.js"></script>
54<script type="text/javascript" src="menu.js"></script>
55<script type="text/javascript">
56/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
57$(function() {
58 initMenu('',true,false,'search.php','Search');
59 $(document).ready(function() { init_search(); });
60});
61/* @license-end */</script>
62<div id="main-nav"></div>
63</div><!-- top -->
64<div id="side-nav" class="ui-resizable side-nav-resizable">
65 <div id="nav-tree">
66 <div id="nav-tree-contents">
67 <div id="nav-sync" class="sync"></div>
68 </div>
69 </div>
70 <div id="splitbar" style="-moz-user-select:none;"
71 class="ui-resizable-handle">
72 </div>
73</div>
74<script type="text/javascript">
75/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
76$(document).ready(function(){initNavTree('parsers.html',''); initResizable(); });
77/* @license-end */
78</script>
79<div id="doc-content">
80<!-- window showing the filter options -->
81<div id="MSearchSelectWindow"
82 onmouseover="return searchBox.OnSearchSelectShow()"
83 onmouseout="return searchBox.OnSearchSelectHide()"
84 onkeydown="return searchBox.OnSearchSelectKey(event)">
85</div>
86
87<!-- iframe showing the search results (closed by default) -->
88<div id="MSearchResultsWindow">
89<iframe src="javascript:void(0)" frameborder="0"
90 name="MSearchResults" id="MSearchResults">
91</iframe>
92</div>
93
94<div class="PageDoc"><div class="header">
95 <div class="headertitle">
96<div class="title">Parsers </div> </div>
97</div><!--header-->
98<div class="contents">
99<div class="textblock"><p>Execute models from different machine learning platforms efficiently with our parsers. Simply choose a parser according to the model you want to run e.g. If you've got a model in onnx format (&lt;model_name&gt;.onnx) use our onnx-parser.</p>
100<p>If you would like to run a Tensorflow Lite (TfLite) model you probably also want to take a look at our <a class="el" href="delegate.html">TfLite Delegate</a>.</p>
101<p>All parsers are written in C++ but it is also possible to use them in python. For more information on our python bindings take a look into the <a class="el" href="md_python_pyarmnn__r_e_a_d_m_e.html">PyArmNN</a> section.</p>
102<p><br />
103<br />
104</p>
105<h1><a class="anchor" id="S5_onnx_parser"></a>
106Arm NN Onnx Parser</h1>
107<h2>Note: Arm NN will be dropping support for Onnx Parser in 24.08.</h2>
108<p><code><a class="el" href="namespacearmnn_onnx_parser.html">armnnOnnxParser</a></code> is a library for loading neural networks defined in ONNX protobuf files into the Arm NN runtime.</p>
109<h2>ONNX operators that the Arm NN SDK supports</h2>
110<p>This reference guide provides a list of ONNX operators the Arm NN SDK currently supports.</p>
111<p>The Arm NN SDK ONNX parser currently only supports fp32 operators.</p>
112<h3>Fully supported</h3>
113<ul>
114<li>Add<ul>
115<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Add">Add documentation</a> for more information</li>
116</ul>
117</li>
118<li>AveragePool<ul>
119<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#AveragePool">AveragePool documentation</a> for more information.</li>
120</ul>
121</li>
122<li>Concat<ul>
123<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Concat">Concat documentation</a> for more information.</li>
124</ul>
125</li>
126<li>Constant<ul>
127<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Constant">Constant documentation</a> for more information.</li>
128</ul>
129</li>
130<li>Clip<ul>
131<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Clip">Clip documentation</a> for more information.</li>
132</ul>
133</li>
134<li>Flatten<ul>
135<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Flatten">Flatten documentation</a> for more information.</li>
136</ul>
137</li>
138<li>Gather<ul>
139<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Gather">Gather documentation</a> for more information.</li>
140</ul>
141</li>
142<li>GlobalAveragePool<ul>
143<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#GlobalAveragePool">GlobalAveragePool documentation</a> for more information.</li>
144</ul>
145</li>
146<li>LeakyRelu<ul>
147<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#LeakyRelu">LeakyRelu documentation</a> for more information.</li>
148</ul>
149</li>
150<li>MaxPool<ul>
151<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#MaxPool">max_pool documentation</a> for more information.</li>
152</ul>
153</li>
154<li>Relu<ul>
155<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Relu">Relu documentation</a> for more information.</li>
156</ul>
157</li>
158<li>Reshape<ul>
159<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Reshape">Reshape documentation</a> for more information.</li>
160</ul>
161</li>
162<li>Shape<ul>
163<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Shape">Shape documentation</a> for more information.</li>
164</ul>
165</li>
166<li>Sigmoid<ul>
167<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Sigmoid">Sigmoid documentation</a> for more information.</li>
168</ul>
169</li>
170<li>Tanh<ul>
171<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Tanh">Tanh documentation</a> for more information.</li>
172</ul>
173</li>
174<li>Unsqueeze<ul>
175<li>See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Unsqueeze">Unsqueeze documentation</a> for more information.</li>
176</ul>
177</li>
178</ul>
179<h3>Partially supported</h3>
180<ul>
181<li>Conv<ul>
182<li>The parser only supports 2D convolutions with a group = 1 or group = #Nb_of_channel (depthwise convolution)</li>
183</ul>
184</li>
185<li>BatchNormalization<ul>
186<li>The parser does not support training mode. See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#BatchNormalization">BatchNormalization documentation</a> for more information.</li>
187</ul>
188</li>
189<li>Gemm<ul>
190<li>The parser only supports constant bias or non-constant bias where bias dimension = 1. See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#Gemm">Gemm documentation</a> for more information.</li>
191</ul>
192</li>
193<li>MatMul<ul>
194<li>The parser only supports constant weights in a fully connected layer. See the ONNX <a href="https://github.com/onnx/onnx/blob/master/docs/Operators.md#MatMul">MatMul documentation</a> for more information.</li>
195</ul>
196</li>
197</ul>
198<h2>Tested networks</h2>
199<p>Arm tested these operators with the following ONNX fp32 neural networks:</p><ul>
200<li>Mobilenet_v2. See the ONNX <a href="https://github.com/onnx/models/tree/master/vision/classification/mobilenet">MobileNet documentation</a> for more information.</li>
201<li>Simple MNIST. This is no longer directly documented by ONNX. The model and test data may be downloaded <a href="https://onnxzoo.blob.core.windows.net/models/opset_8/mnist/mnist.tar.gz">from the ONNX model zoo</a>.</li>
202</ul>
203<p>More machine learning operators will be supported in future releases. <br />
204<br />
205<br />
206<br />
207</p>
208<h1><a class="anchor" id="S6_tf_lite_parser"></a>
209Arm NN Tf Lite Parser</h1>
210<p><code><a class="el" href="namespacearmnn_tf_lite_parser.html">armnnTfLiteParser</a></code> is a library for loading neural networks defined by TensorFlow Lite FlatBuffers files into the Arm NN runtime.</p>
211<h2>TensorFlow Lite operators that the Arm NN SDK supports</h2>
212<p>This reference guide provides a list of TensorFlow Lite operators the Arm NN SDK currently supports.</p>
213<h3>Fully supported</h3>
214<p>The Arm NN SDK TensorFlow Lite parser currently supports the following operators:</p>
215<ul>
216<li>ABS</li>
217<li>ADD</li>
218<li>ARG_MAX</li>
219<li>ARG_MIN</li>
220<li>AVERAGE_POOL_2D, Supported Fused Activation: RELU , RELU6 , TANH, NONE</li>
221<li>BATCH_TO_SPACE</li>
222<li>BROADCAST_TO</li>
223<li>CAST</li>
224<li>CEIL</li>
225<li>CONCATENATION, Supported Fused Activation: RELU , RELU6 , TANH, NONE</li>
226<li>CONV_2D, Supported Fused Activation: RELU , RELU6 , TANH, NONE</li>
227<li>CONV_3D, Supported Fused Activation: RELU , RELU6 , TANH, NONE</li>
228<li>DEPTH_TO_SPACE</li>
229<li>DEPTHWISE_CONV_2D, Supported Fused Activation: RELU , RELU6 , TANH, NONE</li>
230<li>DEQUANTIZE</li>
231<li>DIV</li>
232<li>ELU</li>
233<li>EQUAL</li>
234<li>EXP</li>
235<li>EXPAND_DIMS</li>
236<li>FLOOR_DIV</li>
237<li>FULLY_CONNECTED, Supported Fused Activation: RELU , RELU6 , TANH, NONE</li>
238<li>GATHER</li>
239<li>GATHER_ND</li>
240<li>GELU</li>
241<li>GREATER</li>
242<li>GREATER_EQUAL</li>
243<li>HARD_SWISH</li>
244<li>LEAKY_RELU</li>
245<li>LESS</li>
246<li>LESS_EQUAL</li>
247<li>LOG</li>
248<li>LOGICAL_NOT</li>
249<li>LOGISTIC</li>
250<li>LOG_SOFTMAX</li>
251<li>L2_NORMALIZATION</li>
252<li>MAX_POOL_2D, Supported Fused Activation: RELU , RELU6 , TANH, NONE</li>
253<li>MAXIMUM</li>
254<li>MEAN</li>
255<li>MINIMUM</li>
256<li>MIRROR_PAD</li>
257<li>MUL</li>
258<li>NEG</li>
259<li>NOT_EQUAL</li>
260<li>PACK</li>
261<li>PAD</li>
262<li>PADV2</li>
263<li>POW</li>
264<li>PRELU</li>
265<li>QUANTIZE</li>
266<li>RELU</li>
267<li>RELU6</li>
268<li>REDUCE_MAX</li>
269<li>REDUCE_MIN</li>
270<li>REDUCE_PROD</li>
271<li>RESHAPE</li>
272<li>RESIZE_BILINEAR</li>
273<li>RESIZE_NEAREST_NEIGHBOR</li>
274<li>REVERSE_V2</li>
275<li>RSQRT</li>
276<li>SCATTER_ND</li>
277<li>SHAPE</li>
278<li>SIN</li>
279<li>SLICE</li>
280<li>SOFTMAX</li>
281<li>SPACE_TO_BATCH</li>
282<li>SPACE_TO_DEPTH</li>
283<li>SPLIT</li>
284<li>SPLIT_V</li>
285<li>SQUEEZE</li>
286<li>SQRT</li>
287<li>SQUARE</li>
288<li>SQUARE_DIFFERENCE</li>
289<li>STRIDED_SLICE</li>
290<li>SUB</li>
291<li>SUM</li>
292<li>TANH</li>
293<li>TILE</li>
294<li>TRANSPOSE</li>
295<li>TRANSPOSE_CONV</li>
296<li>UNIDIRECTIONAL_SEQUENCE_LSTM</li>
297<li>UNPACK</li>
298</ul>
299<h3>Custom Operator</h3>
300<ul>
301<li>TFLite_Detection_PostProcess</li>
302</ul>
303<h2>Tested networks</h2>
304<p>Arm tested these operators with the following TensorFlow Lite neural network:</p><ul>
305<li><a href="http://download.tensorflow.org/models/mobilenet_v1_2018_02_22/mobilenet_v1_1.0_224_quant.tgz">Quantized MobileNet</a></li>
306<li><a href="http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.tar.gz">Quantized SSD MobileNet</a></li>
307<li>DeepSpeech v1 converted from <a href="https://github.com/mozilla/DeepSpeech/releases/tag/v0.4.1">TensorFlow model</a></li>
308<li>DeepSpeaker</li>
309<li><a href="https://www.tensorflow.org/lite/models/segmentation/overview">DeepLab v3+</a></li>
310<li>FSRCNN</li>
311<li>EfficientNet-lite</li>
312<li>RDN converted from <a href="https://github.com/hengchuan/RDN-TensorFlow">TensorFlow model</a></li>
313<li>Quantized RDN (CpuRef)</li>
314<li><a href="http://download.tensorflow.org/models/tflite_11_05_08/inception_v3_quant.tgz">Quantized Inception v3</a></li>
315<li><a href="http://download.tensorflow.org/models/inception_v4_299_quant_20181026.tgz">Quantized Inception v4</a> (CpuRef)</li>
316<li>Quantized ResNet v2 50 (CpuRef)</li>
317<li>Quantized Yolo v3 (CpuRef)</li>
318</ul>
319<p>More machine learning operators will be supported in future releases. </p>
320</div></div><!-- contents -->
321</div><!-- PageDoc -->
322</div><!-- doc-content -->
323<!-- start footer part -->
324<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
325 <ul>
326 <li class="navelem"><a class="el" href="swtools.html">Software Components</a></li>
327 <li class="footer">Generated on Thu May 16 2024 09:31:54 for Arm NN by
328 <a href="http://www.doxygen.org/index.html">
329 <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.17 </li>
330 </ul>
331</div>
332</body>
333</html>