Ryan OShea | 2bbfaa7 | 2020-02-12 16:15:27 +0000 | [diff] [blame] | 1 | /// Copyright (c) 2020 ARM Limited. |
Ryan OShea | f3a4323 | 2020-02-12 16:15:27 +0000 | [diff] [blame] | 2 | /// |
| 3 | /// SPDX-License-Identifier: MIT |
| 4 | /// |
| 5 | /// Permission is hereby granted, free of charge, to any person obtaining a copy |
| 6 | /// of this software and associated documentation files (the "Software"), to deal |
| 7 | /// in the Software without restriction, including without limitation the rights |
| 8 | /// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell |
| 9 | /// copies of the Software, and to permit persons to whom the Software is |
| 10 | /// furnished to do so, subject to the following conditions: |
| 11 | /// |
| 12 | /// The above copyright notice and this permission notice shall be included in all |
| 13 | /// copies or substantial portions of the Software. |
| 14 | /// |
| 15 | /// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR |
| 16 | /// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, |
| 17 | /// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE |
| 18 | /// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER |
| 19 | /// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, |
| 20 | /// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE |
| 21 | /// SOFTWARE. |
| 22 | /// |
| 23 | |
| 24 | namespace armnn |
| 25 | { |
| 26 | /** |
| 27 | @page backends Backend Developer Guides |
| 28 | @tableofcontents |
| 29 | |
| 30 | @section S12_backend_developer_guide Backend Developer Guide |
| 31 | |
| 32 | Arm NN allows adding new backends through the `Pluggable Backend` mechanism. |
| 33 | |
| 34 | @subsection S12_1_backend_developer_guide How to add a new backend |
| 35 | |
| 36 | Backends reside under [src/backends](./), in separate subfolders. For Linux builds they must have a `backend.cmake` file, |
| 37 | which is read automatically by [src/backends/backends.cmake](backends.cmake). The `backend.cmake` file |
| 38 | under the backend-specific folder is then included by the main CMakeLists.txt file at the root of the |
| 39 | Arm NN source tree. |
| 40 | |
| 41 | ### The backend.cmake file |
| 42 | |
| 43 | The `backend.cmake` has three main purposes: |
| 44 | |
| 45 | 1. It makes sure the artifact (a cmake OBJECT library) is linked into the Arm NN shared library by appending the name of the library to the `armnnLibraries` list. |
| 46 | 2. It makes sure that the subdirectory where backend sources reside gets included into the build. |
| 47 | 3. To include backend-specific unit tests, the object library for the unit tests needs to be added to the `armnnUnitTestLibraries` list. |
| 48 | |
| 49 | Example `backend.cmake` file taken from [reference/backend.cmake](reference/backend.cmake): |
| 50 | |
| 51 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.cmake |
| 52 | # |
| 53 | # Make sure the reference backend is included in the build. |
| 54 | # By adding the subdirectory, cmake requires the presence of CMakeLists.txt |
| 55 | # in the reference (backend) folder. |
| 56 | # |
| 57 | add_subdirectory(${PROJECT_SOURCE_DIR}/src/backends/reference) |
| 58 | |
| 59 | # |
| 60 | # Add the cmake OBJECT libraries built by the reference backend to the |
| 61 | # list of libraries linked against the Arm NN shared library. |
| 62 | # |
| 63 | list(APPEND armnnLibraries armnnRefBackend armnnRefBackendWorkloads) |
| 64 | |
| 65 | # |
| 66 | # Backend specific unit tests can be integrated through the |
| 67 | # armnnUnitTestLibraries variable. This makes sure that the |
| 68 | # UnitTests executable can run the backend-specific unit |
| 69 | # tests. |
| 70 | # |
| 71 | list(APPEND armnnUnitTestLibraries armnnRefBackendUnitTests) |
| 72 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 73 | |
| 74 | ### The CMakeLists.txt file |
| 75 | |
| 76 | As described in the previous section, adding a new backend will require creating a `CMakeLists.txt` in |
| 77 | the backend folder. This follows the standard cmake conventions, and is required to build a static cmake OBJECT library |
| 78 | to be linked into the Arm NN shared library. As with any cmake build, the code can be structured into |
| 79 | subfolders and modules as the developer sees fit. |
| 80 | |
| 81 | Example can be found under [reference/CMakeLists.txt](reference/CMakeLists.txt). |
| 82 | |
| 83 | ### The backend.mk file |
| 84 | |
| 85 | Arm NN on Android uses the native Android build system. New backends are integrated by creating a |
| 86 | `backend.mk` file, which has a single variable called `BACKEND_SOURCES` listing all cpp |
| 87 | files to be built by the Android build system for the Arm NN shared library. |
| 88 | |
| 89 | Optionally, backend-specific unit tests can be added similarly, by |
| 90 | appending the list of cpp files to the `BACKEND_TEST_SOURCES` variable. |
| 91 | |
| 92 | Example taken from [reference/backend.mk](reference/backend.mk): |
| 93 | |
| 94 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.make |
| 95 | BACKEND_SOURCES := \ |
| 96 | RefLayerSupport.cpp \ |
| 97 | RefWorkloadFactory.cpp \ |
| 98 | workloads/Activation.cpp \ |
| 99 | workloads/ElementwiseFunction.cpp \ |
| 100 | workloads/Broadcast.cpp \ |
| 101 | ... |
| 102 | |
| 103 | BACKEND_TEST_SOURCES := \ |
| 104 | test/RefCreateWorkloadTests.cpp \ |
| 105 | test/RefEndToEndTests.cpp \ |
| 106 | test/RefJsonPrinterTests.cpp \ |
| 107 | ... |
| 108 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 109 | |
| 110 | @subsection S12_2_backend_developer_guide How to Add Common Code Across Backends |
| 111 | |
| 112 | For multiple backends that need common code, there is support for including them in the build |
| 113 | similarly to the backend code. This requires adding three files under a subfolder at the same level |
| 114 | as the backends folders. These are: |
| 115 | |
| 116 | 1. common.cmake |
| 117 | 2. common.mk |
| 118 | 3. CMakeLists.txt |
| 119 | |
| 120 | They work the same way as the backend files. The only difference between them is that |
| 121 | common code is built first, so the backend code can depend on them. |
| 122 | |
| 123 | [aclCommon](aclCommon) is an example for this concept and you can find the corresponding files: |
| 124 | |
| 125 | 1. [aclCommon/common.cmake](aclCommon/common.cmake) |
| 126 | 2. [aclCommon/common.mk](aclCommon/common.mk) |
| 127 | 3. [aclCommon/CMakeLists.txt](aclCommon/CMakeLists.txt) |
| 128 | |
| 129 | @subsection S12_3_backend_developer_guide Identifying Backends |
| 130 | |
| 131 | Backends are identified by a string that must be unique across backends. This string is |
| 132 | wrapped in the [BackendId](../../include/armnn/BackendId.hpp) object for backward compatibility |
| 133 | with previous Arm NN versions. |
| 134 | |
| 135 | @subsection S12_4_backend_developer_guide The IBackendInteral Interface |
| 136 | |
| 137 | All backends need to implement the [IBackendInternal](../../include/armnn/backends/IBackendInternal.hpp) interface. |
| 138 | The interface functions to be implemented are: |
| 139 | |
| 140 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.cpp |
| 141 | virtual IMemoryManagerUniquePtr CreateMemoryManager() const = 0; |
| 142 | virtual IWorkloadFactoryPtr CreateWorkloadFactory( |
| 143 | const IMemoryManagerSharedPtr& memoryManager = nullptr) const = 0; |
| 144 | virtual IBackendContextPtr CreateBackendContext(const IRuntime::CreationOptions&) const = 0; |
| 145 | virtual IBackendProfilingContextPtr CreateBackendProfilingContext(const IRuntime::CreationOptions& creationOptions, |
| 146 | armnn::profiling::IBackendProfiling& backendProfiling) const = 0; |
| 147 | virtual ILayerSupportSharedPtr GetLayerSupport() const = 0; |
| 148 | virtual Optimizations GetOptimizations() const = 0; |
| 149 | virtual SubgraphUniquePtr OptimizeSubgraph(const SubgraphView& subgraph, bool& optimizationAttempted) const; |
| 150 | virtual OptimizationViews OptimizeSubgraphView(const SubgraphView& subgraph) const; |
| 151 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 152 | |
| 153 | Note that `GetOptimizations()` and `SubgraphViewUniquePtr OptimizeSubgraphView(const SubgraphView& subgraph, bool& optimizationAttempted)` |
| 154 | have been deprecated. |
| 155 | The method `OptimizationViews OptimizeSubgraph(const SubgraphView& subgraph)` should be used instead to |
| 156 | apply specific optimizations to a given sub-graph. |
| 157 | |
| 158 | The Arm NN framework then creates instances of the IBackendInternal interface with the help of the |
| 159 | [BackendRegistry](../../include/armnn/BackendRegistry.hpp) singleton. |
| 160 | |
| 161 | **Important:** the `IBackendInternal` object is not guaranteed to have a longer lifetime than |
| 162 | the objects it creates. It is only intended to be a single entry point for the factory functions it has. |
| 163 | The best use of this is to be a lightweight, stateless object and make no assumptions between |
| 164 | its lifetime and the lifetime of the objects it creates. |
| 165 | |
| 166 | For each backend one needs to register a factory function that can |
| 167 | be retrieved using a [BackendId](../../include/armnn/BackendId.hpp). |
| 168 | The Arm NN framework creates the backend interfaces dynamically when |
| 169 | it sees fit and it keeps these objects for a short period of time. Examples: |
| 170 | |
| 171 | - During optimization Arm NN needs to decide which layers are supported by the backend. |
| 172 | To do this, it creates a backends and calls the `GetLayerSupport()` function and creates |
| 173 | an `ILayerSupport` object to help deciding this. |
| 174 | - During optimization Arm NN can run backend-specific optimizations. After splitting the graph into |
| 175 | sub-graphs based on backends, it calls the `OptimizeSubgraphView()` function on each of them and, if possible, |
| 176 | substitutes the corresponding sub-graph in the original graph with its optimized version. |
| 177 | - When the Runtime is initialized it creates an optional `IBackendContext` object and keeps this context alive |
| 178 | for the Runtime's lifetime. It notifies this context object before and after a network is loaded or unloaded. |
| 179 | - When the LoadedNetwork creates the backend-specific workloads for the layers, it creates a backend |
| 180 | specific workload factory and calls this to create the workloads. |
| 181 | |
| 182 | @subsection S12_5_backend_developer_guide The BackendRegistry |
| 183 | |
| 184 | As mentioned above, all backends need to be registered through the BackendRegistry so Arm NN knows |
| 185 | about them. Registration requires a unique backend ID string and a lambda function that |
| 186 | returns a unique pointer to the [IBackendInternal interface](../../include/armnn/backends/IBackendInternal.hpp). |
| 187 | |
| 188 | For registering a backend only this lambda function needs to exist, not the actual backend. This |
| 189 | allows dynamically creating the backend objects when they are needed. |
| 190 | |
| 191 | The BackendRegistry has a few convenience functions, like we can query the registered backends and |
| 192 | are able to tell if a given backend is registered or not. |
| 193 | |
| 194 | Dynamic backends are registered during the runtime creation. |
| 195 | |
| 196 | @subsection S12_6_backend_developer_guide The ILayerSupport Interface |
| 197 | |
| 198 | Arm NN uses the [ILayerSupport](../../include/armnn/ILayerSupport.hpp) interface to decide if a layer |
| 199 | with a set of parameters (i.e. input and output tensors, descriptor, weights, filter, kernel if any) are |
| 200 | supported on a given backend. The backends need a way to communicate this information by implementing |
| 201 | the `GetLayerSupport()` function on the `IBackendInternal` interface. |
| 202 | |
| 203 | Examples of this can be found in the [RefLayerSupport header](reference/RefLayerSupport.hpp) |
| 204 | and the [RefLayerSupport implementation](reference/RefLayerSupport.cpp). |
| 205 | |
| 206 | @subsection S12_7_backend_developer_guide The IWorkloadFactory Interface |
| 207 | |
| 208 | The [IWorkloadFactory interface](backendsCommon/WorkloadFactory.hpp) is used for creating the backend |
| 209 | specific workloads. The factory function that creates the IWorkloadFactory object in the IBackendInterface |
| 210 | takes an IMemoryManager object. |
| 211 | |
| 212 | To create a workload object the `IWorkloadFactory` takes a `WorkloadInfo` object that holds |
| 213 | the input and output tensor information and a workload specific queue descriptor. |
| 214 | |
| 215 | @subsection S12_8_backend_developer_guide The IMemoryManager Interface |
| 216 | |
| 217 | Backends may choose to implement custom memory management. Arm NN supports this concept through the following |
| 218 | mechanism: |
| 219 | |
| 220 | - the `IBackendInternal` interface has a `CreateMemoryManager()` function, which is called before |
| 221 | creating the workload factory |
| 222 | - the memory manager is passed to the `CreateWorkloadFactory(...)` function so the workload factory can |
| 223 | use it for creating the backend-specific workloads |
| 224 | - the LoadedNetwork calls `Acquire()` on the memory manager before it starts executing the network and |
| 225 | it calls `Release()` in its destructor |
| 226 | |
| 227 | @subsection S12_9_backend_developer_guide The Optimizations |
| 228 | |
| 229 | The backends may choose to implement backend-specific optimizations. |
| 230 | This is supported through the method `OptimizationViews OptimizeSubgraph(const SubgraphView& subgraph)` of |
| 231 | the backend interface that allows the backends to apply their specific optimizations to a given sub-graph. |
| 232 | |
| 233 | The `OptimizeSubgraph(...)` method returns an OptimizationViews object containing three lists: |
| 234 | |
| 235 | - A list of the sub-graph substitutions: a "substitution" is a pair of sub-graphs, the first is the "substitutable" sub-graph, |
| 236 | representing the part of the original graph that has been optimized by the backend, while the second is the "replacement" sub-graph, |
| 237 | containing the actual optimized layers that will be replaced in the original graph correspondingly to the "substitutable" sub-graph |
| 238 | - A list of the failed sub-graphs: these are the parts of the original sub-graph that are not supported by the backend, |
| 239 | thus have been rejected. Arm NN will try to re-allocate these parts on other backends if available. |
| 240 | - A list of the untouched sub-graphs: these are the parts of the original sub-graph that have not been optimized, |
| 241 | but that can run (unoptimized) on the backend. |
| 242 | |
| 243 | The previous way backends had to provide a list optimizations to the Optimizer (through the `GetOptimizations()` method) |
| 244 | is still in place for backward compatibility, but it's now considered deprecated and will be remove in a future release. |
| 245 | |
| 246 | @subsection S12_10_backend_developer_guide The IBackendContext Interface |
| 247 | |
| 248 | Backends may need to be notified whenever a network is loaded or unloaded. To support that, one can implement the optional |
| 249 | [IBackendContext](../../include/armnn/backends/IBackendContext.hpp) interface. The framework calls the `CreateBackendContext(...)` |
| 250 | method for each backend in the Runtime. If the backend returns a valid unique pointer to a backend context, then the |
| 251 | runtime will hold this for its entire lifetime. It then calls the following interface functions for each stored context: |
| 252 | |
| 253 | - `BeforeLoadNetwork(NetworkId networkId)` |
| 254 | - `AfterLoadNetwork(NetworkId networkId)` |
| 255 | - `BeforeUnloadNetwork(NetworkId networkId)` |
| 256 | - `AfterUnloadNetwork(NetworkId networkId)` |
| 257 | |
| 258 | @subsection S12_11_backend_developer_guide Dynamic Backends |
| 259 | |
| 260 | Backends can also be loaded by Arm NN dynamically at runtime. |
| 261 | To be properly loaded and used, the backend instances must comply to the standard interface for dynamic backends and to the versioning |
| 262 | rules that enforce ABI compatibility. |
| 263 | |
| 264 | @subsection S12_12_backend_developer_guide Dynamic Backends Base Interface |
| 265 | |
| 266 | The dynamic backend shared object must expose the following interface for Arm NN to handle it correctly: |
| 267 | |
| 268 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.cpp |
| 269 | extern "C" |
| 270 | { |
| 271 | const char* GetBackendId(); |
| 272 | void GetVersion(uint32_t* outMajor, uint32_t* outMinor); |
| 273 | void* BackendFactory(); |
| 274 | } |
| 275 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 276 | |
| 277 | Interface details: |
| 278 | |
| 279 | - `extern "C"` is needed to use avoid C++ name mangling, necessary to allow Arm NN to dynamically load the symbols. |
| 280 | - `GetBackendId()`: must return the unique id of the dynamic backends. |
| 281 | If at the time of the loading the id already exists in the internal Arm NN's backend registry, the backend will be skipped and |
| 282 | not loaded in Arm NN |
| 283 | - `GetVersion()`: must return the version of the dynamic backend. |
| 284 | The version must indicate the version of the Backend API the dynamic backend has been built with. |
| 285 | The current Backend API version can be found by inspecting the IBackendInternal interface. |
| 286 | At the time of loading, the version of the backend will be checked against the version of the Backend API Arm NN is built with. |
| 287 | If the backend version is not compatible with the current Backend API, the backend will not be loaded as it will be assumed that |
| 288 | it is not ABI compatible with the current Arm NN build. |
| 289 | - `BackendFactory()`: must return a valid instance of the backend. |
| 290 | The backend instance is an object that must inherit from the version of the IBackendInternal interface declared by GetVersion(). |
| 291 | It is the backend developer's responsibility to ensure that the backend implementation correctly reflects the version declared by |
| 292 | GetVersion(), and that the object returned by the BackendFactory() function is a valid and well-formed instance of the IBackendInternal |
| 293 | interface. |
| 294 | |
| 295 | @subsection S12_13_backend_developer_guide Dynamic Backend Versioning and ABI Compatibility |
| 296 | |
| 297 | Dynamic backend versioning policy: |
| 298 | |
| 299 | Updates to Arm NN's Backend API follow these rules: changes to the Backend API (the IBackendInternal interface) that break |
| 300 | ABI compatibility with the previous API version will be indicated by a change of the API's major version, while changes |
| 301 | that guarantee ABI compatibility with the previous API version will be indicated by a change in API's the minor version. |
| 302 | |
| 303 | For example: |
| 304 | |
| 305 | - Dynamic backend version 2.4 (i.e. built with Backend API version 2.4) is compatible with Arm NN's Backend API version 2.4 |
| 306 | (same version, backend built against the same Backend API) |
| 307 | - Dynamic backend version 2.1 (i.e. built with Backend API version 2.1) is compatible with Arm NN's Backend API version 2.4 |
| 308 | (same major version, backend built against earlier compatible API) |
| 309 | - Dynamic backend version 2.5 (i.e. built with Backend API version 2.5) is not compatible with Arm NN's Backend API version 2.4 |
| 310 | (same major version, backend built against later incompatible API, backend might require update to the latest compatible backend API) |
| 311 | - Dynamic backend version 2.0 (i.e. built with Backend API version 2.0) is not compatible with Arm NN's Backend API version 1.0 |
| 312 | (backend requires a completely new API version) |
| 313 | - Dynamic backend version 2.0 (i.e. built with Backend API version 2.0) is not compatible with Arm NN's Backend API version 3.0 |
| 314 | (backward compatibility in the Backend API is broken) |
| 315 | |
| 316 | @subsection S12_13_backend_developer_guide Dynamic Backend Loading Paths |
| 317 | |
| 318 | During the creation of the Runtime, Arm NN will scan a given set of paths searching for suitable dynamic backend objects to load. |
| 319 | A list of (absolute) paths can be specified at compile-time by setting a define named `DYNAMIC_BACKEND_PATHS` in the form of a colon-separated list of strings. |
| 320 | |
| 321 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.sh |
| 322 | -DDYNAMIC_BACKEND_PATHS="PATH_1:PATH_2...:PATH_N" |
| 323 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 324 | |
| 325 | The paths will be processed in the same order as they are indicated in the macro. |
| 326 | |
| 327 | It is also possible to override those paths at runtime when creating the Runtime object by setting the value of the `m_DynamicBackendsPath` member in the CreationOptions class. |
| 328 | Only one path is allowed for the override via the CreationOptions class. |
| 329 | By setting the value of the `m_DynamicBackendsPath` to a path in the filesystem, Arm NN will entirely ignore the list of paths passed via the |
| 330 | `DYNAMIC_BACKEND_PATHS` compiler directive. |
| 331 | |
| 332 | All the specified paths are validated before processing (they must exist, must be directories, and must be absolute paths), |
| 333 | in case of error a warning message will be added to the log, but Arm NN's execution will not be stopped. |
| 334 | If all paths are not valid, then no dynamic backends will be loaded in the Arm sNN's runtime. |
| 335 | |
| 336 | Passing an empty list of paths at compile-time and providing no path override at runtime will effectively disable the |
| 337 | dynamic backend loading feature, and no dynamic backends will be loaded into Arm NN's runtime. |
| 338 | |
| 339 | @subsection S12_14_backend_developer_guide Dynamic Backend File Naming Convention |
| 340 | |
| 341 | During the creation of a Runtime object, Arm NN will scan the paths specified for dynamic backend loading searching for suitable backend objects. |
| 342 | Arm NN will try to load only the files that match the following accepted naming scheme: |
| 343 | |
| 344 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.sh |
| 345 | <vendor>_<name>_backend.so[<version>] (e.g. "Arm_GpuAcc_backend.so" or "Arm_GpuAcc_backend.so.1.2.3") |
| 346 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 347 | |
| 348 | Only alphanumeric characters are allowed for both the `<vendor>` and the `<name>` fields, namely lowercase and/or uppercase characters, |
| 349 | and/or numerical digits (see the table below for examples). |
| 350 | Only dots and numbers are allowed for the optional `<version>` field. |
| 351 | |
| 352 | Symlinks to other files are allowed to support the standard linux shared object versioning: |
| 353 | |
| 354 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.sh |
| 355 | Arm_GpuAcc_backend.so -> Arm_GpuAcc_backend.so.1.2.3 |
| 356 | Arm_GpuAcc_backend.so.1 -> Arm_GpuAcc_backend.so.1.2.3 |
| 357 | Arm_GpuAcc_backend.so.1.2 -> Arm_GpuAcc_backend.so.1.2.3 |
| 358 | Arm_GpuAcc_backend.so.1.2.3 |
| 359 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 360 | |
| 361 | Files are identified by their full canonical path, so it is allowed to have files with the same name in different directories. |
| 362 | However, if those are actually the same dynamic backend, only the first in order of parsing will be loaded. |
| 363 | |
| 364 | Examples: |
| 365 | |
| 366 | | Filename | Description | |
| 367 | | -------------------------------------------------------- | ------------------------------------------------- | |
| 368 | | Arm_GpuAcc_backend.so | valid: basic backend name | |
| 369 | | Arm_GpuAcc_backend.so.1 | valid: single field version number | |
| 370 | | Arm_GpuAcc_backend.so.1.2 | valid: multiple field version number | |
| 371 | | Arm_GpuAcc_backend.so.1.2.3 | valid: multiple field version number | |
| 372 | | Arm_GpuAcc_backend.so.10.1.27 | valid: Multiple digit version | |
| 373 | | Arm_GpuAcc_backend.so.10.1.33. | not valid: dot not followed by version number | |
| 374 | | Arm_GpuAcc_backend.so.3.4..5 | not valid: dot not followed by version number | |
| 375 | | Arm_GpuAcc_backend.so.1,1.1 | not valid: comma instead of dot in the version | |
| 376 | | Arm123_GpuAcc_backend.so | valid: digits in vendor name are allowed | |
| 377 | | Arm_GpuAcc456_backend.so | valid: digits in backend id are allowed | |
| 378 | | Arm%Co_GpuAcc_backend.so | not valid: invalid character in vendor name | |
| 379 | | Arm_Gpu.Acc_backend.so | not valid: invalid character in backend id | |
| 380 | | GpuAcc_backend.so | not valid: missing vendor name | |
| 381 | | _GpuAcc_backend.so | not valid: missing vendor name | |
| 382 | | Arm__backend.so | not valid: missing backend id | |
| 383 | | Arm_GpuAcc.so | not valid: missing "backend" at the end | |
| 384 | | __backend.so | not valid: missing vendor name and backend id | |
| 385 | | __.so | not valid: missing all fields | |
| 386 | | Arm_GpuAcc_backend | not valid: missing at least ".so" at the end | |
| 387 | | Arm_GpuAcc_backend_v1.2.so | not valid: extra version info at the end | |
| 388 | | Arm_CpuAcc_backend.so | valid: basic backend name | |
| 389 | | Arm_CpuAcc_backend.so.1 -> Arm_CpuAcc_backend.so | valid: symlink to valid backend file | |
| 390 | | Arm_CpuAcc_backend.so.1.2 -> Arm_CpuAcc_backend.so.1 | valid: symlink to valid symlink | |
| 391 | | Arm_CpuAcc_backend.so.1.2.3 -> Arm_CpuAcc_backend.so.1.2 | valid: symlink to valid symlink | |
| 392 | | Arm_no_backend.so -> nothing | not valid: symlink resolves to non-existent file | |
| 393 | | pathA/Arm_GpuAcc_backend.so | valid: basic backend name | |
| 394 | | pathB/Arm_GpuAcc_backend.so | valid: but duplicated from pathA/ | |
| 395 | |
| 396 | Arm NN will try to load the dynamic backends in the same order as they are parsed from the filesystem. |
| 397 | |
| 398 | @subsection S12_15_backend_developer_guide Dynamic Backend Examples |
| 399 | |
| 400 | The source code includes an example that is used to generate some mock dynamic backends for testing purposes. The source files are: |
| 401 | |
| 402 | - TestDynamicBackend.hpp |
| 403 | - TestDynamicBackend.cpp |
| 404 | |
| 405 | This example is useful for going through all the use cases that constitute an invalid dynamic backend object, such as |
| 406 | an invalid/malformed implementation of the shared object interface, or an invalid value returned by any of the interface methods |
| 407 | that would prevent Arm NN from making use of the dynamic backend. |
| 408 | |
| 409 | A dynamic implementation of the reference backend is also provided. The source files are: |
| 410 | |
| 411 | - RefDynamicBackend.hpp |
| 412 | - RefDynamicBackend.cpp |
| 413 | |
| 414 | The implementation itself is quite simple and straightforward. Since an implementation of this particular backend was already available, |
| 415 | the dynamic version is just a wrapper around the original code that simply returns the backend id, version and an instance of the |
| 416 | backend itself via the factory function. |
| 417 | For the sake of the example, the source code of the reference backend is used to build the dynamic version (as you would for any new |
| 418 | dynamic backend), while all the other symbols needed are provided by linking the dynamic backend against Arm NN. |
| 419 | |
| 420 | The makefile used for building the reference dynamic backend is also provided: [CMakeLists.txt](dynamic/reference/CMakeLists.txt) |
| 421 | |
| 422 | A unit test that loads the reference backend dynamically and that exercises it is also included in the file |
| 423 | [DynamicBackendTests.cpp](dynamic/backendsCommon/test/DynamicBackendTests.cpp), by the test case `CreateReferenceDynamicBackend`. |
| 424 | In the test, a path on the filesystem is scanned for valid dynamic backend files (using the override option in `CreationOptions`) |
| 425 | where only the reference dynamic backend is. |
| 426 | In this example the file is named `Arm_CpuRef_backend.so`, which is compliant with the expected naming scheme for dynamic backends. |
| 427 | A `DynamicBackend` is created in the runtime to represent the newly loaded backend, then the backend is registered in the Backend |
| 428 | Registry with the id "CpuRef" (returned by `GetBackendId()`). |
| 429 | The unit test makes sure that the backend is actually registered in Arm NN, before trying to create an instance of the backend by |
| 430 | calling the factory function provided through the shared object interface (`BackendFactory()`). |
| 431 | The backend instance is used to verify that everything is in order, testing basic 2D convolution support by making use of the |
| 432 | Layer Support API and the Workload Factory. |
| 433 | At the end of test, the runtime object goes out of scope and the dynamic backend instance is automatically destroyed, and the handle to |
| 434 | the shared object is closed. |
| 435 | |
Ryan OShea | 2bbfaa7 | 2020-02-12 16:15:27 +0000 | [diff] [blame] | 436 | <br/><br/><br/><br/> |
Ryan OShea | f3a4323 | 2020-02-12 16:15:27 +0000 | [diff] [blame] | 437 | |
| 438 | @section S13_dynamic_backend_guide Standalone Dynamic Backend Developer Guide |
| 439 | |
| 440 | Arm NN allows adding new dynamic backends. Dynamic Backends can be compiled as standalone against Arm NN |
| 441 | and can be loaded by Arm NN dynamically at runtime. |
| 442 | |
| 443 | To be properly loaded and used, the backend instances must comply to the standard interface for dynamic backends |
| 444 | and to the versioning rules that enforce ABI compatibility. |
| 445 | The details of how to add dynamic backends can be found in [src/backends/README.md](../backends/README.md). |
| 446 | |
| 447 | @subsection S13_1_dynamic_backend_guide Standalone Dynamic Backend Example |
| 448 | |
| 449 | The source code includes an example that is used to generate a dynamic implementation of the reference backend |
| 450 | is provided at |
| 451 | - RefDynamicBackend.hpp |
| 452 | - RefDynamicBackend.cpp |
| 453 | |
| 454 | The makefile used for building the standalone reference dynamic backend is also provided: |
| 455 | CMakeLists.txt |
| 456 | |
| 457 | @subsection S13_2_dynamic_backend_guide Dynamic Backend Loading Paths |
| 458 | |
| 459 | During the creation of the Runtime, Arm NN will scan a given set of paths searching for suitable dynamic backend objects to load. |
| 460 | A list of (absolute) paths can be specified at compile-time by setting a define named `DYNAMIC_BACKEND_PATHS` |
| 461 | in the form of a colon-separated list of strings. |
| 462 | |
| 463 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.sh |
| 464 | -DDYNAMIC_BACKEND_PATHS="PATH_1:PATH_2...:PATH_N" |
| 465 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 466 | |
| 467 | The paths will be processed in the same order as they are indicated in the macro. |
| 468 | |
| 469 | **/ |
| 470 | } |