MLBEDSW-4783: Add config file directory structure
Add directory structure to support third party config files. Config
files should now be placed in an appropriately named directory under
the config_files directory, but can also be accessed by providing its
absolute path to vela --config.
Signed-off-by: Rickard Bolin <rickard.bolin@arm.com>
Change-Id: I2fcf52e7b2ddd2c4491dc370c85c0b3937d18062
diff --git a/OPTIONS.md b/OPTIONS.md
index e24f963..b4132d3 100644
--- a/OPTIONS.md
+++ b/OPTIONS.md
@@ -66,6 +66,19 @@
vela --supported-ops-report
```
+### List Configuration Files
+
+Displays the configuration files in the `ethosu/config_files` directory. All
+configuration files must have the .ini extension and be placed in an
+appropriately named directory under `ethosu/config_files`. Note that the file
+depth from `ethosu/config_files` must be exactly 2 for the file to be
+discovered (e.g. `config_files/directory_name/my_config_file.ini`). Can be
+used without the required Network argument.
+
+```bash
+vela --list-config-files
+```
+
### Output Directory
Specifies the output directory of the optimised network model as well as the
@@ -93,12 +106,15 @@
Specifies the path to the Vela configuration file. The format of the file is a
Python ConfigParser `.ini` file. This option can be specified multiple times to
allow multiple files to be searched for the required system config and memory
-mode. More details can be found in the Configuration File section below.
+mode. Custom configuration files can be used by adding a .ini file in an
+appropriate directory under the `ethosu/config_files` directory or by providing
+its absolute path. More details can be found in the Configuration File and List
+Configuration Files sections.
**Type: POSIX path**
**Default: use default configuration**
```bash
-vela network.tflite --config my_vela_cfg1.ini --config my_vela_cfg2.ini --system-config My_Sys_Cfg --memory-mode My_Mem_Mode
+vela network.tflite --config DirectoryName/my_vela_cfg1.ini --config absolute/path/to/my_vela_cfg2.ini --system-config My_Sys_Cfg --memory-mode My_Mem_Mode
```
### Timing
@@ -400,11 +416,25 @@
CLI option. This can also be used to display the internal-default values and to
see a full list of all the available options.
-An example Vela configuration file, called `vela_cfg.ini`, is included in the
-directory containing this file. Example usage based on this file is:
+An example Vela configuration file, called `vela.ini`, is included in the
+`ethosu/config_files/Arm` directory. Example usage based on this file is:
```bash
-vela network.tflite --accelerator-config ethos-u55-256 --config vela_cfg.ini --system-config Ethos_U55_High_End_Embedded --memory-mode Shared_Sram
+vela network.tflite --accelerator-config ethos-u55-256 --config Arm/vela.ini --system-config Ethos_U55_High_End_Embedded --memory-mode Shared_Sram
+```
+
+Hardware vendors and/or users may wish to contribute their own configuration
+files for various SoC platforms by adding a .ini file in an appropriate
+directory under the ethosu/config_files directory. This can be done by
+following the process outlined in CONTRIBUTIONS.md. These can then be accessed
+with `--config <DirectoryName>/config.ini` as in the example above.
+
+To use configuration files located outside the config_files directory, provide
+its absolute path to `--config`. The `--list-config-files` option can be used to
+view all available configuration files:
+
+```bash
+vela --list-config-files
```
The following is an in-line explanation of the Vela configuration file format:
diff --git a/README.md b/README.md
index eb8871e..9ed3602 100644
--- a/README.md
+++ b/README.md
@@ -163,13 +163,20 @@
7) Compile a network using a particular embedded system configuration defined in
Vela's configuration file. The following command selects the `My_Sys_Config`
-system configuration along with the `My_Mem_Mode` memory mode from the `vela_cfg.ini` configuration file.
+system configuration along with the `My_Mem_Mode` memory mode from the `vela.ini`
+configuration file located in the config_files directory.
```bash
-vela --config vela_cfg.ini --system-config My_Sys_Config --memory-mode My_Mem_Mode my_model.tflite
+vela --config Arm/vela.ini --system-config My_Sys_Config --memory-mode My_Mem_Mode my_model.tflite
```
-8) To get a list of all available options (see CLI Options section below):
+8) To get a list of all available configuration files in the config_files directory:
+
+```bash
+vela --list-config-files
+```
+
+9) To get a list of all available options (see CLI Options section below):
```bash
vela --help
diff --git a/vela.ini b/ethosu/config_files/Arm/vela.ini
similarity index 100%
rename from vela.ini
rename to ethosu/config_files/Arm/vela.ini
diff --git a/ethosu/vela/vela.py b/ethosu/vela/vela.py
index 2108e20..b5f9f98 100644
--- a/ethosu/vela/vela.py
+++ b/ethosu/vela/vela.py
@@ -18,6 +18,7 @@
#
# Provides command line interface, options parsing, and network loading. Before calling the compiler driver.
import argparse
+import glob
import os
import sys
import time
@@ -49,6 +50,8 @@
from .tosa_supported_operators import TosaSupportedOperators
from ethosu.vela.architecture_features import ArchitectureFeatures
+CONFIG_FILES_PATH = os.path.abspath(f"{__file__}/../../config_files")
+
def process(input_name, enable_debug_db, arch, model_reader_options, compiler_options, scheduler_options):
if compiler_options.timing:
@@ -282,6 +285,12 @@
print(f"Report file: {filepath}")
+def list_config_files():
+ print("\nAvailable config files:\n")
+ for config in glob.glob(os.path.join(CONFIG_FILES_PATH, "*", "*.ini")):
+ print(config.lstrip(CONFIG_FILES_PATH + os.path.sep))
+
+
def main(args=None):
try:
if args is None:
@@ -298,6 +307,15 @@
help="Generate the SUPPORTED_OPS.md file in the current working directory and exit",
)
+ parser.add_argument(
+ "--list-config-files",
+ action="store_true",
+ help=(
+ "Display all available configurations in the `config_files` folder and exit. To select config file, "
+ "use the --config argument with one of the listed config files (For example: --config Arm/vela.ini )"
+ ),
+ )
+
# set network nargs to be optional to allow the support-ops-report CLI option to be used standalone
parser.add_argument(
"network",
@@ -428,14 +446,36 @@
generate_supported_ops()
return 0
+ if args.list_config_files:
+ list_config_files()
+ return 0
+
if args.network is None:
parser.error("the following argument is required: NETWORK")
+ def _parse_config(config):
+ if not config.endswith(".ini"):
+ raise InputFileError(config, "Configuration files must use the .ini extension")
+
+ if len(config.split(os.path.sep)) == 2 and not config.startswith(os.path.sep):
+ config_path = os.path.join(CONFIG_FILES_PATH, config)
+ else:
+ print(
+ f"Warning: Configuration file `{config}` not located in a folder directly under the "
+ "`config_files` directory. Note that the file depth from the `config_files` directory must be "
+ "exactly 2 to be discovered via the --list-config-files mechanism "
+ "(e.g. `config_files/directory_name/my_config_file.ini`). This config file will still be "
+ "parsed however, and treated as an absolute path config instead."
+ )
+ config_path = config
+
+ if not os.access(config_path, os.R_OK):
+ raise InputFileError(config_path, "File not found or is not readable")
+
+ return config_path
+
# check all config files exist because they will be read as a group
- if args.config is not None:
- for filename in args.config:
- if not os.access(filename, os.R_OK):
- raise InputFileError(filename, "File not found or is not readable")
+ config_files = [_parse_config(cfg) for cfg in args.config] if args.config else None
if args.cpu_tensor_alignment < 16 or args.cpu_tensor_alignment & (args.cpu_tensor_alignment - 1) != 0:
parser.error(
@@ -457,7 +497,7 @@
sys.setrecursionlimit(args.recursion_limit)
arch = architecture_features.ArchitectureFeatures(
- vela_config_files=args.config,
+ vela_config_files=config_files,
system_config=args.system_config,
memory_mode=args.memory_mode,
accelerator_config=args.accelerator_config,
diff --git a/setup.py b/setup.py
index 244fc4e..d6e03b1 100644
--- a/setup.py
+++ b/setup.py
@@ -83,12 +83,14 @@
],
keywords=["ethos-u", "vela compiler", "tflite", "npu"],
packages=[
+ "ethosu",
"ethosu.vela",
"ethosu.vela.ethos_u55_regs",
"ethosu.vela.tflite",
"ethosu.vela.tosa",
"ethosu.mlw_codec",
],
+ package_data={"ethosu": ["config_files/*/*.ini"]},
python_requires="~=3.7",
install_requires=[
"flatbuffers==1.12.0",