This directory contains the scripts and data files to generate the conformance tests.
The data files are in JSON format and they describe what tests to create and/or select from the tosa_verif_build_tests
and tosa_verif_framework_*
generator scripts.
Naming: tosa_PPP_profile_ops_info.json
Contains a dictionary of operator names. Where PPP
is the profile subset of either base
for all integer tests, or main
for all floating point tests.
Each operator entry contains:
In the generation dictionary each entry is a name for a generation group - a set of tests generated together and then selected from using the selection criteria.
Each generation group is a dictionary that contains:
tosa_verif_build_tests
(see that tool for more details)In the selection dictionary each entry is a name for a selection criteria - there must be one called "default" which is used by default. Others may exist and be used by the different generation groups.
Each selection criteria is a dictionary that contains:
or (more information for each entry in test_select.py
):
DEPRECATED - not supported for conformance testing.
NOTE: Currently assumed all framework ops will be TFLite.
Naming: tosa_PPP_profile_framework_ops_info.json
Contains a dictionary of operator names. Where PPP
is the profile subset of either base
for all integer tests, or main
for all floating point tests.
Each operator entry contains:
Example:
"average_pool_2d": { "alternate_names": [ "avg_pool2d" ], "tests": [ "average_pool_2d_1x4x4x4_qi8_st11_padSAME_kern11", "average_pool_2d_1x4x8x19_qi16_st21_padSAME_kern22", "average_pool_2d_1x7x7x9_qi8_st22_padSAME_kern11", "average_pool_2d_1x32x32x8_qu8_st12_padVALID_kern12", "average_pool_2d_1x8x4x17_qu8_st21_padVALID_kern21" ], "profile": [ "tosa-bi", "tosa-mi" ] },