Vela Community Bug Reporting

If the Vela community encounters a bug, then we highly encourage the community to report it. When issuing a report, use the Maniphest bug tracker that is part of Phabricator on ML Platform. This site enables any Vela user to discuss and share any Vela related issue with the community.

The only prerequisite for this process is a GitHub account, which will be used when creating an account on ML platform.

If you also have a fix for the bug you have identified, then we encourage you to follow the contributions guide.

The bug reporting process in its entirety is detailed below.

Maniphest bug tracker

ML Platform uses a bug tracking system known as Maniphest, which is part of the Phabricator tool set.

On the Maniphest platform, one can add active bugs using the Tasks and Bugs tab which will then be addressable by the Vela community.

The system is accessible directly via the link above. Alternatively, one can navigate to ML Platform and select the "Developers" drop down menu followed by "Development Tracker", which will bring you to Phabricator. In order to initiate a bug report, select "Tasks and Bugs".

Anyone can browse the Maniphest bug tracker, since it is a public forum. However, in order to file your own ticket you need to be logged in to an account on ML Platform. These can be created by clicking "Register" at https://developer.mlplatform.org/auth/ and then linking a GitHub ID.

Proprietary or Sensitive Information

As mentioned previously, ML Platform and the Maniphest Bug tracking system are public forums. As such, anyone can see information posted to them. Therefore, any proprietary information you wish to share should be done using Arm's ML Model Review described below.

Bug Reporting Process

The entire process of handling a bug is summarized in the following steps.

  1. Discovering the bug

    • A user notices a discrepancy in the functionality of Vela. This can be as simple as Vela crashing upon runtime, where an error message is printed out for the user to read. However, in certain cases the bug is much harder to spot. For example, Vela might run gracefully but optimize a network incorrectly. Thus the error will manifest in a silent error, such as incorrect outputs. In this case, as much information about the system and settings should be noted. This will be used later on when handling the root cause. A sufficient documentation of the problem is crucial when dealing with these silent errors.

  2. Sharing the existence of the bug

    • In order to resolve the issue, its existence needs to be shared with the Vela community. Even though the user could fix it by themselves in the first step, it is important that is is shared with the community. This step will be detailed in the following sections.

  3. The community tackles the problem

    • When the bug has been properly detailed, the Vela community will try to find a solution. During this phase, further information may be requested or shared through the corresponding bug ticket. If a solution is found, the ticket will be updated with the relevant information.

As mentioned, the second step is detailed below.

Opening the form

To add a new bug report, navigate to the Bug tracker.

Click the button "+ Create Task" or navigate to the following page

Title

Write a title that describes the problem in a short and meaningful way. An example of a good title is "AssertionError in weight_compressor.py when optimizing DS-CNN". An example of a poor title is "Vela crashes on Linux".

Assigned To

Leave this field blank, the report will be assigned when work is started on it.

Status

If you were able to resolve the bug before submission and this ticket is simply a notification to the Vela Community, select the "Resolved" option. Also, follow the contributions guide in order to apply your fix. If the bug has not yet been resolved, select the "Open" option instead.

Description

This section should contain as much information as possible, in order to allow someone else to reproduce the issue.

Please provide the following details:

  • Vela command line and options.

  • Configuration files. If these contain information that you do not wish to make public then please upload them using Arm's ML Model Review process described below.

  • Version of Vela used.

  • Error messages or other console output from Vela.

  • Link to a publicly accessible copy of the input network. If this is not possible then you will need to upload the network using Arm's ML Model Review process described below.

  • License files related to your network, if applicable.

Attached to the description you can find a toolbox, which contains a button to upload files. This shall not be used to upload any network files, instead use ML Model Review.

Visible To

ML Platform and the Manifest Bug Tracking System are public forums. As such, anyone can see information posted to them.

Thus, any proprietary information you wish to share should be done using Arm's ML Model Review. described below.

Hence, the content of the report should be freely viewable to the public and thus the "Public" option should be selected in the "Visility" field.

Editable by

In this field, restrict the editing privileges to yourself and members of the Vela Optimizer group. This is done by selecting "Subscribers" in the "Object Policies" entry of the "Editable by" drop down menu.

Proceed by navigating down to the "Subscribers" field and typing in both your username and "Vela Optimizer". To clarify, your username and "Vela Optimizer" shall be separate entries in the "Subscribers" field.

Tags

Finally, add "Vela Optimizer" in the "Tags" field. This ensures that the bug is directed to the Vela team, enabling a rapid triage of the problem.

Submitting the report

Submit the report by clicking the "Create New Task" button.

A confirmation e-mail will be sent out to the linked e-mail address. Any further changes to the report will be sent out in the same way.

Arm's ML Model Review process.

As mentioned earlier, proprietary or sensitive information should not be shared on this page. Instead, any required sensitive content should be sent directly to Arm via Arm's ML Model Review process.

For instance, if you can't find any publicly hosted examples of your network, e.g. a network hosted on Model Zoo, upload your own network to ML Model Review.

It is a simple process that requires you to complete a short online form after which you will be sent instructions on how to upload your files.