v3.5: Model zoo enhancements - detailed catalog structure and model cards #1309
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Model zoo improvements
The current models’ representation has an enumerative format in the model-list directory. The proposed model zoo structure adds a detailed catalog and has the following format:
For every integrated model in the new structure, the following components were added:
inference.sh
: Get inference of the model by given inputperformance.sh
: Get performance metrics (FPS, E2E, DPU_MEAN)quality.sh
: Evaluate the quality of the model (metric is task-dependent)vaitrace.sh
: Profile the model by the vaitrace tool. (located in the model zoo root, in the /scripts directory)All scripts are made to provide ease of the model use and utilize the Vitis .cpp sources for the artifacts build.
Model directory example:
In total, there were 16 models integrated into the new proposed model zoo structure:
All integrated models have the configurations from the model-list that are supported by the Alveo V70 and each script for every model was tested on the Alveo V70 card device.
The integrated model directories cover 37 out of 103 of all models’ configurations supported V70 (36% coverage) that are presented in the model-list for the Xilinx v3.5 latest version.
Model cards integration into the GitHubIO documentation
To represent model cards in the GitHubIO documentation, but avoid the duplication of information, there several enhancements in the documentation build were introduced:
build_model_cards.sh
including Getting Started guide and model cards.build_model_cards.sh
was added to theMakefile
commands so that no extra steps are required in the GitHubIO documentation build.Example of the added content to the documentation: