Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3.5: Model zoo enhancements - detailed catalog structure and model cards #1309

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

LeoSvalov
Copy link

@LeoSvalov LeoSvalov commented Aug 10, 2023

Model zoo improvements

The current models’ representation has an enumerative format in the model-list directory. The proposed model zoo structure adds a detailed catalog and has the following format:


image

For every integrated model in the new structure, the following components were added:

  1. Model card - The information about the model and basic theory behind its architecture, original paper, dataset description, and instructions on how to get inference, evaluate and profile the model.
  2. Standardized scripts - bash scripts that helps to use the model zoo models in unified format:
  • inference.sh: Get inference of the model by given input
  • performance.sh: Get performance metrics (FPS, E2E, DPU_MEAN)
  • quality.sh: Evaluate the quality of the model (metric is task-dependent)
  • vaitrace.sh: Profile the model by the vaitrace tool. (located in the model zoo root, in the /scripts directory)
    All scripts are made to provide ease of the model use and utilize the Vitis .cpp sources for the artifacts build.

Model directory example:


image

In total, there were 16 models integrated into the new proposed model zoo structure:

All integrated models have the configurations from the model-list that are supported by the Alveo V70 and each script for every model was tested on the Alveo V70 card device.

The integrated model directories cover 37 out of 103 of all models’ configurations supported V70 (36% coverage) that are presented in the model-list for the Xilinx v3.5 latest version.

Model cards integration into the GitHubIO documentation

To represent model cards in the GitHubIO documentation, but avoid the duplication of information, there several enhancements in the documentation build were introduced:

  • All markdown documentation files in the model zoo directory were exported transparently by the script build_model_cards.sh including Getting Started guide and model cards.
  • The integration script build_model_cards.sh was added to the Makefile commands so that no extra steps are required in the GitHubIO documentation build.

Example of the added content to the documentation:

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant