Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding new model #32

Open
samarth1612 opened this issue Sep 2, 2024 · 2 comments
Open

Adding new model #32

samarth1612 opened this issue Sep 2, 2024 · 2 comments

Comments

@samarth1612
Copy link

The below mentioned point where the link for GPTModel is broken and can't get the exact config for the yaml file hence not able to add new model (llama2-13b) as not able to profile the data for that model.

  1. Add a YAML model config for the new model in data/model_configs.
  • Use the model's HuggingFace model id for the file name eg. data/model_configs/meta-llama/Llama-2-70b-hf.yml.
  • Refer HuggingFace config.json for the model eg. https://huggingface.co/meta-llama/Llama-2-70b-hf/blob/main/config.json.
  • Ensure that correct parameters are set in the YAML file so that the reference transformer model GPTModel closely resembles the new model.
  • We use this reference model to profile only the MLP operations of all the models so the attention operations are no-op'ed here.

Would like to know if there is any solution for this as even the model_configs folder is not present in the base directory.

@AgrawalAmey
Copy link
Contributor

@anmolagarwalcp810 please share the updated instructions? and let's also make sure that we reflect those in the docs.

@samarth1612
Copy link
Author

Any update on this front ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants