Skip to content

Latest commit

 

History

History
22 lines (14 loc) · 1.42 KB

CONTRIBUTING.md

File metadata and controls

22 lines (14 loc) · 1.42 KB
Raw

Contributing to LiveBench

Thanks for taking the time to contribute!

Reporting bugs

To report a bug, please open an issue or email the authors. When you are creating a bug report, please include as many details as possible. For example, describe the bug, include a small script to reproduce the bug, and give the expected behavior of the script.

Code contribution

Contributions are welcome! Feel free to open a pull request to the main branch.

Model contribution

We'd be happy to add more models to the leaderboard (depending on demand). Please send your model or evals requests to [email protected] in one of these formats

  • Huggingface model, while also telling us the fastchat model adapter to use
  • API endpoint, ideally using the OpenAI api format (If not, please submit a PR with the necessary code changes to run this model)
  • The above methods are preferred, but you can also send instructions on how to run your model if it differs from the above methods.
  • If you want to run the model yourself, please evaluate it using the commands in the README, and then send us the model_answer jsonl files. You can simply zip the data/live_bench folder that is created when you evaluate your model.