Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
justinwlin committed Feb 18, 2024
1 parent 5ab44e0 commit e6ebb6b
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,20 @@
This Docker configuration utilizes [OpenLLM](https://github.com/bentoml/OpenLLM) for both GPU and Serverless deployments on Runpod. It employs an environment variable, `MODE_TO_RUN`, to dictate the startup behavior. Depending on the `MODE_TO_RUN` value, the configuration may launch `handler.py` for serverless operations or initiate OpenSSH and Jupyter Lab for GPU pods. This adaptable setup allows for straightforward modifications to meet various deployment requirements.

## Avaliable Models on CI/CD
https://hub.docker.com/r/justinwlin/mistral7b_openllm/tags
Llama2 70B:
https://runpod.io/gsc?template=iw1bjna1tc&ref=wqryvm1m
https://hub.docker.com/r/justinwlin/llama2_70b_openllm/tags

Mixtral 8x7B:
https://runpod.io/gsc?template=od8336xqgo&ref=wqryvm1m
https://hub.docker.com/r/justinwlin/mixtral_8x7b_openllm/tags

Mistral:
https://runpod.io/gsc?template=pu8uaqw765&ref=wqryvm1m
https://hub.docker.com/r/justinwlin/mistral7b_openllm/tags

OpenLLM Llama2 13b - Pod and Serverless Ready
https://runpod.io/gsc?template=0dok49hohe&ref=wqryvm1m
https://hub.docker.com/r/justinwlin/llama2_13b_openllm/tags

If you use them make sure to always specify fully with the tag in Runpod `justinwlin/llama2_13b_openllm:latest`
Expand Down

0 comments on commit e6ebb6b

Please sign in to comment.