Skip to content

Commit

Permalink
Update preload.py
Browse files Browse the repository at this point in the history
  • Loading branch information
justinwlin authored Feb 18, 2024
1 parent 1b81c35 commit 7daa19a
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions preload.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,6 @@
# Use the MODEL environment variable; fallback to a default if not set
model_name = os.getenv('MODEL', 'mistralai/Mistral-7B-Instruct-v0.1')

print("Model name: ", model_name)

llm = openllm.LLM(model_name, backend="vllm")

0 comments on commit 7daa19a

Please sign in to comment.