Skip to content

Commit

Permalink
use sqlcoder-7b-2 as default example
Browse files Browse the repository at this point in the history
  • Loading branch information
rishsriv authored Feb 23, 2024
1 parent 9faf4fa commit a36190f
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ python -W ignore main.py \
-o results/results.csv \
-g hf \
-f prompts/prompt.md \
-m defog/sqlcoder2 \
-m defog/sqlcoder-7b-2 \
-n 10
```
We also support loading a peft adapter here as well via the `-a` flag. Note that the loading of the adapter with the model will take slightly longer than usual.
Expand All @@ -168,7 +168,7 @@ python -W ignore main.py \
-o "results/results.csv" \
-g vllm \
-f "prompts/prompt.md" \
-m defog/sqlcoder2
-m defog/sqlcoder-7b-2
```

Optionally, if you're running evals on a model that is quantized with AWQ, add the `-qz` or `--quantized` parameter. Only applicable for the vllm runner.
Expand All @@ -177,7 +177,7 @@ If running with different settings, you can setup an api server to avoid reloadi
```bash
# to set up a vllm server
python -m vllm.entrypoints.api_server \
--model defog/sqlcoder-70b-alpha \
--model defog/sqlcoder-7b-2 \
--tensor-parallel-size 4 \
--dtype float16

Expand Down

0 comments on commit a36190f

Please sign in to comment.