Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
clebert committed Oct 19, 2023
1 parent ef60432 commit f560ee8
Showing 1 changed file with 9 additions and 1 deletion.
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,15 @@ python3 convert_hf_model.py /path/to/Llama-2-7b-hf models/llama2_7b_hf

```sh
zig build -Doptimize=ReleaseFast
./zig-out/bin/llama2-generator models/llama2_7b_hf --prompt "Once Upon a Time"
./zig-out/bin/llama2-generator models/llama2_7b_hf --temperature 0 --sequence_length 28 --prompt "Once Upon a Time" --verbose
```

The output on an Apple M1 Pro with 32 GB of memory:

```
Once Upon a Time in Hollywood is a 2019 American comedy-drama film written and directed by Quentin Tarantino
achieved: 1.797 tok/s
```

## Help
Expand Down

0 comments on commit f560ee8

Please sign in to comment.