llama.cpp bindings and utilities for zig. Currently targeting zig 0.13.x.
- Provides build from source using zig build.
- Implements llama.h for nicer interaction with zig.
- Removes prefixes, changes naming for functions to camelCase. Groups functions within most appropriete struct. etc.
- Bindings partially depend on translate-c partially rewritten for ease of use.
- utilities:
- buffered Tokenizer & Detokenizer
Clone: git clone --recursive https://github.com/Deins/llama.cpp.zig.git
- Download llama.cpp supported model (usually *.gguf format). For example this one.
- build and run with:
zig build run-simple -Doptimize=ReleaseFast -- --model_path path_to/model.gguf --prompt "Hello! I am LLM, and here are the 5 things I like to think about:"Subset of llama cpp samples have been included in build scripts. Use -Dcpp_samples option to install them.
Or run them directly, for example: zig build run-cpp-main -Doptimize=ReleaseFast -- -m path/to/model.gguf -p "hello my name is"
- ☑️ x86_64 windows
- ☑️ x86_64 linux (WSL Ubuntu 22)
| Backend | Support | Comment |
|---|---|---|
| cpu | ☑️ | |
| cuda | ||
| metal | ||
| sycl | ||
| vulkan | ||
| opencl | ||
| cann | ||
| blas | ||
| rpc | ||
| kompute | ||
| CLBlast | ❌ | deprecated, was supported in older: 8798dea |