From 430ca44ae2803ed3f7c669f7453d70ab701cda5d Mon Sep 17 00:00:00 2001 From: Lei Wang <34334180+LeiWang1999@users.noreply.github.com> Date: Fri, 20 Dec 2024 00:41:12 +0800 Subject: [PATCH] Update docs/source/quantization/bitblas.rst Co-authored-by: Michael Goin --- docs/source/quantization/bitblas.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/source/quantization/bitblas.rst b/docs/source/quantization/bitblas.rst index e01598a4416d1..bc2074b2556f8 100644 --- a/docs/source/quantization/bitblas.rst +++ b/docs/source/quantization/bitblas.rst @@ -30,7 +30,8 @@ Read bitblas format checkpoint. llm = LLM(model=model_id, dtype=torch.bfloat16, trust_remote_code=True, quantization="bitblas") Read gptq format checkpoint. --------------------------- +------------------------------- + .. code-block:: python from vllm import LLM