From 2ba712465c2ca2cf4e6257e749e006647f0faa08 Mon Sep 17 00:00:00 2001 From: Ying Sheng Date: Thu, 25 Jul 2024 20:01:09 -0700 Subject: [PATCH] minor wording adjustment --- blog/2024-07-25-sglang-llama3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/2024-07-25-sglang-llama3.md b/blog/2024-07-25-sglang-llama3.md index 672f3b94..fb88bd41 100644 --- a/blog/2024-07-25-sglang-llama3.md +++ b/blog/2024-07-25-sglang-llama3.md @@ -15,7 +15,7 @@ SGLang is an open-source project licensed under the Apache 2.0 license. It has b ## Benchmark Setup -> **Update (2024-07-25 7PM):** We've identified issues in our synthetic dataset generation pipeline, resulting in mostly short prompts. While this means the datasets don't match our earlier descriptions, the comparison remains fair since all engines are benchmarked under the same conditions. We've updated the benchmark setup description to reflect the characteristics of the generated synthetic datasets. We're working on obtaining more benchmark results for longer prompts, but we expect the speedup of SGLang to be less since it primarily accelerates the decoding phase. +> **Update (2024-07-25 7PM PST):** We've identified issues in our synthetic dataset generation pipeline, resulting in mostly short prompts. While this means the datasets don't match our earlier descriptions, the comparison remains fair since all engines are benchmarked under the same conditions. We've updated the benchmark setup description below to reflect the characteristics of the generated synthetic datasets. We're working on obtaining more benchmark results for longer prompts, but we expect the speedup of SGLang to be less for long prompts since it primarily accelerates the decoding phase. We benchmark both offline and online use cases: