[Core] Update to outlines >= 0.1.8

[russellb]

This PR updates to the latest release of outlines that works with vllm.

It is a draft while we wait for 0.1.8 to be on pypi.

FIX #3794
FIX #10489

279ccc9 [Core] Update to outlines >= 0.1.8

commit 279ccc9
Author: Russell Bryant [email protected]
Date: Thu Nov 21 21:25:22 2024 +0000

[Core] Update to outlines >= 0.1.8

0.1.x prior to 0.1.8 + outlines-core 0.1.18 had issues with
serialization that broke vllm integration.

Also change our code slightly to account for an API change in
outlines.

Signed-off-by: Russell Bryant <[email protected]>

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[russellb]

0.1.5 has been released, so this is ready to test

[russellb]

converted back to a draft because I don't have 0.1.5 working locally, yet ...

[joennlae]

I introduce this PR to outlines:

dottxt-ai/outlines-core#99

The caching issue (or not being able to pickle) also affects the multiprocessing-based engine in vllm as we need to be able to pickle the RegexGuide. I introduce the pickling capability with the above PR.

References:

vllm/vllm/engine/multiprocessing/client.py

Line 604 in 7e4bbda

lp_bytes = cloudpickle.dumps(logits_processors)

[markmc]

I introduce this PR to outlines:

dottxt-ai/outlines-core#99

The caching issue (or not being able to pickle) also affects the multiprocessing-based engine in vllm as we need to be able to pickle the RegexGuide. I introduce the pickling capability with the above PR.

Thank you @joennlae - this works for me

Here are some performance measurements with (a) older outlines, (b) newer outlines with your patch but no caching, (c) newer outlines with your patch and caching reinstated:

(a) commit f9310cbd0c1109c4f22cf9f1dc615b2d08f06408 from main
   with	pull/10557/head:xuechendi-benchmark-structured-output commit 41966603
   outlines==0.0.46

   $ python benchmarks/benchmark_guided.py --async-engine --model meta-llama/Llama-3.2-1B-Instruct --output-len 2048 --num-prompts 10 --save-results
   Throughput: 0.14 requests/s, 361.06 total tokens/s, 281.59 output tokens/s Correct rate is 100.0 %
   Throughput: 0.16 requests/s, 414.12 total tokens/s, 322.97 output tokens/s Correct rate is 100.0 %
   Throughput: 0.16 requests/s, 414.91 total tokens/s, 323.59 output tokens/s Correct rate is 100.0 %

(b) commit b8c2895a2cbe249e86713615c9ed3ab132812b08 from russellb/outlines-0.1.4
   with pull/10557/head:xuechendi-benchmark-structured-output commit 41966603
   outlines==0.1.7 (from PyPi)
   outlines_core 412ef296392a0814a5490ccc15080e79f98cd411 from 44ai-labs/serialization "release build" (pip install .)

   $ python benchmarks/benchmark_guided.py --async-engine --model meta-llama/Llama-3.2-1B-Instruct --output-len 2048 --num-prompts 10 --save-results
   Throughput: 0.06 requests/s, 167.18 total tokens/s, 130.38 output tokens/s Correct rate is 100.0 %
   Throughput: 0.06 requests/s, 167.48 total tokens/s, 130.61 output tokens/s Correct rate is 100.0 %
   Throughput: 0.07 requests/s, 171.11 total tokens/s, 133.45 output tokens/s Correct rate is 100.0 %

(c)   same	as above, with "Stop using outlines.caching.cache" reverted
   Throughput: 0.16 requests/s, 410.16 total tokens/s, 319.88 output tokens/s Correct rate is 100.0 %
   Throughput: 0.15 requests/s, 406.16 total tokens/s, 316.76 output tokens/s Correct rate is 100.0 %
   Throughput: 0.16 requests/s, 408.68 total tokens/s, 318.73 output tokens/s Correct rate is 100.0 %

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @russellb.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[russellb]

I think this will be ready once outlines 0.1.8 is available on pypi.

[rlouf]

It is !

[mgoin]

LGTM pending green CI!