Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream merge 24 12 09 #314

Merged
merged 81 commits into from
Dec 9, 2024
Merged

Upstream merge 24 12 09 #314

merged 81 commits into from
Dec 9, 2024

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Dec 9, 2024

No description provided.

jeejeelee and others added 30 commits December 2, 2024 17:53
@jeejeelee
[Misc][LoRA] Move the implementation of lora bias to punica.py (vllm-…
b45f0d7 
…project#10829)

Signed-off-by: Jee Jee Li <[email protected]>
@yma11 @youkaichao
[Misc][XPU] Avoid torch compile for XPU platform (vllm-project#10747)
519cc6c 
Signed-off-by: yan ma <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@janimo
Fix openvino on GPU (vllm-project#10793)
9b14d97
@Isotr0py @DarkLight1337
[Model] Add TP and BNB quantization support to LlavaMultiModalProject…
4c05edb 
…or (vllm-project#10834)

Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@mgoin
[Bugfix] Prevent benchmark_throughput.py from using duplicated random…
4433195 
… prompts (vllm-project#10753)
@zixuanzhang226
[Model] support bitsandbytes quantization with minicpm model (vllm-pr…
d746268 
…oject#10842)

Signed-off-by: Ubuntu <[email protected]>
@jeejeelee
[Bugfix] Fix QKVParallelLinearWithShardedLora bias bug (vllm-project#…
a4cf256 
…10844)

Signed-off-by: Jee Jee Li <[email protected]>
@youkaichao
[core][distributed] add pynccl broadcast (vllm-project#10843)
21fe7b4 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[torch.compile] remove compilation_context and simplify code (vllm-pr…
dc5ce86 
…oject#10838)

Signed-off-by: youkaichao <[email protected]>
@russellb @DarkLight1337
[Doc] Add github links for source code references (vllm-project#10672)
ef51831 
Signed-off-by: Russell Bryant <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Misc] Remove deprecated names (vllm-project#10817)
3257d44 
Signed-off-by: DarkLight1337 <[email protected]>
@aarnphm @mgoin
[Core][Performance] Add XGrammar support for guided decoding and set …
9323a31 
…it as default (vllm-project#10785)

Signed-off-by: Aaron Pham <[email protected]>
Signed-off-by: mgoin <[email protected]>
Co-authored-by: mgoin <[email protected]>
@zhengy001
[Speculative Decoding] Move indices to device before filtering output (…
f6084f6 
…vllm-project#10850)

Co-authored-by: Yang Zheng(SW)(Alex) <[email protected]>
@alexm-neuralmagic @mgoin @ywang96
[V1] VLM - Run the mm_mapper preprocessor in the frontend process (vl…
3bc94ca 
…lm-project#10640)

Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@yma11
[MISC][XPU] quick fix for XPU CI (vllm-project#10859)
2f2cdc7 
Signed-off-by: yan ma <[email protected]>
@mgoin
[Bugfix] Only require XGrammar on x86 (vllm-project#10865)
7090c27 
Signed-off-by: mgoin <[email protected]>
@tomeras91
[Frontend] correctly record prefill and decode time metrics (vllm-pro…
7c32b68 
…ject#10853)

Signed-off-by: Tomer Asida <[email protected]>
@gshtras
[Build][Bugfix] Using the correct type hint (vllm-project#10866)
a061fe6 
Signed-off-by: Gregory Shtrasberg <[email protected]>
@xuechendi @aarnphm
[Benchmark] Benchmark structured output with datasets (vllm-project#1…
381ac93 
…0557)

Signed-off-by: Aaron Pham <[email protected]>
Signed-off-by: Chendi Xue <[email protected]>
Co-authored-by: Aaron Pham <[email protected]>
@tlrmchlsmth
[CI/Build] Replace mean with torch.all in test_pynccl.py (vllm-projec…
d2bd88b 
…t#10876)

Signed-off-by: Tyler Michael Smith <[email protected]>
@wangxiyuan
Drop ROCm load format check (vllm-project#10767)
b5b647b 
Signed-off-by: wangxiyuan <[email protected]>
@khluu
[ci/build] Change queue name for Release jobs (vllm-project#10875)
fa2dea6
@khluu
[ci/build] Job to build and push release image (vllm-project#10877)
c9ca4fc
@o2363286
[bugfix] fixed parameter “n” when set parameter “bestof” > 1 (vllm-pr…
8db957e 
…oject#10854)

Signed-off-by: jianzheng <[email protected]>
@khluu
[ci/build] Update vLLM postmerge ECR repo (vllm-project#10887)
c92acb9
@xyang16
[LoRA] Change lora_tokenizers capacity (vllm-project#10796)
01d079f 
Signed-off-by: Xin Yang <[email protected]>
@Isotr0py
[Model] Consolidate ViTs attention implementation without mask (vllm-…
10398b4 
…project#10893)

Signed-off-by: Isotr0py <[email protected]>
@xuechendi @mgoin
Benchmark serving structured output (vllm-project#10880)
82eb5ea 
Signed-off-by: Chendi Xue <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
@dtrifiro @youkaichao
[CI/Build] improve python-only dev setup (vllm-project#9621)
e4c34c2 
Signed-off-by: Daniele Trifirò <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@WoosukKwon
[V1] Fix when max_model_len is not divisible by block_size (vllm-proj…
2a56e12 
…ect#10903)

Signed-off-by: Woosuk Kwon <[email protected]>
DarkLight1337 and others added 27 commits December 7, 2024 17:10
@DarkLight1337
[Model] Update multi-modal processor to support Mantis(LLaVA) model (v…
39e227c 
…llm-project#10711)

Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Doc] Explicitly state that PP isn't compatible with speculative deco…
c889d58 
…ding yet (vllm-project#10975)

Signed-off-by: DarkLight1337 <[email protected]>
@xffxff
[BugFix][Kernel]: fix illegal memory access in causal_conv1d when con…
78029b3 
…v_states is None (vllm-project#10928)

Signed-off-by: xffxff <[email protected]>
@youkaichao
[core][executor] simplify instance id (vllm-project#10976)
1b62745 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[core][misc] remove use_dummy driver for _run_workers (vllm-project#1…
7be15d9 
…0920)

Signed-off-by: youkaichao <[email protected]>
@youkaichao
[torch.compile] allow candidate compile sizes (vllm-project#10984)
fd57d2b 
Signed-off-by: youkaichao <[email protected]>
@ywang96
[V1] Initial support of multimodal models for V1 re-arch (vllm-projec…
a11f326 
…t#10699)

Signed-off-by: Roger Wang <[email protected]>
@youkaichao
[torch.compile][misc] fix comments (vllm-project#10993)
43b05fa 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[misc] clean up and unify logging (vllm-project#10999)
46004e8 
Signed-off-by: youkaichao <[email protected]>
@ywang96
[Doc][V1] Add V1 support column for multimodal models (vllm-project#1…
af7c4a9 
…0998)

Signed-off-by: Roger Wang <[email protected]>
@youkaichao
[torch.compile] add dynamo time tracking (vllm-project#11005)
d1c2e15 
Signed-off-by: youkaichao <[email protected]>
@ywang96
[V1] Fix Detokenizer loading in AsyncLLM (vllm-project#10997)
c690357 
Signed-off-by: Roger Wang <[email protected]>
@russellb
[Core] Require xgrammar >= 0.1.6 (vllm-project#11021)
e691b26 
Signed-off-by: Russell Bryant <[email protected]>
@wangxiyuan
[Platform] Move async output check to platform (vllm-project#10768)
aea2fc3 
Signed-off-by: wangxiyuan <[email protected]>
@varun-sundar-rabindranath
[V1] Input Batch Relocation (vllm-project#10962)
25b79d9 
Signed-off-by: Varun Sundar Rabindranath <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]>
@khluu
[ci/build] Recompile CI dependencies list with Python 3.12 (vllm-proj…
edc4fa3 
…ect#11013)

Signed-off-by: kevin <[email protected]>
@WoosukKwon
[V1] Further reduce CPU overheads in flash-attn (vllm-project#10989)
3b61cb4 
Signed-off-by: Woosuk Kwon <[email protected]>
@jeejeelee
[Misc][LoRA] Abstract PunicaWrapper (vllm-project#10955)
ca87149 
Signed-off-by: Jee Jee Li <[email protected]>
@Isotr0py @DarkLight1337
[Model] merged input processor for Phi-3-Vision models (vllm-project#…
a811dd6 
…10977)

Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@kzawora-intel
[Bugfix][Hardware][Gaudi] Bump vllm_hpu_extension version (vllm-proje…
cbcbdb1 
…ct#11028)

Signed-off-by: Konrad Zawora <[email protected]>
@youkaichao
[v1] fix use compile sizes (vllm-project#11000)
1a2f8fb 
Signed-off-by: youkaichao <[email protected]>
@xendo
[Neuron] Upgrade neuron to 2.20.2 (vllm-project#11016)
9c6459e 
Signed-off-by: Jerzy Zagorski <[email protected]>
Co-authored-by: Jerzy Zagorski <[email protected]>
@gshtras
[ROCm][bugfix] scpecilative decoding worker class (vllm-project#11035)
b63ba84 
Signed-off-by: Gregory Shtrasberg <[email protected]>
@gshtras
Merge remote-tracking branch 'upstream/main' into develop
7c61516
@gshtras
format
401a541
@gshtras
Merge remote-tracking branch 'origin/main' into upstream_merge_24_12_09
c9f5c24
@gshtras
Merge remote-tracking branch 'upstream/main' into upstream_merge_24_1…
c324ea8 
…2_09
shajrawi
shajrawi approved these changes Dec 9, 2024
View reviewed changes
Copy link
Collaborator

@shajrawi shajrawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ship it

@gshtras gshtras merged commit dcc3f45 into main Dec 9, 2024
11 of 13 checks passed
@gshtras gshtras deleted the upstream_merge_24_12_09 branch December 9, 2024 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.