Fix: Build error seen on Power Architecture #10421

mikejuliet13 · 2024-11-18T13:14:09Z

The issue in the vLLM library involves the introduction of the -mf16c flag and FP16 vector types, which are specific to x86 architecture. These changes cause build failures on Power architecture, though the library builds and functions correctly on x86. To resolve this, architecture-specific checks will be implemented. The -mf16c flag will not be applied on the Power architecture systems, while FP32Vec types will be used as a fallback for unsupported FP16 vector types on Power architecture. This fix ensures compatibility and successful builds across different architectures.

github-actions · 2024-11-18T13:14:29Z

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

Add ready label to the PR
Enable auto-merge.

🚀

sumitd2 · 2024-11-18T13:35:13Z

cc: @khluu

simon-mo · 2024-11-18T18:01:00Z

Please fix clang-format

mgoin · 2024-11-18T18:52:40Z

cmake/cpu_extension.cmake

+if (CMAKE_SYSTEM_PROCESSOR STREQUAL "ppc64le")
+    list(APPEND CXX_COMPILE_FLAGS
+    "-fopenmp"
+    "-DVLLM_CPU_EXTENSION")
+else()
 list(APPEND CXX_COMPILE_FLAGS
    "-fopenmp"
    "-mf16c"
    "-DVLLM_CPU_EXTENSION")
+endif()


Maybe you could invert this to always apply

list(APPEND CXX_COMPILE_FLAGS "-fopenmp" "-DVLLM_CPU_EXTENSION")

And then add a case just if the processor is x86 to add "-mf16c"

similar to this block -

vllm/cmake/cpu_extension.cmake

Line 57 in 284203f

if (AVX512_FOUND AND NOT AVX512_DISABLED)

Above flag was applied for all architectures. We know for Power its not applicable so added exclusive check to avoid it for Power.
Adding exclusive check for x86 may avoid it applying for other supported architectures.

x86 is the only architecture that supports this flag, so I don't see how it would hurt
Please see this incoming PR for ARM support for an illustration of what I'm asking for here
https://github.com/vllm-project/vllm/pull/9957/files#diff-46b07f808e558aff84dcc365e53a91400c4be399757ff40c2da4278a50e89fbdL19-R24

DarkLight1337 · 2024-11-19T03:32:53Z

Please merge from main to fix the failing tests.

Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: B-201 <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

…0414) Signed-off-by: Isotr0py <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: B-201 <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

…#10406) Signed-off-by: youkaichao <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: ismael-dm <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: Andrew Nesbitt <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: mgoin <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: yan ma <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

…t#9014) Signed-off-by: Angus Wang <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: Lucas Wilkinson <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: youkaichao <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

…ct#9929) Signed-off-by: rickyx <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

We have too many dependencies and all patch updates can be a little noisy. This is to have dependabot ignore all patch version updates. Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

…form` (vllm-project#10358) Signed-off-by: Mengqing Cao <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

…rst (vllm-project#10433) Signed-off-by: Manjul Mohan <[email protected]>

…oject#10430) Signed-off-by: Travis Johnson <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: youkaichao <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Signed-off-by: Manjul Mohan <[email protected]>

mgoin

I guess we can land for now and fix the CPU cmake in the ARM support PRs

Signed-off-by: Manjul Mohan <[email protected]> Signed-off-by: B-201 <[email protected]> Signed-off-by: Isotr0py <[email protected]> Signed-off-by: youkaichao <[email protected]> Signed-off-by: ismael-dm <[email protected]> Signed-off-by: Andrew Nesbitt <[email protected]> Signed-off-by: mgoin <[email protected]> Signed-off-by: yan ma <[email protected]> Signed-off-by: Angus Wang <[email protected]> Signed-off-by: Lucas Wilkinson <[email protected]> Signed-off-by: rickyx <[email protected]> Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Mengqing Cao <[email protected]> Signed-off-by: Travis Johnson <[email protected]> Co-authored-by: Manjul Mohan [email protected] <[email protected]> Co-authored-by: B-201 <[email protected]> Co-authored-by: Isotr0py <[email protected]> Co-authored-by: youkaichao <[email protected]> Co-authored-by: ismael-dm <[email protected]> Co-authored-by: Andrew Nesbitt <[email protected]> Co-authored-by: Michael Goin <[email protected]> Co-authored-by: Yan Ma <[email protected]> Co-authored-by: Angus Wang <[email protected]> Co-authored-by: Lucas Wilkinson <[email protected]> Co-authored-by: Ricky Xu <[email protected]> Co-authored-by: Kevin H. Luu <[email protected]> Co-authored-by: Jee Jee Li <[email protected]> Co-authored-by: Mengqing Cao <[email protected]> Co-authored-by: Travis Johnson <[email protected]> Co-authored-by: Russell Bryant <[email protected]>

Signed-off-by: Manjul Mohan <[email protected]> Signed-off-by: B-201 <[email protected]> Signed-off-by: Isotr0py <[email protected]> Signed-off-by: youkaichao <[email protected]> Signed-off-by: ismael-dm <[email protected]> Signed-off-by: Andrew Nesbitt <[email protected]> Signed-off-by: mgoin <[email protected]> Signed-off-by: yan ma <[email protected]> Signed-off-by: Angus Wang <[email protected]> Signed-off-by: Lucas Wilkinson <[email protected]> Signed-off-by: rickyx <[email protected]> Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Mengqing Cao <[email protected]> Signed-off-by: Travis Johnson <[email protected]> Co-authored-by: Manjul Mohan [email protected] <[email protected]> Co-authored-by: B-201 <[email protected]> Co-authored-by: Isotr0py <[email protected]> Co-authored-by: youkaichao <[email protected]> Co-authored-by: ismael-dm <[email protected]> Co-authored-by: Andrew Nesbitt <[email protected]> Co-authored-by: Michael Goin <[email protected]> Co-authored-by: Yan Ma <[email protected]> Co-authored-by: Angus Wang <[email protected]> Co-authored-by: Lucas Wilkinson <[email protected]> Co-authored-by: Ricky Xu <[email protected]> Co-authored-by: Kevin H. Luu <[email protected]> Co-authored-by: Jee Jee Li <[email protected]> Co-authored-by: Mengqing Cao <[email protected]> Co-authored-by: Travis Johnson <[email protected]> Co-authored-by: Russell Bryant <[email protected]> Signed-off-by: rickyx <[email protected]>

Signed-off-by: Manjul Mohan <[email protected]> Signed-off-by: B-201 <[email protected]> Signed-off-by: Isotr0py <[email protected]> Signed-off-by: youkaichao <[email protected]> Signed-off-by: ismael-dm <[email protected]> Signed-off-by: Andrew Nesbitt <[email protected]> Signed-off-by: mgoin <[email protected]> Signed-off-by: yan ma <[email protected]> Signed-off-by: Angus Wang <[email protected]> Signed-off-by: Lucas Wilkinson <[email protected]> Signed-off-by: rickyx <[email protected]> Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Mengqing Cao <[email protected]> Signed-off-by: Travis Johnson <[email protected]> Co-authored-by: Manjul Mohan [email protected] <[email protected]> Co-authored-by: B-201 <[email protected]> Co-authored-by: Isotr0py <[email protected]> Co-authored-by: youkaichao <[email protected]> Co-authored-by: ismael-dm <[email protected]> Co-authored-by: Andrew Nesbitt <[email protected]> Co-authored-by: Michael Goin <[email protected]> Co-authored-by: Yan Ma <[email protected]> Co-authored-by: Angus Wang <[email protected]> Co-authored-by: Lucas Wilkinson <[email protected]> Co-authored-by: Ricky Xu <[email protected]> Co-authored-by: Kevin H. Luu <[email protected]> Co-authored-by: Jee Jee Li <[email protected]> Co-authored-by: Mengqing Cao <[email protected]> Co-authored-by: Travis Johnson <[email protected]> Co-authored-by: Russell Bryant <[email protected]> Signed-off-by: Tyler Michael Smith <[email protected]>

Signed-off-by: Manjul Mohan <[email protected]> Signed-off-by: B-201 <[email protected]> Signed-off-by: Isotr0py <[email protected]> Signed-off-by: youkaichao <[email protected]> Signed-off-by: ismael-dm <[email protected]> Signed-off-by: Andrew Nesbitt <[email protected]> Signed-off-by: mgoin <[email protected]> Signed-off-by: yan ma <[email protected]> Signed-off-by: Angus Wang <[email protected]> Signed-off-by: Lucas Wilkinson <[email protected]> Signed-off-by: rickyx <[email protected]> Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Mengqing Cao <[email protected]> Signed-off-by: Travis Johnson <[email protected]> Co-authored-by: Manjul Mohan [email protected] <[email protected]> Co-authored-by: B-201 <[email protected]> Co-authored-by: Isotr0py <[email protected]> Co-authored-by: youkaichao <[email protected]> Co-authored-by: ismael-dm <[email protected]> Co-authored-by: Andrew Nesbitt <[email protected]> Co-authored-by: Michael Goin <[email protected]> Co-authored-by: Yan Ma <[email protected]> Co-authored-by: Angus Wang <[email protected]> Co-authored-by: Lucas Wilkinson <[email protected]> Co-authored-by: Ricky Xu <[email protected]> Co-authored-by: Kevin H. Luu <[email protected]> Co-authored-by: Jee Jee Li <[email protected]> Co-authored-by: Mengqing Cao <[email protected]> Co-authored-by: Travis Johnson <[email protected]> Co-authored-by: Russell Bryant <[email protected]>

mergify bot added the ci/build label Nov 18, 2024

DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label Nov 18, 2024

simon-mo approved these changes Nov 18, 2024

View reviewed changes

mgoin reviewed Nov 18, 2024

View reviewed changes

mikejuliet13 force-pushed the vllm-power-issue branch from 4d0d6cd to 52e3d52 Compare November 19, 2024 12:13

Manjul Mohan [email protected] and others added 21 commits November 19, 2024 07:26

Fix: Build error seen on Power Architecture

0ce09ab

Signed-off-by: Manjul Mohan <[email protected]>

[Model][LoRA]LoRA support added for glm-4v (vllm-project#10418)

201a6da

Signed-off-by: B-201 <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Model] Remove transformers attention porting in VITs (vllm-project#1…

9120161

…0414) Signed-off-by: Isotr0py <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Doc] Update doc for LoRA support in GLM-4V (vllm-project#10425)

d7b14ce

Signed-off-by: B-201 <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[5/N][torch.compile] torch.jit.script --> torch.compile (vllm-project…

06de800

…#10406) Signed-off-by: youkaichao <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Doc] Add documentation for Structured Outputs (vllm-project#9943)

d3a6317

Signed-off-by: ismael-dm <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Fix open_collective value in FUNDING.yml (vllm-project#10426)

b2a1685

Signed-off-by: Andrew Nesbitt <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Model][Bugfix] Support TP for PixtralHF ViT (vllm-project#10405)

4c610a5

Signed-off-by: mgoin <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Hardware][XPU] AWQ/GPTQ support for xpu backend (vllm-project#10107)

96d56ba

Signed-off-by: yan ma <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Kernel] Explicitly specify other value in tl.load calls (vllm-projec…

21da67d

…t#9014) Signed-off-by: Angus Wang <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Kernel] Initial Machete W4A8 support + Refactors (vllm-project#9855)

2b855b1

Signed-off-by: Lucas Wilkinson <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[3/N][torch.compile] consolidate custom op logging (vllm-project#10399)

887d326

Signed-off-by: youkaichao <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[ci][bugfix] fix kernel tests (vllm-project#10431)

db5dddb

Signed-off-by: youkaichao <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[misc] partial prefix & random input generation benchmark (vllm-proje…

53e3a96

…ct#9929) Signed-off-by: rickyx <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[ci/build] Have dependabot ignore all patch update (vllm-project#10436)

ec45058

We have too many dependencies and all patch updates can be a little noisy. This is to have dependabot ignore all patch version updates. Signed-off-by: Manjul Mohan <[email protected]>

[Bugfix]Fix Phi-3 BNB online quantization (vllm-project#10417)

cce69dc

Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Platform][Refactor] Extract func get_default_attn_backend to `Plat…

2ce7cd4

…form` (vllm-project#10358) Signed-off-by: Mengqing Cao <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Add openai.beta.chat.completions.parse example to structured_outputs.…

6372003

…rst (vllm-project#10433) Signed-off-by: Manjul Mohan <[email protected]>

[Bugfix] Guard for negative counter metrics to prevent crash (vllm-pr…

c0482f6

…oject#10430) Signed-off-by: Travis Johnson <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Misc] Avoid misleading warning messages (vllm-project#10438)

3fcfe67

Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

[Doc] Add the start of an arch overview page (vllm-project#10368)

392acf9

Signed-off-by: Manjul Mohan <[email protected]>

youkaichao and others added 2 commits November 19, 2024 07:26

[misc][plugin] improve plugin loading (vllm-project#10443)

30deada

Signed-off-by: youkaichao <[email protected]> Signed-off-by: Manjul Mohan <[email protected]>

Fix for clang-format (3.11)

cd96cde

Signed-off-by: Manjul Mohan <[email protected]>

mikejuliet13 force-pushed the vllm-power-issue branch from a07e591 to cd96cde Compare November 19, 2024 12:26

mikejuliet13 requested review from robertgshaw2-neuralmagic, tlrmchlsmth, WoosukKwon, zhuohan123, youkaichao, alexm-neuralmagic, comaniac and njhill as code owners November 19, 2024 12:26

Merge branch 'vllm-project:main' into vllm-power-issue

6b053cb

DarkLight1337 enabled auto-merge (squash) November 19, 2024 15:16

mgoin approved these changes Nov 19, 2024

View reviewed changes

DarkLight1337 disabled auto-merge November 19, 2024 15:18

DarkLight1337 enabled auto-merge (squash) November 19, 2024 15:18

youkaichao disabled auto-merge November 19, 2024 17:34

youkaichao merged commit 1ea291a into vllm-project:main Nov 19, 2024
70 of 72 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix: Build error seen on Power Architecture #10421

Fix: Build error seen on Power Architecture #10421

mikejuliet13 commented Nov 18, 2024 •

edited by github-actions bot

Loading

github-actions bot commented Nov 18, 2024

sumitd2 commented Nov 18, 2024

simon-mo commented Nov 18, 2024

mgoin Nov 18, 2024

mkumatag Nov 19, 2024

mikejuliet13 Nov 19, 2024

mgoin Nov 19, 2024

DarkLight1337 commented Nov 19, 2024

mgoin left a comment

Fix: Build error seen on Power Architecture #10421

Fix: Build error seen on Power Architecture #10421

Conversation

mikejuliet13 commented Nov 18, 2024 • edited by github-actions bot Loading

github-actions bot commented Nov 18, 2024

sumitd2 commented Nov 18, 2024

simon-mo commented Nov 18, 2024

mgoin Nov 18, 2024

Choose a reason for hiding this comment

mkumatag Nov 19, 2024

Choose a reason for hiding this comment

mikejuliet13 Nov 19, 2024

Choose a reason for hiding this comment

mgoin Nov 19, 2024

Choose a reason for hiding this comment

DarkLight1337 commented Nov 19, 2024

mgoin left a comment

Choose a reason for hiding this comment

mikejuliet13 commented Nov 18, 2024 •

edited by github-actions bot

Loading