Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ci][gh200] dockerfile clean up #11351

Merged
merged 27 commits into from
Dec 20, 2024
Merged

[ci][gh200] dockerfile clean up #11351

merged 27 commits into from
Dec 20, 2024

Conversation

youkaichao
Copy link
Member

built upon #11302 , check it out with another branch in the upstream repo so that we can test it before merging.

drikster80 and others added 23 commits November 20, 2024 21:37
@drikster80
Update Docker for aarch64 builds
d2cbe42 
Signed-off-by: drikster80 <[email protected]>
@drikster80
Update docs for arm64 docker builds & GH200 example
29ed358 
Signed-off-by: drikster80 <[email protected]>
@drikster80
Fix FLASHINFER not installing by default on x86_64
f2635a4 
Signed-off-by: drikster80 <[email protected]>
@drikster80
Removed build of flash-attn. Added libnccl2
ccef455
@cennn
simplified Dockerfile build process for ARM64 systems
8b3c0b3
@cennn
bitsandbytes>=0.45.0
69bb567
@cennn
Resolve merge conflict in Dockerfile
600b805
@cennn
rm needless kmod/libnccl2 && rm needless build from source for triton…
d5d2911 
…/causal-conv1d/mamba/flashinfer/bitsandbytes
@cennn
update Memory usage && rename image && pinned version for torch torch…
075ab9d 
…vision
@cennn
Merge remote-tracking branch 'origin/main'
0ddc464 
# Conflicts:
#	docs/source/serving/deploying_with_docker.rst
@cennn
rm requirements-cuda-arm64.txt && rm redundant torch install/uninstal…
34e7140 
…l && rm arm64 platform condition in requirements && add python3 use_existing_torch.py before build in run-gh200-test.sh
@cennn
fix docs --platform "linux/arm64"
c65bf72
@cennn
rm docs blanks
6bddcd4
@cennn
add torch nightly build in run-gh200-test.sh
6b80175
@cennn
edit doc note && rm build-arg max_jobs nvcc_threads
93b82d7
@cennn
add args max_jobs nvcc_threads to speed up && install pytorch first &…
d026489 
…& outlines == 0.1.11 # Requires pytorch && rm torch install before docker build
@cennn
update vllm-gh200-openai Image size: 3.23GB
fb1759e
@cennn
add debug "python3 -m pip list | grep torch || true" && compressed-te…
62c8e29 
…nsors requires pytorch
@cennn
rm "python3 -m pip list | grep torch || true" && add cloudpickle && u…
dd483d9 
…pdate image size
@cennn
rm sudo in doc
022eaf9
@youkaichao
avoid installing python
c61c579 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
add comments
5c056e6 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
fix comments
383f799 
Signed-off-by: youkaichao <[email protected]>
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@mergify mergify bot added documentation Improvements or additions to documentation ci/build labels Dec 20, 2024
@youkaichao
Copy link
Member Author

thanks for the great work from @cennn

and also cc @drikster80 FYI , we decided to treat GH200 build a use case of "use existing torch", instead of hardcode the platform in requirements.txt .

@youkaichao youkaichao mentioned this pull request Dec 20, 2024
Closed
@youkaichao
Copy link
Member Author

the build is successful at https://buildkite.com/vllm/ci/builds/10942#0193e181-8519-48fc-857d-8ac350833690

@youkaichao
doc change
fccb985 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
fix docs
67129a7 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
fix docs
f7f54f6 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
fix docs
7dbe69e 
Signed-off-by: youkaichao <[email protected]>
@cennn
Copy link
Contributor

cennn commented Dec 20, 2024

look good to me!

cennn
cennn approved these changes Dec 20, 2024
View reviewed changes
Copy link
Contributor

@cennn cennn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

cennn
cennn approved these changes Dec 20, 2024
View reviewed changes
Copy link
Contributor

@cennn cennn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved

@youkaichao youkaichao merged commit 7801f56 into main Dec 20, 2024
22 of 26 checks passed
@youkaichao youkaichao deleted the gh200_clean branch December 20, 2024 02:13
lucas-tucker pushed a commit to lucas-tucker/vllm-lucas-tucker that referenced this pull request Dec 21, 2024
@youkaichao @drikster80 @cennn
[ci][gh200] dockerfile clean up (vllm-project#11351)
8c034f4 
Signed-off-by: drikster80 <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: drikster80 <[email protected]>
Co-authored-by: cenzhiyao <[email protected]>
Signed-off-by: lucast2021 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cennn cennn cennn approved these changes

Assignees
No one assigned
Labels
ci/build documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@youkaichao @cennn @drikster80