Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Additional mypy checking for vllm/executor #11344

Closed
wants to merge 0 commits into from
Closed

Additional mypy checking for vllm/executor #11344

wants to merge 0 commits into from

Conversation

lucas-tucker
Copy link

@lucas-tucker lucas-tucker commented Dec 19, 2024 •
edited by github-actions bot
Loading

This is a PR to add type checking for variables that map node ids to gpu ids and worker ranks, to clarify that both integers and strings are currently allowed for gpu ids. The placement_group type hint is derived from vllm/config.py

#3680

@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@lucas-tucker lucas-tucker changed the title updated ray_gpu_executor type checking for node_workers and node_gpus… Additional mypy checking for vllm/executor Dec 19, 2024
DarkLight1337
DarkLight1337 reviewed Dec 20, 2024
View reviewed changes
vllm/executor/ray_gpu_executor.py Outdated
Comment on lines 39 to 44
self.use_ray_compiled_dag: bool = envs.VLLM_USE_RAY_COMPILED_DAG
# If the env var is set, then we do not distinguish between the
# "driver worker" vs other workers. Also, the rank 0 worker will
# be executed in a remote Ray worker. Currently this requires
# USE_RAY_COMPILED_DAG=True.
self.use_ray_spmd_worker = envs.VLLM_USE_RAY_SPMD_WORKER
self.use_ray_spmd_worker: bool = envs.VLLM_USE_RAY_SPMD_WORKER
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? I think the types should be annotated in envs already.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes they are, so I suppose this is not necessary

DarkLight1337
DarkLight1337 approved these changes Dec 20, 2024
View reviewed changes
Copy link
Member

@DarkLight1337 DarkLight1337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this! Please fix the lint errors.

@mergify mergify bot added documentation Improvements or additions to documentation ci/build frontend labels Dec 21, 2024
DarkLight1337
DarkLight1337 requested changes Dec 21, 2024
View reviewed changes
Copy link
Member

@DarkLight1337 DarkLight1337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you accidentally changed a bunch of other stuff. Please fix your commit history.

@DarkLight1337 DarkLight1337 enabled auto-merge (squash) December 21, 2024 03:47
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Dec 21, 2024
@DarkLight1337 DarkLight1337 removed documentation Improvements or additions to documentation frontend ci/build labels Dec 21, 2024
@DarkLight1337 DarkLight1337 removed the request for review from ywang96 December 21, 2024 03:48
auto-merge was automatically disabled December 21, 2024 04:00

Head branch was pushed to by a user without write access

auto-merge was automatically disabled December 21, 2024 04:00

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DarkLight1337 DarkLight1337 DarkLight1337 approved these changes

@youkaichao youkaichao Awaiting requested review from youkaichao

Assignees
No one assigned
Labels
ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@lucas-tucker @DarkLight1337