Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable docker run arguments when starting the pod #338

Open
cblmemo opened this issue Aug 29, 2024 · 0 comments
Open

Enable docker run arguments when starting the pod #338

cblmemo opened this issue Aug 29, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@cblmemo
Copy link

cblmemo commented Aug 29, 2024

Is your feature request related to a problem? Please describe.
Yes - we would like to use --entrypoint=/bin/bash to enable using image with non-default entrypoint as RunPod base image, like the official vLLM image vllm/vllm-openai:latest. That image comes with an entrypoint ["python3", "-m", "vllm.entrypoints.openai.api_server"] (reference here).

Describe the solution you'd like
Adding another argument docker_run_args in the create_pod API. That will be

Describe alternatives you've considered
Maybe just adds a boolean flag to override the entrypoint with /bin/bash - I guess it is common to use an image with proper dependency but a non-default entrypoint.

Additional context
Using vllm/vllm-openai:latest as base image gave me this error:

2024-08-29T23:42:51.938684641Z                      [--max-lora-rank MAX_LORA_RANK]
2024-08-29T23:42:51.938688657Z                      [--lora-extra-vocab-size LORA_EXTRA_VOCAB_SIZE]
2024-08-29T23:42:51.938692581Z                      [--lora-dtype {auto,float16,bfloat16,float32}]
2024-08-29T23:42:51.938696434Z                      (a lot of arguments here)
2024-08-29T23:42:51.938825141Z api_server.py: error: unrecognized arguments: bash -c echo xxx | base64 --decode > init.sh; bash init.sh

The script I use to create the pod is similar to #337, but only change the image_name arguments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant