-
-
Notifications
You must be signed in to change notification settings - Fork 5.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Misc][LoRA] Clean up the function interface of Punica #10917
[Misc][LoRA] Clean up the function interface of Punica #10917
Conversation
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for cleaning up the duplicate function! I added some initial comments about the punica interface. PTAL!
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall LGTM!
…10917) Signed-off-by: Jee Jee Li <[email protected]>
…10917) Signed-off-by: Jee Jee Li <[email protected]>
Description
The current Punica interfaces(functions whose names start with
add_
in punica.py) are somewhat complex and disorganized. This PR primarily categorizes all LoRA layers into the following three types and clean up related interfaces. The benefits include simplifying and standardizing functions to facilitate LoRA support across other platforms, such as: #4830 #105651. Linear
Three
functions are provided, where add_lora_linear is for all linear lora in layers.py, while add_shrink and add_expand are designed for full sharded lora in fully_sharded_layers.py2. Embedding
Only
one
function is needed, see: add_lora_embedding3. Logits
Only
one
function is needed,see:add_lora_logitscc @SanjuCSudhakaran
ping @DarkLight1337
@Isotr0py Since you're the author of #4830, would you please take a look at this PR as well?