Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] add yi-6b and yi-34b sft script #216

Merged
merged 6 commits into from
Nov 13, 2023
Merged

[Feature] add yi-6b and yi-34b sft script #216

merged 6 commits into from
Nov 13, 2023

Conversation

amulil
Copy link
Contributor

@amulil amulil commented Nov 10, 2023

No description provided.

@amulil amulil changed the title add yi-6b and yi-34b sft script [Feature] add yi-6b and yi-34b sft script Nov 10, 2023
@amulil amulil marked this pull request as draft November 13, 2023 03:57
@LZHgrla LZHgrla mentioned this pull request Nov 13, 2023
Closed
@LZHgrla
Copy link
Collaborator

LZHgrla commented Nov 13, 2023

Thanks very much for your contribution!

I have reviewed this PR and there are two comments:

  1. Please set accumulative_counts=16 and dataloader_num_workers=0 to align with other configs
  2. Use PagedAdamW32bit instead of AdamW is better?

@amulil
Copy link
Contributor Author

amulil commented Nov 13, 2023 •
edited
Loading

Thanks very much for your contribution!

I have reviewed this PR and there are two comments:

  1. Please set accumulative_counts=16 and dataloader_num_workers=0 to align with other configs
  2. Use PagedAdamW32bit instead of AdamW is better?

Thanks for your advice.I update them.
And I add dispatch_yi_attn_forward function.

@amulil amulil marked this pull request as ready for review November 13, 2023 15:47
@LZHgrla
Copy link
Collaborator

LZHgrla commented Nov 13, 2023

@amulil
Please fix the pre-commit~
Guide: https://github.com/InternLM/xtuner/blob/main/.github/CONTRIBUTING.md#2-configure-pre-commit

@amulil
Copy link
Contributor Author

amulil commented Nov 13, 2023

@amulil Please fix the pre-commit~ Guide: https://github.com/InternLM/xtuner/blob/main/.github/CONTRIBUTING.md#2-configure-pre-commit

I'm sorry.It seems I have fixed it.Please check it. @LZHgrla

LZHgrla
LZHgrla approved these changes Nov 13, 2023
View reviewed changes
Copy link
Collaborator

@LZHgrla LZHgrla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amulil
LGTM!
Is this PR ready to be merged?

@amulil
Copy link
Contributor Author

amulil commented Nov 13, 2023

LGTM!

@amulil LGTM! Is this PR ready to be merged?

Yes!It can be merged. @LZHgrla

@LZHgrla LZHgrla merged commit 90f7d84 into InternLM:main Nov 13, 2023
1 check passed
@amulil amulil deleted the yi branch November 13, 2023 17:02
llkn-2 pushed a commit to llkn-2/xtuner that referenced this pull request Jul 31, 2024
@amulil
[Feature] add yi-6b and yi-34b sft script (InternLM#216)
ed21931 
* add yi-6b and yi-34b sft script

* update yi-6b and yi-34b sft script

* update yi-6b and yi-34b sft script

* add dispatch_yi_attn_forward function

* update yi-6b and yi-34b sft script

* update yi script
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LZHgrla LZHgrla LZHgrla approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@amulil @LZHgrla