Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

异构训练是否支持sft? #294

Open
balcklive opened this issue Dec 18, 2024 · 3 comments
Open

异构训练是否支持sft? #294

balcklive opened this issue Dec 18, 2024 · 3 comments

Comments

@balcklive
Copy link

我看文档中对于异构训练的数据集格式是pre-training的格式，请问是否支持sft的训练模式呢？ 假如支持的话该如何操作呢？
@aoyulong
Copy link
Contributor

@balcklive 异构混训目前测试和使用还是pre-training，全参数sft的非异构功能是支持的，但异构这块暂时不支持。

@balcklive
Copy link
Author

同构sft是支持的话，是否有文档说明呢？ 好像没有看到这方面的说明

@Caozhou1995
Copy link
Collaborator

Caozhou1995 commented Dec 19, 2024 •
edited
Loading

同构SFT文档我们还在准备中。目前已经有同构SFT的相关示例代码合入，可以先参考：#278 @balcklive

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@aoyulong @Caozhou1995 @balcklive