-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
请问pp异构并行训练时,划分模型层列表是否只能通过手动设置? #153
Comments
Yes, users currently have to set them manually. However, this can be avoided once we support auto-tuning for heterogeneous training. |
感谢回复,请问自动划分pp异构列表是否有计划?预计什么时间点支持? |
预计Q4支持,这也是我们未来的一个比较重要feature。不仅仅是PP异构并行自动调优,我们未来也会陆续支持异构全并行策略下的自动调优。 敬请期待@JinXiaozhao |
@Caozhou1995 您好,请问最新的版本0.6.0中的并行策略自动寻优(AutoTuner)模块,是否支持PP异构stage列表划分?能否提供相关示例?期待您的解答! |
现在仍在开发中,上线后回复您。@JinXiaozhao |
@Caozhou1995 您好,现在最新版本0.6.0版本中仅只支持对DP、TP、PP、CP进行自动寻优,不支持PP并行stage列表划分进行自动寻优? |
目前还未支持,PP stage仍是均匀划分。不均匀划分目前正在与异构自动调优开发中。 @JinXiaozhao |
好的,感谢您的回复,期待异构并行策略自动寻优模型的发布! |
No description provided.
The text was updated successfully, but these errors were encountered: