-
Notifications
You must be signed in to change notification settings - Fork 523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
remove multiprocess and buffer in multitask mode #3630
Conversation
for more information, see https://pre-commit.ci
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## devel #3630 +/- ##
=======================================
Coverage 77.79% 77.79%
=======================================
Files 438 438
Lines 37693 37699 +6
Branches 1623 1623
=======================================
+ Hits 29322 29327 +5
- Misses 7507 7508 +1
Partials 864 864 ☔ View full report in Codecov by Sentry. |
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there efficiency comparison before and after this modification in multitask training with numerous datasets?
the trainer choose random model for each step. Prefetching for each model may be inefficient.