Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix resuming dataset state #217

Merged
merged 17 commits into from
Jul 11, 2024
Merged

Fix resuming dataset state #217

merged 17 commits into from
Jul 11, 2024

Conversation

awaelchli
Copy link
Contributor

@awaelchli awaelchli commented Jul 9, 2024

What does this PR do?

Fixes #213

The issue was the combined effect of two unrelated bugs. The added tests cover both of them and fail when run on the main branch.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@awaelchli awaelchli requested a review from tchaton as a code owner July 9, 2024 16:07
Copy link
Collaborator

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind adding some tests ?

@awaelchli awaelchli force-pushed the bugfix/resume-dataloader branch from 1eb47a1 to aaab88f Compare July 10, 2024 15:37
@awaelchli awaelchli changed the title WIP: Fix resuming dataset state Fix resuming dataset state Jul 10, 2024
tests/streaming/test_dataset.py Show resolved Hide resolved
tests/streaming/test_dataset.py Outdated Show resolved Hide resolved
@awaelchli awaelchli added the bug Something isn't working label Jul 11, 2024
@tchaton tchaton merged commit 58168c8 into main Jul 11, 2024
31 of 32 checks passed
@tchaton tchaton deleted the bugfix/resume-dataloader branch July 11, 2024 11:35
@awaelchli awaelchli restored the bugfix/resume-dataloader branch July 11, 2024 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Resuming the StreamingDataLoader from a checkpoint makes the iterator hang
2 participants