Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --backfill option to download_weather worker #309

Open
douglatornell opened this issue Dec 4, 2024 · 0 comments
Open

Add --backfill option to download_weather worker #309

douglatornell opened this issue Dec 4, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request Workers
Milestone

Comments

@douglatornell
Copy link
Member

The download_weather worker presently raises and error and exits if the destination directory it is to write to already exists when lib.mkdir() is called for it. While that is good to avoid accidental overwrites, there are situations during recovery from automation issues that it is desirable to allow the overwite. Specifically, to recover from a missed HRDPS forecast download we want to start the crop_gribs worker, which creates the destination directory, then start download_weather so that cropping happens as the download progresses rather than having to wait until after the download finishes to start cropping.

Adding a boolean --backfill option that defaults to False and passing its value to lib.mkdir() via the exist_ok argument will facilitate that use case.

@douglatornell douglatornell added enhancement New feature or request Workers labels Dec 4, 2024
@douglatornell douglatornell added this to the v24.2 milestone Dec 4, 2024
@douglatornell douglatornell modified the milestones: v24.2, v25.1 Jan 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Workers
Projects
None yet
Development

No branches or pull requests

1 participant