Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add download callback to ClimaArtifactsHelper #76

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

imreddyTeja
Copy link
Contributor

@imreddyTeja imreddyTeja commented Nov 19, 2024

There are two callbacks in this PR. The first only prints the download progress, while the other also prints the download rate.

For both, total size is printed as MB as an int if it is less than 1 GB, and GB with 2 decimal places otherwise. The same applies to dowloaded size and download rate.

The second callback is created with the function create_download_rate_callback which returns
a function. The callback calculates the
download rate over an interval, and prints the result at the end of
the interval. It also prints download progress. This callback
uses a closure to keep track of the time and bytes downloaded
at the end of the previous interval. If this callback is good,
then we can probably remove download_progress_callback from this PR.

I considered creating a wrapper for the download function, which would take in the file url and file name, and use the callback while downloading it, and then change the file name. This seems like it might be excessive. Thoughts?

Should I go through the existing artifacts and add the callback to the create_artifact.jl scripts?

Closes #68

The callback handles the condition when total size is unknown.
Total size is printed as  MB as an int if it is less than 1 GB, and GB with 2 decimal
places otherwise. The same applies to dowloaded size.
Add function `create_download_rate_callback` which returns
a callback for Downloads.downloads. The callback calculates the
download rate over an interval, and prints the result at the end of
the interval. It also prints download progress. This callback
uses a closure to keep track of the time and bytes downloaded
at the end of the previous interval. If this callback is good,
then we can probably remove `download_progress_callback`
@Sbozzolo
Copy link
Member

Is it possible to create a Downloader that uses this in ClimaArtifactsHelper? So that people would import the downloader and use it directly

@Sbozzolo Sbozzolo force-pushed the tr/add-download-progress-info branch from 8c0d45b to 8d86932 Compare November 25, 2024 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add status bar while downloading files
2 participants