Skip to content

Commit

Permalink
Mention how the progress bar updates (#75)
Browse files Browse the repository at this point in the history
Co-authored-by: Jirka Borovec <[email protected]>
Co-authored-by: tchaton <[email protected]>
  • Loading branch information
3 people authored Apr 3, 2024
1 parent 015a546 commit 237b44c
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion src/litdata/processing/data_processor.py
Original file line number Diff line number Diff line change
Expand Up @@ -949,7 +949,10 @@ def run(self, data_recipe: DataRecipe) -> None:

self._cleanup_cache()

print(f"Starting {self.num_workers} workers with {num_items} items.")
print(
f"Starting {self.num_workers} workers with {num_items} items."
f" The progress bar is only updated when a worker finishes."
)

if self.input_dir is None and self.src_resolver is not None and self.input_dir:
self.input_dir = self.src_resolver(self.input_dir)
Expand Down

0 comments on commit 237b44c

Please sign in to comment.