Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
williamFalcon authored Jul 6, 2024
1 parent e7f9f21 commit d9509fb
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,10 @@ Transform Optimize
 

# Transform data at scale. Optimize for fast model training.
Use LitData to speed up data processing (resizing, embedding, etc) or to optimize datasets to speed up model training.
LitData scales data processing tasks (data scraping, image resizing, distributed inference, embedding creation) on local or cloud machines. It also enables optimizing datasets to accelerate AI model training and work with large remote datasets without local loading.

### Transform datasets
Parallelize (map) transforms across many machines at once to process large datasets much faster.
### Transform datasets
Accelerate data processing tasks (data scraping, image resizing, embedding creation) by parallelizing (map) the work across many machines at once.

<pre>
✅ Paralellize processing: Reduce processing time by transforming data across multiple machines simultaneously.
Expand Down Expand Up @@ -166,7 +166,7 @@ for sample in dataloader:
----

# Transform datasets
Use LitData to apply transforms to large datasets across multiple machines in parallel. Common usecases are to create vector embeddings, run distributed inference and more.
Accelerate data processing tasks (data scraping, image resizing, embedding creation, distributed inference) by parallelizing (map) the work across many machines at once.

Here's an example that resizes and crops a large image dataset:

Expand Down

0 comments on commit d9509fb

Please sign in to comment.