Question: Is there a list for publicly available s3 links of datasets of litdata.StreamingDataset format?

[2catycm]

If there is a list that collects some popular datasets that have been preprocessed by litdata and upload to lightning studio or S3, then the usability of this project will be really awesome for me.

For example, is there a streaming dataset for imagenet that is publicly available?

[tchaton]

Hey @2catycm. Yes, there is. I haven't processed much datasets so far.

Here are my published Studios: https://lightning.ai/thomasgridai

The dataset is available under s3://optimized-imagenet-1m/lightning_data_imagenet I think to remember