Upload single csv files to zenodo

[FlorianK13]

Here they asked if we could upload single csv files to zenodo instead of zipped folders PyPSA/powerplantmatching#165 (comment)
What do you think @nesnoj

[nesnoj]

Hey @FlorianK13,

great to hear that the data are being used and I like the idea!
Yes, we could use CSV. However, we would run into trouble pretty fast due to the max. size of 50 GB for one dataset, since sizes of all releases are summed up (we have faced this limitation in another project...).

In numbers for August release:
6.6 GB raw, 1.1 GB zipped. Let's say we'd setup a separate package for the CSVs, the limit was hit after 7 releases (neglecting increasing size). Of course, we will also reach the limit for the zipped versions but much later...

So I do not have a good solution for this at the moment, sorry :(
Open for further ideas..

[nesnoj]

Idea: Additional Zenodo record with individually zipped CSVs

[fneum]

That would work well for us!

Parquet files could also be an option rather than zipped CSVs. https://parquet.apache.org/