Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add csv library support #260

Open
Kampfkarren opened this issue Sep 26, 2024 · 4 comments
Open

Add csv library support #260

Kampfkarren opened this issue Sep 26, 2024 · 4 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@Kampfkarren
Copy link

I would like to be able to use Lune for manipulating my LocalizationTables. I don't really want to write my own Luau CSV parser, it's somewhat annoying.

@Kampfkarren Kampfkarren changed the title Add csv support Add csv library support Sep 26, 2024
@CompeyDev
Copy link
Contributor

We do actually have a pure-luau CSV parser of sorts as a lune script. See .lune/csv_printer.luau.

Ideally this sort of thing is intended to be done as a library, I do not see much reason to include this in the runtime.

@deviaze
Copy link

deviaze commented Sep 28, 2024

As someone who's recently worked with 150k+ row csvs in Lune, I'd also appreciate dedicated runtime support for (fast) Rust-based serde.encode/decode("csv", text) functions in Lune's serde standard library.

My Lune-based impl. w/ string.splits took an average of 6+ seconds to parse and split the file into a 2d array-like table, whereas my string.split() + generalized character iteration approach took upwards of 15 seconds (before i ctrl Ced it) w/ the same file.

Although csvs are kinda old fashioned, having a fast way to read or write to csv is a pretty important feature to have in a runtime.

@CompeyDev
Copy link
Contributor

I still personally think this kind of thing should be implemented in the language itself, or if performance is really imperative, using FFI.

@filiptibell filiptibell added enhancement New feature or request good first issue Good for newcomers labels Oct 1, 2024
@filiptibell
Copy link
Collaborator

We do actually have a pure-luau CSV parser of sorts as a lune script

This is really just an example file and is meant to be easy to read - not parse all CSV files correctly. I wouldn't mind adding CSV support to @lune/serde, iff we can:

  • Agree on what CSV should look like in Lua - is it a plain 2D array? Or do we assume CSVs always have headers and use a map of header key -> columns?
  • Get a more robust implementation (handles quoting and so on, maybe even tab separated values)
  • Get noticeably improved performance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants