Stats errors out on big files, even when streaming-only stats

[dafeder]

I'm trying to pull stats on a very large CSV: 6gigs, ~11 million rows. I've indexed it, and am not asking for non-streaming stats. But the command simply ends with "Killed" after about a minute of running the CPU at 100%. Am I doing something wrong? My understanding was this should not be resource-limited.

Running Arch Linux on a 4-core Ryzen processor, with 32gb RAM.

[jqnatividad]

Hi @dafeder ,
I've handled bigger files without issue. My daily driver is macOS Sonoma on an M2 Pro with 32gb though and we deploy on Ubuntu Linux 22.04 LTS.

Have you tried validating the file to see if it's a well-formed CSV? Just run qsv validate on it.

Also, set the env var QSV_LOG_LEVEL=debug and see what the log file says.

Finally, what does qsv --version return?

[jqnatividad]

Hi @dafeder ,
Have you tried it with 0.125.0?

0.125.0 stats is also 25% faster...