How to use infile-list extension?

[aborruso]

Hi,
I have created the file lista.infile-list.

I contains

./Opendata251.csv
./Opendata157.csv
./Opendata139.csv
./Opendata292.csv
./Opendata47.csv

If I run

qsv sqlp lista.infile-list --ignore-errors --infer-len 30000 -d ";" 'select * from lista limit 1000'

I have

Failed to execute query: select * from lista limit 1000: relation 'lista' was not found

What am I doing wrong?

Thank you

[jqnatividad]

Hi @aborruso ,
.infile-list is an alternate way of providing a list of input files to qsv. Instead of passing the list through the command line (which, if you have hundreds of files, is not only unwieldy but is also relatively slow), you pass an .infile-list text file listing each input file. So your command:

qsv sqlp lista.infile-list --ignore-errors --infer-len 30000 -d ";" 'select * from lista limit 1000'

is basically equivalent to:

qsv sqlp Opendata251.csv Opendata157.csv Opendata139.csv Opendata292.csv Opendata47.csv --ignore-errors --infer-len 30000 -d ";" 'select * from lista limit 1000'

so there is no lista table in the Polars SQLcontext. Instead, the following tables are created in the SQLcontext - Opendata251, Opendata157, Opendata139, Opendata292 and Opendata47, with their corresponding _t_N alias in list order.

Perhaps, you should qsv cat rowskey the files first and then use the concatenated input in sqlp?

qsv cat rowskey testinfile.infile-list -d ";" | qsv sqlp - "select * from stdin"

Or maybe, do a SQL union? Something like:

qsv sqlp lista.infile-list -d ";" 'select * from _t_1 union select * from _t_2 union select * from _t_3 union select * from _t_4 union select * from _t_5'