Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only check external access flag on raw GDAL vis handlers, disable filter pushdown in st_read #464

Merged
merged 1 commit into from
Dec 5, 2024

Conversation

Maxxen
Copy link
Member

@Maxxen Maxxen commented Dec 5, 2024

This PR make two changes related to how we interface with GDAL.

  1. Given the changes in Manage enable_external_access at the FileSystem level, and add allowed_paths and allowed_directories option duckdb#14568 we no longer explicitly check if external access is disabled in the st_read/st_read_meta/st_read_osm functions as that is now handled by the DuckDB filesystem. However, if a user supplies a /vsi/-prefixed path to circumvent the DuckDB filesystem handler we check the external access flag explicitly, but don't consider any additional whitelisted paths or directories. This should fix Failing to read files using DirFileSystem #461

  2. We no longer push down filters into GDAL when using st_read. This was never tested properly and since we don't really support any remote drivers (like pgsql/mysql) where filter pushdown make sense, id expect the performance benefits to be negligible. In some cases it may even be more efficient to allow duckdb to execute the filters, as we discovered in the case of sqlite and DuckDBs sqlite_scanner extension. This should fix Failure Casting String Literal to DATE in WHERE clause #457

@Maxxen Maxxen merged commit a60aa37 into duckdb:main Dec 5, 2024
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failing to read files using DirFileSystem Failure Casting String Literal to DATE in WHERE clause
1 participant