By default match fields by position #80
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #80 +/- ##
==========================================
+ Coverage 92.01% 92.13% +0.11%
==========================================
Files 70 71 +1
Lines 8879 9009 +130
==========================================
+ Hits 8170 8300 +130
Misses 709 709 ☔ View full report in Codecov by Sentry. |
aykut-bozkurt
force-pushed
the
aykut/match_by_position
branch
2 times, most recently
from
ed4907a to
e6b10bb
Compare
We add an option for `COPY FROM` called `match_by_name` which matches Parquet file fields to PostgreSQL table columns `by their names` rather than `by their order` in the schema. By default, the option is `false`. The option is useful when field order differs between the Parquet file and the table, but their names match. **!!IMPORTANT!!**: This is a breaking change. Before the PR, we match always by name. This is a bit strict and not common way to match schemas. (e.g. COPY FROM csv at postgres or COPY FROM of duckdb match by field position by default) This is why we match by position by default and have a COPY FROM option `match_by_name` that can be set to true for the old behaviour. Closes #39.
aykut-bozkurt
force-pushed
the
aykut/match_by_position
branch
from
e6b10bb to
4241cae
Compare
aykut-bozkurt
changed the title
aykut-bozkurt
changed the title
marcoslot
approved these changes
Nov 28, 2024
lets move it to breaking changes section when we release. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We add an option for
COPY FROMcalledmatch_by [position(default) | name]which determines the match method for the Postgres table columns and Parquet file fields. By default, the match method isposition. Match bynameis useful when field order differs between the Parquet file and the table, but their names match.!!IMPORTANT!!: This is a breaking change. Before the PR, we match always by name. This is a bit strict and not common way to match schemas. (e.g. COPY FROM csv at postgres or COPY FROM of duckdb match by field position by default) This is why we match by position by default now and have a COPY FROM option
match_bythat can be set tonamefor the old behavior.Closes #39.