-
Notifications
You must be signed in to change notification settings - Fork 415
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add out_of_range flag to load_cdf #3040
base: main
Are you sure you want to change the base?
Conversation
ACTION NEEDED delta-rs follows the Conventional Commits specification for release automation. The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3040 +/- ##
==========================================
+ Coverage 72.68% 72.72% +0.03%
==========================================
Files 129 129
Lines 41462 41578 +116
Branches 41462 41578 +116
==========================================
+ Hits 30137 30237 +100
Misses 9376 9376
- Partials 1949 1965 +16 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! Just couple comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me now, just rerunning CI right now local concurrency test is notoriously flakey.
Awesome, seems that all the tests passed now |
@pblocz if you could sign off on your commits we can get this merged soon |
Signed-off-by: Pablo Cabeza <[email protected]>
Signed-off-by: Pablo Cabeza <[email protected]>
… clarity Signed-off-by: Pablo Cabeza <[email protected]>
Signed-off-by: Pablo Cabeza <[email protected]>
@hntd187 Done. I don't know why, but thought the sign off was something else 😅 |
Description
This PR changes load_cdf so:
By default
And you can use
.with_allow_out_of_range()
in rust orallow_out_of_range=True
in python to make it so that both cases return an empty dataframe.This aligns it closer with how reading the change feed works in spark and makes the behaviour consistent.
This in contrast to the current behavior:
Related Issue(s)