-
Notifications
You must be signed in to change notification settings - Fork 990
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[hive]add retention seconds in hive catalog. #4367
Conversation
@@ -814,6 +814,11 @@ private Table newHmsTable( | |||
if (provider == null) { | |||
provider = "paimon"; | |||
} | |||
CoreOptions coreOptions = CoreOptions.fromMap(tableParameters); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Duration partitionExpireTime = coreOptions.partitionExpireTime();
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
int retention = | ||
coreOptions.partitionExpireTime() == null | ||
? Integer.MAX_VALUE | ||
: (int) coreOptions.partitionExpireTime().getSeconds(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a risk of data overflow here?
getSeconds return long, then cast int.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
Can you provide the specific application of this attribute? For example, Hive first introduced it with JIRA, etc. |
java.time.Duration partitionExpireTime = coreOptions.partitionExpireTime(); | ||
int retention = Integer.MAX_VALUE; | ||
if (partitionExpireTime != null && partitionExpireTime.getSeconds() <= Integer.MAX_VALUE) { | ||
retention = (int) partitionExpireTime.getSeconds(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe only set it with metastore.partitioned-table enabled.
I am not sure this field is for partition retention. Maybe we can use properties to store this option. |
Close this first, feel free to re-open this. |
Purpose
Add
retention
properties for hive metastore according to 'partition.expiration-time'.Linked issue: close #xxx
Tests
API and Format
Documentation