Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hive] repair table support sync all properties to hms #4047

Merged
merged 3 commits into from
Aug 26, 2024

Conversation

herefree
Copy link
Contributor

Purpose

Linked issue: close #xxx
Follow up #4038

Tests

API and Format

Documentation

@@ -819,6 +819,20 @@ private void updateHmsTable(Table table, Identifier identifier, TableSchema sche
private void updateHmsTablePars(Table table, TableSchema schema) {
if (syncAllProperties()) {
table.getParameters().putAll(schema.options());
if (!schema.primaryKeys().isEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please extract a method to reuse.

@xuzifu666
Copy link
Member

Just curious,in real scene Paimon should not support alter pk,partitionkey? @herefree

@JingsongLi
Copy link
Contributor

Just curious,in real scene Paimon should not support alter pk,partitionkey? @herefree

Yes

@herefree
Copy link
Contributor Author

herefree commented Aug 23, 2024

Just curious,in real scene Paimon should not support alter pk,partitionkey? @herefree

Yes, if you have some history paimon table which was not syn pk or partitionkey to hive meta, you can use repair table.It also need to add some code to support it, just like this.
image

@xuzifu666
Copy link
Member

Just curious,in real scene Paimon should not support alter pk,partitionkey? @herefree

Yes, if you have some history paimon table which was not syn pk or partitionkey to hive meta, you can use repair table.It also need to add some code to support it, just like this. image

Get,but the title maybe not fitable,alter primary-key can take a look at SparkSchemaEvolutionITCase##testSetAndRemoveOption which can not support.

@herefree herefree changed the title [hive] alter table option support sync primary-key/partition-key/bucket-id to hms [hive] repair table support sync all properties to hms Aug 23, 2024
@herefree
Copy link
Contributor Author

Just curious,in real scene Paimon should not support alter pk,partitionkey? @herefree

Yes, if you have some history paimon table which was not syn pk or partitionkey to hive meta, you can use repair table.It also need to add some code to support it, just like this. image

Get,but the title maybe not fitable,alter primary-key can take a look at SparkSchemaEvolutionITCase##testSetAndRemoveOption which can not support.

Yes,you are right,i'll change the title.

@JingsongLi
Copy link
Contributor

+1

@JingsongLi JingsongLi merged commit ae6f2bc into apache:master Aug 26, 2024
10 checks passed
@melin
Copy link
Contributor

melin commented Aug 28, 2024

When a table is created by spark sql, the primary key and partition information are not synchronized to the hms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants