Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hive] Support recreate the hive table #3084

Merged
merged 4 commits into from
Apr 8, 2024
+58 −7
Merged

[hive] Support recreate the hive table #3084

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d3d9047
fix the bug #3083,support recreat the hive table
Mar 24, 2024
2b942f7
[fix] Fix support recreate hive table,add unit test (#3083)
Mar 27, 2024
2be86e0
add unit test,spotless formate code (#3083)
Apr 8, 2024
11fe542
modify unit test (#3083)
Apr 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 25 additions & 7 deletions paimon-core/src/main/java/org/apache/paimon/schema/SchemaManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.concurrent.Callable;
import java.util.concurrent.atomic.AtomicInteger;
Expand Down Expand Up @@ -124,16 +125,28 @@ public List<Long> listAllIds() {
}
}

/** Check TableScheme is be modified. */
public void checkTableSchema(TableSchema oldSchema, TableSchema newSchema) {
boolean isCommon =
oldSchema.version() == newSchema.version()
&& oldSchema.id() == newSchema.id()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to compare schema id and options.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, if there's no change in the entire logic, I'll remove the schema id and options

&& Objects.equals(oldSchema.fields(), newSchema.fields())
&& oldSchema.highestFieldId() == newSchema.highestFieldId()
&& Objects.equals(oldSchema.partitionKeys(), newSchema.partitionKeys())
&& Objects.equals(oldSchema.primaryKeys(), newSchema.primaryKeys())
&& Objects.equals(oldSchema.options(), newSchema.options());

if (!isCommon) {
throw new IllegalStateException(
"Schema in filesystem exists, please use updating,"
+ " latest schema is: "
+ oldSchema);
}
}

/** Create a new schema from {@link Schema}. */
public TableSchema createTable(Schema schema) throws Exception {
while (true) {
latest().ifPresent(
latest -> {
throw new IllegalStateException(
"Schema in filesystem exists, please use updating,"
+ " latest schema is: "
+ latest());
});

List<DataField> fields = schema.fields();
List<String> partitionKeys = schema.partitionKeys();
Expand All @@ -151,6 +164,11 @@ public TableSchema createTable(Schema schema) throws Exception {
options,
schema.comment());

if (latest().isPresent()) {
checkTableSchema(latest().get(), newSchema);
return newSchema;
}

boolean success = commit(newSchema);
if (success) {
return newSchema;
Expand Down
Loading