Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[590] Add interfaces for CatalogSyncClient and CatalogSync #603

Merged
merged 1 commit into from
Dec 26, 2024

Conversation

vinishjail97
Copy link
Contributor

@vinishjail97 vinishjail97 commented Dec 18, 2024

Important Read

  • Please ensure the GitHub issue is mentioned at the beginning of the PR

What is the purpose of the pull request

Add interface for CatalogSyncClient which allows syncing table format metadata to a catalog.
Add interface for CatalogTableIdentifier and an implementation HierarchicalTableIdentifier for a three level naming convention. This allows future implementations to be plugged in following a different convention.
Add a new class CatalogSync syncs the canonical object (InternalTable) to multiple catalogs.
Introduce a new method getCurrentTable in the interface ConversionSource which extracts the latest table state and will be used when using CatalogSync

See RFC PR for more details about design and flow chart -> #605

Brief change log

(for example:)

  • Add interface for CatalogSyncClient and CatalogSync
  • Introduce getCurrentTable method in ConversionSource interface for catalog sync

Verify this pull request

(Please pick either of the following options)

This change added tests and can be verified as follows:

(example:)

  • xtable-api/src/test/java/org/apache/xtable/spi/sync/TestCatalogSync.java
  • xtable-api/src/test/java/org/apache/xtable/spi/sync/TestCatalogUtils.java
  • xtable-core/src/test/java/org/apache/xtable/delta/ITDeltaConversionTargetSource.java#getCurrentTableTest
  • xtable-core/src/test/java/org/apache/xtable/hudi/ITHudiConversionSourceSource.java#getCurrentTableTest
  • xtable-core/src/test/java/org/apache/xtable/iceberg/ITIcebergConversionTargetSource.java#getCurrentTableTest

@vinishjail97 vinishjail97 merged commit fb724f0 into apache:main Dec 26, 2024
2 checks passed
@vinishjail97 vinishjail97 deleted the 590-CatalogSync-API branch December 26, 2024 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants