Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[No Merge] Changes with BaseMetadata #10

Closed
wants to merge 2 commits into from

Conversation

nk1506
Copy link
Owner

@nk1506 nk1506 commented Feb 12, 2024

No description provided.

@nk1506 nk1506 force-pushed the hive_table_ops_refactor_with_issue_9514 branch from 052ff12 to 9e440b4 Compare February 12, 2024 08:30
@github-actions github-actions bot removed the SPARK label Feb 12, 2024
protected void doCommit(TableMetadata base, TableMetadata metadata) {
protected void doCommit(BaseMetadata baseMetadata, BaseMetadata newMetadata) {
TableMetadata metadata = (TableMetadata) newMetadata;
TableMetadata base = (TableMetadata) baseMetadata;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we cast to table metadata I think we can't extract the common code for view,

I am expecting NessieBaseMetastoreTableOperations extends BaseMetastoreTableOperations
to handle common code and call one interface doCommitImpl() which will be implemented by NessieTableOperations and NessieViewOperations.

Similarly for other catalogs.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be doable. I am waiting for feedback on parent PR.

protected abstract void deleteRemovedMetadataFiles(
BaseMetadata baseMetadata, BaseMetadata newMetadata);;

protected void doCommit(BaseMetadata baseMetadata, BaseMetadata newMetadata) {
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can have a new API with BaseMetadata. Although I am seeing challenge on Backward compatibility.
Couple of questions I have :

  1. We can't change signature of doCommit on tableOperation or ViewOperation for backward compatible.
  2. If we deprecate the original one and create doCommit(BaseMetadata baseMetadata, BaseMetadata newMetadata) with Base class it can cause issue on runtime.
  3. Should we rename doCommit on base class? and gradually deprecate ?

@nk1506
Copy link
Owner Author

nk1506 commented Feb 13, 2024

This is just a draft patch to understand the cleanup with BaseMetadata interface. I think this is making code more cleaner for tableOperations and viewOperations. Only challenge I am seeing for compatibility.

@nk1506 nk1506 closed this Apr 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants