Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support clone latest snapshot #3287

Merged
merged 4 commits into from
Jun 4, 2024
Merged

support clone latest snapshot #3287

merged 4 commits into from
Jun 4, 2024

Conversation

wwj6591812
Copy link
Contributor

Purpose

Linked issue: close #xxx

Tests

API and Format

Documentation

@wwj6591812 wwj6591812 force-pushed the support_clone_latest_snapshot_0429 branch from bd32ebd to b67e04e Compare April 30, 2024 01:16
@wwj6591812 wwj6591812 mentioned this pull request May 5, 2024
Closed
tsreaper
tsreaper reviewed May 13, 2024
View reviewed changes
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/CloneAction.java Outdated Show resolved Hide resolved
...k/paimon-flink-common/src/main/java/org/apache/paimon/flink/clone/CloneFileInfoTypeInfo.java Outdated Show resolved Hide resolved
...link/paimon-flink-common/src/main/java/org/apache/paimon/flink/clone/CloneSourceBuilder.java Outdated Show resolved Hide resolved
...link/paimon-flink-common/src/main/java/org/apache/paimon/flink/clone/CloneSourceBuilder.java Outdated Show resolved Hide resolved
...imon-flink-common/src/main/java/org/apache/paimon/flink/clone/PickFilesForCloneOperator.java Outdated Show resolved Hide resolved
paimon-core/src/main/java/org/apache/paimon/clone/CopyFileUtils.java Outdated Show resolved Hide resolved
paimon-core/src/main/java/org/apache/paimon/clone/CopyFileUtils.java Outdated Show resolved Hide resolved
paimon-core/src/main/java/org/apache/paimon/clone/CopyFileUtils.java Outdated Show resolved Hide resolved
paimon-core/src/main/java/org/apache/paimon/clone/CloneFileInfo.java Outdated Show resolved Hide resolved
...nk/paimon-flink-common/src/main/java/org/apache/paimon/flink/clone/SnapshotHintOperator.java Outdated Show resolved Hide resolved
@wwj6591812 wwj6591812 force-pushed the support_clone_latest_snapshot_0429 branch from b67e04e to 034753f Compare May 13, 2024 12:46
tsreaper
tsreaper reviewed May 15, 2024
View reviewed changes
Copy link
Contributor

@tsreaper tsreaper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your IT cases should also include copying changelog, index files, etc. Everything you copy must be tested.

You'll also need a random test with very fast expiration speed to make sure your code really works when the source table is modified at the same time.

...ink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/CloneActionFactory.java Outdated Show resolved Hide resolved
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/CloneAction.java Show resolved Hide resolved
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/CloneAction.java Show resolved Hide resolved
...link/paimon-flink-common/src/main/java/org/apache/paimon/flink/clone/CloneSourceBuilder.java Outdated Show resolved Hide resolved
...link/paimon-flink-common/src/main/java/org/apache/paimon/flink/clone/CloneSourceBuilder.java Outdated Show resolved Hide resolved
...on-flink-common/src/main/java/org/apache/paimon/flink/clone/SnapshotHintChannelComputer.java Outdated Show resolved Hide resolved
...nk/paimon-flink-common/src/main/java/org/apache/paimon/flink/clone/SnapshotHintOperator.java Outdated Show resolved Hide resolved
...-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/clone/CopyFileOperator.java Show resolved Hide resolved
...aimon-flink-common/src/test/java/org/apache/paimon/flink/procedure/CloneProcedureITCase.java Outdated Show resolved Hide resolved
...link/paimon-flink-common/src/test/java/org/apache/paimon/flink/action/CloneActionITCase.java Outdated Show resolved Hide resolved
@wwj6591812 wwj6591812 force-pushed the support_clone_latest_snapshot_0429 branch from 034753f to 20abd9d Compare May 18, 2024 15:08
tsreaper
tsreaper reviewed May 20, 2024
View reviewed changes
...link/paimon-flink-common/src/main/java/org/apache/paimon/flink/procedure/CloneProcedure.java Outdated Show resolved Hide resolved
...mon-flink-common/src/test/java/org/apache/paimon/flink/procedure/CompactProcedureITCase.java Outdated Show resolved Hide resolved
...flink/paimon-flink-common/src/test/java/org/apache/paimon/flink/action/ActionITCaseBase.java Outdated Show resolved Hide resolved
...link/paimon-flink-common/src/test/java/org/apache/paimon/flink/action/CloneActionITCase.java Outdated Show resolved Hide resolved
...link/paimon-flink-common/src/test/java/org/apache/paimon/flink/action/CloneActionITCase.java Outdated Show resolved Hide resolved
...link/paimon-flink-common/src/test/java/org/apache/paimon/flink/action/CloneActionITCase.java Outdated Show resolved Hide resolved
@wwj6591812 wwj6591812 force-pushed the support_clone_latest_snapshot_0429 branch from 20abd9d to 915c01f Compare May 21, 2024 08:31
@wwj6591812 wwj6591812 force-pushed the support_clone_latest_snapshot_0429 branch from 915c01f to b51eb15 Compare May 21, 2024 13:33
@tsreaper tsreaper merged commit 97aa226 into apache:master Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JingsongLi JingsongLi JingsongLi left review comments

@tsreaper tsreaper tsreaper approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@wwj6591812 @JingsongLi @tsreaper