Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Snapshot expiration won't work, space usage and file number continue to grow #4743

Open
2 tasks done
bridgeDream opened this issue Dec 20, 2024 · 0 comments
Open
2 tasks done
Labels
bug Something isn't working

Comments

@bridgeDream
Copy link
Contributor

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

0.9

Compute Engine

flink 1.16

Minimal reproduce step

  1. starting a flink job writing to paimon with snapshot.time-retained configured.
  2. delete some snapshot (>10) between earliest and latest manually
  3. after a while, flink job while throw exception frequently like as
    image
  4. Snapshot expiration won't work anymore, Space usage and the number of files will continue to grow

What doesn't meet your expectations?

Even if the snapshot sequence is incomplete, Paimon should have a certain level of robustness to complete the snapshot expiration process rather than simply stop functioning.

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@bridgeDream bridgeDream added the bug Something isn't working label Dec 20, 2024
@bridgeDream bridgeDream changed the title [Bug] [Bug] Snapshot expiration won't work, space usage and file number continue to grow Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant