Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Remote Store] Validate checksum of each segment file post succesful download #8740

Closed
sachinpkale opened this issue Jul 18, 2023 · 1 comment
Closed

[Remote Store] Validate checksum of each segment file post succesful download #8740

sachinpkale opened this issue Jul 18, 2023 · 1 comment
Assignees
@sachinpkale
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework Storage Issues and PRs relating to data and metadata storage v2.11.0 Issues and PRs related to version 2.11.0

Comments

@sachinpkale
Copy link
Member

Is your feature request related to a problem? Please describe.

  • Currently, we download all the files and rely on Lucene validations while creating SegmentInfos. This means we get to know about any checksum failures after downloading all the segment files.
  • If there is any checksum failure post download, we can fail fast. This will also enable us to add retry logic.
@sachinpkale sachinpkale added enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.10.0 labels Jul 18, 2023
@Bukhtawar Bukhtawar added the Storage Issues and PRs relating to data and metadata storage label Jul 27, 2023
@sachinpkale sachinpkale self-assigned this Aug 3, 2023
@sachinpkale sachinpkale added v2.11.0 Issues and PRs related to version 2.11.0 and removed v2.10.0 labels Sep 19, 2023
@sachinpkale sachinpkale mentioned this issue Sep 19, 2023
Merged
6 tasks
@sachinpkale
Copy link
Member Author

The change was pushed as part of #10119 but we reverted it due to performance issue #10847

@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Storage Project Board Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sachinpkale sachinpkale

Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework Storage Issues and PRs relating to data and metadata storage v2.11.0 Issues and PRs related to version 2.11.0
Projects
Storage Project Board
Status: ✅ Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sachinpkale @Bukhtawar