Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[core] Reduce File IO for snapshot read #3849

Merged
merged 12 commits into from
Jul 30, 2024
Merged

[core] Reduce File IO for snapshot read #3849

merged 12 commits into from
Jul 30, 2024

Conversation

JingsongLi
Copy link
Contributor

@JingsongLi JingsongLi commented Jul 30, 2024 •
edited
Loading

Purpose

#3829 The next step is to avoid many IO operations in the code.

and we can also introduce cache for snapshots because snapshot files are immutable objects.

Cache is not added in this PR, maybe it is better to add cache in table instead of static. For example, branch fast_forward is not good.

Tests

API and Format

Documentation

yuzelin
yuzelin approved these changes Jul 30, 2024
View reviewed changes
Copy link
Contributor

@yuzelin yuzelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi changed the title [core] Introduce snapshot cache and reduce file io [core] Reduce File IO for snapshot read Jul 30, 2024
@JingsongLi JingsongLi merged commit d5290fc into apache:master Jul 30, 2024
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yuzelin yuzelin yuzelin approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@JingsongLi @yuzelin