Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] [Remote Store] Make metadata store independent of data store #12919

Open
sachinpkale opened this issue Mar 26, 2024 · 0 comments
Labels
enhancement Enhancement or improvement to existing feature or request good first issue Good for newcomers Storage:Remote

Comments

@sachinpkale
Copy link
Member

Is your feature request related to a problem? Please describe

  • Currently, when we configure remote store for a cluster, both data and metadata (of segment, translog and cluster state) is stored in the same store.
  • But the access pattern and consistency requirements for data and metadata are different.
  • For example, data files are uploaded and downloaded by providing a filename whereas metadata is fetched based on certain characteristics such as recency.
  • Due to these differences, it is possible that metadata can be stored in a different data store (for example, a key-value store) as we store the data files in a blob store.

Describe the solution you'd like

The ideal solution should allow configuring a different remote store for metadata.

Related component

Storage:Remote

Describe alternatives you've considered

No response

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request good first issue Good for newcomers Storage:Remote
Projects
Status: Next (Next Quarter)
Development

No branches or pull requests

1 participant