-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Remote State] Create service to publish cluster state to remote store #9160
[Remote State] Create service to publish cluster state to remote store #9160
Conversation
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
server/src/main/java/org/opensearch/cluster/coordination/Coordinator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/repositories/blobstore/BlobStoreRepository.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/GatewayMetaState.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/GatewayMetaState.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/Coordinator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/Coordinator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/Coordinator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/store/ClusterMetadataMarker.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/store/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
Do rebase the branch once, current last commit is from Jul 4. We changed a few things related to remote store in past month. |
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
server/src/main/java/org/opensearch/cluster/store/ClusterMetadataMarker.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/store/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
server/src/main/java/org/opensearch/cluster/coordination/Coordinator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/GatewayMetaState.java
Outdated
Show resolved
Hide resolved
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:Checks if related components are compatible with change 5a25350 Incompatible componentsIncompatible components: [https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/asynchronous-search.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git] |
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Sooraj Sinha <[email protected]>
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Sooraj Sinha <[email protected]>
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:Checks if related components are compatible with change e386afc Incompatible componentsIncompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git] |
Compatibility status:Checks if related components are compatible with change 42af4dd Incompatible componentsIncompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git] |
Gradle Check (Jenkins) Run Completed with:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the changes @soosinha .
@soosinha Looks like we need to update the tests after recent commit -> Remove redundant remote state assertions Test Result (1 failure / -1)
|
Signed-off-by: Sooraj Sinha <[email protected]>
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:Checks if related components are compatible with change 99734f1 Incompatible componentsIncompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/reporting.git] |
Compatibility status:Checks if related components are compatible with change a07486b Incompatible componentsIncompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/reporting.git] |
Gradle Check (Jenkins) Run Completed with:
|
Gradle check failed because of flaky test here - #9624 |
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-9160-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 79e5aeed7adc5a02d3d974378d1b7f29df3ded3e
# Push it to GitHub
git push --set-upstream origin backport/backport-9160-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x Then, create a pull request where the |
opensearch-project#9160) * Upload all index metadata to remote store using BlobStoreRepository interface Signed-off-by: Sooraj Sinha <[email protected]> (cherry picked from commit 79e5aee)
opensearch-project#9160) * Upload all index metadata to remote store using BlobStoreRepository interface Signed-off-by: Sooraj Sinha <[email protected]> (cherry picked from commit 79e5aee)
#9160) (#9657) * Upload all index metadata to remote store using BlobStoreRepository interface Signed-off-by: Sooraj Sinha <[email protected]> (cherry picked from commit 79e5aee)
opensearch-project#9160) * Upload all index metadata to remote store using BlobStoreRepository interface Signed-off-by: Sooraj Sinha <[email protected]> Signed-off-by: Kaushal Kumar <[email protected]>
opensearch-project#9160) * Upload all index metadata to remote store using BlobStoreRepository interface Signed-off-by: Sooraj Sinha <[email protected]> Signed-off-by: Ivan Brusic <[email protected]>
opensearch-project#9160) * Upload all index metadata to remote store using BlobStoreRepository interface Signed-off-by: Sooraj Sinha <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>
Description
This change creates an RemoteClusterStateService orchestrates the upload of cluster state to remote store.
There are some pending items yet to be addressed:
Related Issues
#9336, #9337
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.