Batch Async Fetcher class changes #8742

Gaurav614 · 2023-07-18T07:38:03Z

Description

This pull request is part of the improvement #5098

It will add a async batcher class that will send the broadcast operation to all nodes for fetching relevant shard data for a batch of shards. It is analogous to [AsyncShardFetch](

OpenSearch/server/src/main/java/org/opensearch/gateway/AsyncShardFetch.java

Line 75 in 2a1eff5

    
           public abstract class AsyncShardFetch<T extends BaseNodeResponse> implements Releasable {

) in terms of structure for fetching and handling of data. Please check inline comments in the PR for highlighted borrowed logic from it.

Related Issues

Resolves #8956

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-07-18T07:45:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/20385/
CommitID: a32b0ee
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/gateway/AsyncBatchShardFetch.java

amkhar

Thanks @Gaurav614 for raising this smaller PR. Added minor comments.

server/src/main/java/org/opensearch/gateway/AsyncBatchShardFetch.java

amkhar

@Gaurav614 Please re-verify the ignore nodes logic once. I'm good with other responses 👍🏼 .

server/src/main/java/org/opensearch/gateway/AsyncBatchShardFetch.java

github-actions · 2023-08-30T10:53:34Z

Compatibility status:

Checks if related components are compatible with change 7dc30a4

Incompatible components

Incompatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

github-actions · 2023-08-30T10:53:51Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23806/
CommitID: 63b0ed4
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Gaurav Chandani <[email protected]>

github-actions · 2023-08-30T11:07:43Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23808/
CommitID: 6c82091
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-30T11:08:33Z

Compatibility status:

Checks if related components are compatible with change 6c82091

Incompatible components

Skipped components

Compatible components

Signed-off-by: Gaurav Chandani <[email protected]>

github-actions · 2023-08-30T11:15:14Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23809/
CommitID: ea6e081
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Gaurav Chandani <[email protected]>

github-actions · 2023-08-30T11:19:15Z

Compatibility status:

Checks if related components are compatible with change ea6e081

Incompatible components

Skipped components

Compatible components

Compatible components: []

github-actions · 2023-08-30T11:34:12Z

Compatibility status:

Checks if related components are compatible with change 4def974

Incompatible components

Skipped components

Compatible components

Compatible components: []

github-actions · 2023-08-30T11:47:55Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23811/
CommitID: 4def974
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-30T12:27:24Z

Compatibility status:

Checks if related components are compatible with change ce39ac1

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

Signed-off-by: Gaurav Chandani <[email protected]>

github-actions · 2023-09-22T13:06:07Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/26111/
CommitID: f4e15e1
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/gateway/AsyncShardFetch.java

.../main/java/org/opensearch/action/admin/indices/shards/TransportIndicesShardStoresAction.java

server/src/main/java/org/opensearch/gateway/AsyncShardFetch.java

1. Instead of using Map<ShardId,String> in Fetcher class now using Map<ShardId,ShardAttributes> for making code more extensible 2. Added UT for newly added constructor in fetcher class 3. Renamed logKey to RerouteKey 4. Add IllegalStateException for non-batched fetch Signed-off-by: Gaurav Chandani <[email protected]>

github-actions · 2023-12-05T08:56:50Z

❌ Gradle check result for 6242d2e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Gaurav Chandani <[email protected]>

github-actions · 2023-12-05T12:03:18Z

❌ Gradle check result for 7dc30a4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/gateway/AsyncShardFetch.java

Signed-off-by: Shivansh Arora <[email protected]>

github-actions · 2023-12-19T08:09:46Z

❕ Gradle check result for 8554388: UNSTABLE

TEST FAILURES:

      2 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.classMethod
      1 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.testRefreshAfterCommit
      1 org.opensearch.action.admin.cluster.node.tasks.ResourceAwareTasksTests.testTaskResourceTrackingDuringTaskCancellation

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Signed-off-by: Gaurav Chandani <[email protected]>

github-actions · 2023-12-19T10:14:05Z

❕ Gradle check result for 8d56eca: UNSTABLE

TEST FAILURES:

      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

shwetathareja

no other major comments

server/src/main/java/org/opensearch/indices/store/ShardAttributes.java

server/src/main/java/org/opensearch/gateway/AsyncShardFetch.java

* Async Fetcher class changes Signed-off-by: Gaurav Chandani <[email protected]>

* Async Fetcher class changes Signed-off-by: Gaurav Chandani <[email protected]> (cherry picked from commit 63f4f13) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Async Fetcher class changes (cherry picked from commit 63f4f13) Signed-off-by: Gaurav Chandani <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Async Fetcher class changes Signed-off-by: Gaurav Chandani <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>

Gaurav614 commented Jul 18, 2023

View reviewed changes

Gaurav614 changed the title ~~Async Fetcher class changes~~ Batch Async Fetcher class changes Jul 18, 2023

Gaurav614 mentioned this pull request Jul 18, 2023

Async Batch shards changes for GatewayAllocator #8746

Merged

4 tasks

amkhar suggested changes Jul 19, 2023

View reviewed changes

This was referenced Jul 19, 2023

BaseGatewayShardAllocator changes for Assigning the batch of shards #8776

Merged

FetchData changes for primaries and replicas #8865

Closed

Allocation service changes for batch assignment #8888

Open

This was referenced Jul 26, 2023

Added transport action for bulk async shard fetch for primary shards #8218

Merged

[BUG] Async shard fetches taking up GBs of memory causing ClusterManager JVM to spike with large no. of shards (> 50k) #5098

Open

amkhar suggested changes Jul 26, 2023

View reviewed changes

This was referenced Jul 27, 2023

Add PrimaryShardBatchAllocator to take allocation decisions for a batch of shards #8916

Merged

[META] Cluster Manager Async Shard Fetch Revamp #8098

Open

shwetathareja reviewed Aug 10, 2023

View reviewed changes

server/src/main/java/org/opensearch/gateway/AsyncBatchShardFetch.java Outdated Show resolved Hide resolved

Gaurav614 added 3 commits August 30, 2023 16:31

Async Fetcher class changes

352fb55

Signed-off-by: Gaurav Chandani <[email protected]>

Removed AsyncShardBatchFetch class

43beda5

Signed-off-by: Gaurav Chandani <[email protected]>

Applied spotlesscheck

6c82091

Signed-off-by: Gaurav Chandani <[email protected]>

Gaurav614 force-pushed the batcher-PR branch from 63b0ed4 to 6c82091 Compare August 30, 2023 11:02

Missed deleting the older file

ea6e081

Signed-off-by: Gaurav Chandani <[email protected]>

Applied spotlesscheck on tests

4def974

Signed-off-by: Gaurav Chandani <[email protected]>

Merge branch 'opensearch-project:main' into batcher-PR

ce39ac1

Added non null checks and test changes for new constructor

f4e15e1

Signed-off-by: Gaurav Chandani <[email protected]>

github-actions bot added bug Something isn't working Cluster Manager labels Sep 22, 2023

vikasvb90 approved these changes Sep 27, 2023

View reviewed changes

khushbr reviewed Nov 29, 2023

View reviewed changes

Gaurav614 requested review from abbashus and adnapibar as code owners December 5, 2023 08:46

Apply spotless checked in AsyncShardFetch PR

7dc30a4

Signed-off-by: Gaurav Chandani <[email protected]>

khushbr approved these changes Dec 6, 2023

View reviewed changes

server/src/main/java/org/opensearch/gateway/AsyncShardFetch.java Show resolved Hide resolved

shiv0408 added a commit to Gaurav614/OpenSearch that referenced this pull request Dec 12, 2023

Remove ShardAttributes because added in opensearch-project#8742

4a62b19

Signed-off-by: Shivansh Arora <[email protected]>

Merge branch 'main' into batcher-PR

8554388

Added ShardAttributes UTs

8d56eca

Signed-off-by: Gaurav Chandani <[email protected]>

amkhar approved these changes Dec 22, 2023

View reviewed changes

shwetathareja reviewed Dec 27, 2023

View reviewed changes

server/src/main/java/org/opensearch/indices/store/ShardAttributes.java Show resolved Hide resolved

server/src/main/java/org/opensearch/gateway/AsyncShardFetch.java Show resolved Hide resolved

shwetathareja approved these changes Dec 29, 2023

View reviewed changes

shwetathareja added the skip-changelog label Dec 29, 2023

shwetathareja merged commit 63f4f13 into opensearch-project:main Jan 2, 2024
39 of 48 checks passed

rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024

Batch Async Fetcher class changes (opensearch-project#8742)

6930f68

* Async Fetcher class changes Signed-off-by: Gaurav Chandani <[email protected]>

amkhar added the backport 2.x Backport to 2.x branch label Mar 19, 2024

opensearch-trigger-bot bot mentioned this pull request Mar 19, 2024

[Backport 2.x] Batch Async Fetcher class changes #12757

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Batch Async Fetcher class changes #8742

Batch Async Fetcher class changes #8742

Gaurav614 commented Jul 18, 2023 •

edited

Loading

github-actions bot commented Jul 18, 2023

amkhar left a comment

amkhar left a comment •

edited

Loading

github-actions bot commented Aug 30, 2023 •

edited

Loading

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Sep 22, 2023

github-actions bot commented Dec 5, 2023

github-actions bot commented Dec 5, 2023

github-actions bot commented Dec 19, 2023

github-actions bot commented Dec 19, 2023

shwetathareja left a comment

Batch Async Fetcher class changes #8742

Batch Async Fetcher class changes #8742

Conversation

Gaurav614 commented Jul 18, 2023 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Jul 18, 2023

Gradle Check (Jenkins) Run Completed with:

amkhar left a comment

Choose a reason for hiding this comment

amkhar left a comment • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Aug 30, 2023 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 30, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 30, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 30, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 30, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Sep 22, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Dec 5, 2023

github-actions bot commented Dec 5, 2023

github-actions bot commented Dec 19, 2023

github-actions bot commented Dec 19, 2023

shwetathareja left a comment

Choose a reason for hiding this comment

Gaurav614 commented Jul 18, 2023 •

edited

Loading

amkhar left a comment •

edited

Loading

github-actions bot commented Aug 30, 2023 •

edited

Loading