[Backport 2.x] Backport SegmentReplication test muting missing from 2.x. #5945

mch2 · 2023-01-19T23:26:16Z

This PR backports 6 segrep PRs from main missing from 2.x and resolves conflicts. See commits for each PR - all of these are test muting other than Fix flaky Segment Replication test testStartReplicaAfterPrimaryIndexe….

There are a few PRs still missing backport that are larger and should go separately:

44aabe9 - remote xlog
d56965c

…ication feature (opensearch-project#3993) * [Segment Replication] Add snapshots tests with segment replication enabled Signed-off-by: Suraj Singh <[email protected]> * Fix spotless failures Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry, address review comments, add failover test Signed-off-by: Suraj Singh <[email protected]> * Fix spotless failures Signed-off-by: Suraj Singh <[email protected]> * Address review comments 2 Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Marc Handalian <[email protected]>

…project#5714) Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]>

…sDocs. (opensearch-project#5722) * Fix flaky SR test testStartReplicaAfterPrimaryIndexesDocs. This test was failing because we are validating post recovery if a shard is able to perform segrep while also performing validation if a passed in checkopint. In the post recovery test this checkpoint is always empty, yet the shard will be ahead of this checkpoint after docs are indexed. This change differentiates shard validation from checkpoint validation. Signed-off-by: Marc Handalian <[email protected]> Fix spotless. Signed-off-by: Marc Handalian <[email protected]> Fix testIsSegmentReplicationAllowed_WrongEngineType. Signed-off-by: Marc Handalian <[email protected]> Update warn logs in isSegmentReplicationAllowed. Signed-off-by: Marc Handalian <[email protected]> * PR feedback. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]>

Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Marc Handalian <[email protected]>

github-actions · 2023-01-19T23:52:39Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/9629/
CommitID: 2637744
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

dreamer-89 · 2023-01-19T23:56:02Z

...er/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationIT.java

@@ -598,6 +625,61 @@ public void testDeleteOperations() throws Exception {
        }
    }

+    @AwaitsFix(bugUrl = "https://github.com/opensearch-project/OpenSearch/issues/5669")


Couldn't comment on line containing test testCancelPrimaryAllocation. This also needs to be muted.
https://github.com/opensearch-project/OpenSearch/pull/5945/files#diff-befd32a159f6d02a5c753c4226800355efcbdbb6c126e761c9eae1d04d60ff59R188

will backport that PR directly after this cleanup.

...er/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationIT.java

server/src/internalClusterTest/java/org/opensearch/snapshots/SegmentReplicationSnapshotIT.java

Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]>

github-actions · 2023-01-20T00:35:14Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/9630/
CommitID: be30964

codecov-commenter · 2023-01-20T00:37:59Z

Codecov Report

Merging #5945 (be30964) into 2.x (cdb2b26) will decrease coverage by 0.01%.
The diff coverage is 50.00%.

❗ Current head be30964 differs from pull request most recent head fc78792. Consider uploading reports for the commit fc78792 to get more accurate results

@@             Coverage Diff              @@
##                2.x    #5945      +/-   ##
============================================
- Coverage     70.49%   70.48%   -0.01%     
- Complexity    58887    58927      +40     
============================================
  Files          4766     4766              
  Lines        282418   282429      +11     
  Branches      41130    41129       -1     
============================================
- Hits         199077   199066      -11     
- Misses        66670    66765      +95     
+ Partials      16671    16598      -73

Impacted Files	Coverage Δ
...ch/indices/cluster/IndicesClusterStateService.java	`64.67% <0.00%> (+1.43%)`	⬆️
...in/java/org/opensearch/index/shard/IndexShard.java	`69.96% <52.63%> (-0.70%)`	⬇️
...adonly/AddIndexBlockClusterStateUpdateRequest.java	`0.00% <0.00%> (-75.00%)`	⬇️
...readonly/TransportVerifyShardIndexBlockAction.java	`9.75% <0.00%> (-58.54%)`	⬇️
...ch/transport/ReceiveTimeoutTransportException.java	`50.00% <0.00%> (-50.00%)`	⬇️
...cluster/coordination/PendingClusterStateStats.java	`20.00% <0.00%> (-48.00%)`	⬇️
...opensearch/persistent/PersistentTasksExecutor.java	`22.22% <0.00%> (-44.45%)`	⬇️
...adcast/BroadcastShardOperationFailedException.java	`55.55% <0.00%> (-44.45%)`	⬇️
.../admin/cluster/reroute/ClusterRerouteResponse.java	`60.00% <0.00%> (-40.00%)`	⬇️
...indices/readonly/TransportAddIndexBlockAction.java	`20.68% <0.00%> (-37.94%)`	⬇️
... and 457 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

server/src/internalClusterTest/java/org/opensearch/snapshots/SegmentReplicationSnapshotIT.java

github-actions · 2023-01-20T01:01:24Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/9632/
CommitID: fc78792

….x. (#5945) * [Segment Replication] Add snapshot and restore tests for segment replication feature (#3993) * [Segment Replication] Add snapshots tests with segment replication enabled Signed-off-by: Suraj Singh <[email protected]> * Fix spotless failures Signed-off-by: Suraj Singh <[email protected]> * Add changelog entry, address review comments, add failover test Signed-off-by: Suraj Singh <[email protected]> * Fix spotless failures Signed-off-by: Suraj Singh <[email protected]> * Address review comments 2 Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Remove changelog update. Signed-off-by: Marc Handalian <[email protected]> * Mute flaky test testStartReplicaAfterPrimaryIndexesDocs. (#5714) Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Fix flaky Segment Replication test testStartReplicaAfterPrimaryIndexesDocs. (#5722) * Fix flaky SR test testStartReplicaAfterPrimaryIndexesDocs. This test was failing because we are validating post recovery if a shard is able to perform segrep while also performing validation if a passed in checkopint. In the post recovery test this checkpoint is always empty, yet the shard will be ahead of this checkpoint after docs are indexed. This change differentiates shard validation from checkpoint validation. Signed-off-by: Marc Handalian <[email protected]> Fix spotless. Signed-off-by: Marc Handalian <[email protected]> Fix testIsSegmentReplicationAllowed_WrongEngineType. Signed-off-by: Marc Handalian <[email protected]> Update warn logs in isSegmentReplicationAllowed. Signed-off-by: Marc Handalian <[email protected]> * PR feedback. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * [Segment Replication] Mute flaky tests (#5739) Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * [Segment Replication] Mute flaky tests (#5742) Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Fix spotless. Signed-off-by: Marc Handalian <[email protected]> * Muting flaky SegmentReplication ITs. (#5700) Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Marc Handalian <[email protected]> Co-authored-by: Suraj Singh <[email protected]>

dreamer-89 and others added 6 commits January 19, 2023 15:06

Remove changelog update.

b51b1a5

Signed-off-by: Marc Handalian <[email protected]>

Mute flaky test testStartReplicaAfterPrimaryIndexesDocs. (opensearch-…

a32993e

…project#5714) Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]>

[Segment Replication] Mute flaky tests (opensearch-project#5739)

4915d02

Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]>

[Segment Replication] Mute flaky tests (opensearch-project#5742)

2637744

Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]>

mch2 requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, gbbafna, setiah, kartg, kotwanikunal, nknize, owaiskazi19, adnapibar, Rishikesh1159, ryanbogan, saratvemulapalli, shwetathareja, dreamer-89, tlfeng, VachaShah and xuezhou25 as code owners January 19, 2023 23:26

mch2 added the skip-changelog label Jan 19, 2023

Fix spotless.

be30964

Signed-off-by: Marc Handalian <[email protected]>

andrross mentioned this pull request Jan 19, 2023

[Backport 2.x] Apply OpenSearchIntegTestCase::featureFlagSettings by default (#5940) #5944

Merged

dreamer-89 reviewed Jan 20, 2023

View reviewed changes

Muting flaky SegmentReplication ITs. (opensearch-project#5700)

fc78792

Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]>

dreamer-89 approved these changes Jan 20, 2023

View reviewed changes

server/src/internalClusterTest/java/org/opensearch/snapshots/SegmentReplicationSnapshotIT.java Show resolved Hide resolved

dreamer-89 merged commit 37f5c22 into opensearch-project:2.x Jan 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport 2.x] Backport SegmentReplication test muting missing from 2.x. #5945

[Backport 2.x] Backport SegmentReplication test muting missing from 2.x. #5945

mch2 commented Jan 19, 2023

github-actions bot commented Jan 19, 2023

dreamer-89 Jan 19, 2023 •

edited

Loading

mch2 Jan 20, 2023

github-actions bot commented Jan 20, 2023

codecov-commenter commented Jan 20, 2023

github-actions bot commented Jan 20, 2023

[Backport 2.x] Backport SegmentReplication test muting missing from 2.x. #5945

[Backport 2.x] Backport SegmentReplication test muting missing from 2.x. #5945

Conversation

mch2 commented Jan 19, 2023

github-actions bot commented Jan 19, 2023

Gradle Check (Jenkins) Run Completed with:

dreamer-89 Jan 19, 2023 • edited Loading

Choose a reason for hiding this comment

mch2 Jan 20, 2023

Choose a reason for hiding this comment

github-actions bot commented Jan 20, 2023

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Jan 20, 2023

Codecov Report

github-actions bot commented Jan 20, 2023

Gradle Check (Jenkins) Run Completed with:

dreamer-89 Jan 19, 2023 •

edited

Loading