[Backport-2.x] Add remote segment transfer stats on NodesStats API #9301

shourya035 · 2023-08-14T13:28:14Z

Description

Backport #9168 to 2.x branch

Related Issues

N/A

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

opensearch-trigger-bot · 2023-08-14T13:57:52Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git]

BUILD SUCCESSFUL in 29m 12s

github-actions · 2023-08-14T14:13:57Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/22631/
CommitID: 1a28460
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot · 2023-08-14T15:20:02Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 30m 51s

github-actions · 2023-08-14T15:29:55Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/22649/
CommitID: 6193905
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot · 2023-08-14T17:21:28Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 24m 30s

github-actions · 2023-08-14T17:44:22Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/22669/
CommitID: c4983d2

codecov · 2023-08-14T17:45:19Z

Codecov Report

Merging #9301 (c5086db) into 2.x (2669362) will decrease coverage by 0.16%.
Report is 1 commits behind head on 2.x.
The diff coverage is 77.39%.

@@             Coverage Diff              @@
##                2.x    #9301      +/-   ##
============================================
- Coverage     70.95%   70.80%   -0.16%     
+ Complexity    57616    57593      -23     
============================================
  Files          4761     4762       +1     
  Lines        271849   271983     +134     
  Branches      40087    40093       +6     
============================================
- Hits         192901   192576     -325     
- Misses        62447    62939     +492     
+ Partials      16501    16468      -33

Files Changed	Coverage Δ
...arch/aggregations/DefaultAggregationProcessor.java	`13.04% <0.00%> (-0.92%)`	⬇️
...opensearch/search/query/QueryCollectorContext.java	`62.33% <0.00%> (-19.02%)`	⬇️
.../GlobalAggCollectorManagerWithSingleCollector.java	`66.66% <50.00%> (-4.77%)`	⬇️
...rch/aggregations/NonGlobalAggCollectorManager.java	`91.66% <50.00%> (-8.34%)`	⬇️
...nGlobalAggCollectorManagerWithSingleCollector.java	`66.66% <50.00%> (-4.77%)`	⬇️
...ava/org/opensearch/index/engine/SegmentsStats.java	`72.44% <72.72%> (-1.70%)`	⬇️
...rg/opensearch/index/remote/RemoteSegmentStats.java	`97.61% <97.61%> (ø)`
...in/java/org/opensearch/index/shard/IndexShard.java	`68.64% <100.00%> (-0.74%)`	⬇️
...arch/aggregations/AggregationCollectorManager.java	`88.46% <100.00%> (+9.89%)`	⬆️
...h/aggregations/ConcurrentAggregationProcessor.java	`87.87% <100.00%> (+6.62%)`	⬆️
... and 2 more

... and 465 files with indirect coverage changes

opensearch-trigger-bot · 2023-08-14T18:10:51Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 26m 57s

github-actions · 2023-08-14T18:24:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/22675/
CommitID: 0b714bb
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot · 2023-08-16T05:39:23Z

Compatibility status:

Checks if related components are compatible with change c6cd325

Incompatible components

Incompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

…pensearch-project#9168) --------- Signed-off-by: Shourya Dutta Biswas <[email protected]> (cherry picked from commit c9dbd90)

Signed-off-by: Shourya Dutta Biswas <[email protected]>

github-actions · 2023-08-16T05:44:07Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/22866/
CommitID: cfab182
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot · 2023-08-16T06:11:33Z

Compatibility status:

Checks if related components are compatible with change c6cd325

Incompatible components

Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

github-actions · 2023-08-16T06:25:37Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/22867/
CommitID: ce8d743

Signed-off-by: Shourya Dutta Biswas <[email protected]>

opensearch-trigger-bot · 2023-08-16T11:08:12Z

Compatibility status:

Checks if related components are compatible with change c6cd325

Incompatible components

Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git]

github-actions · 2023-08-16T11:12:27Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/22883/
CommitID: c5086db

shourya035 marked this pull request as ready for review August 14, 2023 14:49

shourya035 added 4 commits August 16, 2023 11:12

[Remote Store] Add remote segment transfer stats on NodesStats API (o…

649265d

…pensearch-project#9168) --------- Signed-off-by: Shourya Dutta Biswas <[email protected]> (cherry picked from commit c9dbd90)

Retrigger integs

6d9feca

Signed-off-by: Shourya Dutta Biswas <[email protected]>

Retrigger integs

d92731d

Signed-off-by: Shourya Dutta Biswas <[email protected]>

Retrigger integs

ce8d743

Signed-off-by: Shourya Dutta Biswas <[email protected]>

shourya035 force-pushed the backport/backport-9168-to-2.x branch from cfab182 to ce8d743 Compare August 16, 2023 05:43

BhumikaSaini-Amazon mentioned this pull request Aug 16, 2023

[Remote Store] Rename RemoteRefreshSegmentPressureSettings and RemoteRefreshSegmentPressureService #9387

Merged

6 tasks

Retrigger integs

c5086db

Signed-off-by: Shourya Dutta Biswas <[email protected]>

gbbafna approved these changes Aug 16, 2023

View reviewed changes

gbbafna merged commit a1dc776 into opensearch-project:2.x Aug 16, 2023

shourya035 mentioned this pull request Aug 16, 2023

[Remote Store] Changing remotestore based NodesStats metrics version to 2.10 #9389

Merged

6 tasks

shourya035 deleted the backport/backport-9168-to-2.x branch August 16, 2023 12:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport-2.x] Add remote segment transfer stats on NodesStats API #9301

[Backport-2.x] Add remote segment transfer stats on NodesStats API #9301

shourya035 commented Aug 14, 2023

opensearch-trigger-bot bot commented Aug 14, 2023

github-actions bot commented Aug 14, 2023

opensearch-trigger-bot bot commented Aug 14, 2023

github-actions bot commented Aug 14, 2023

opensearch-trigger-bot bot commented Aug 14, 2023

github-actions bot commented Aug 14, 2023

codecov bot commented Aug 14, 2023 •

edited

Loading

opensearch-trigger-bot bot commented Aug 14, 2023

github-actions bot commented Aug 14, 2023

opensearch-trigger-bot bot commented Aug 16, 2023

github-actions bot commented Aug 16, 2023

opensearch-trigger-bot bot commented Aug 16, 2023

github-actions bot commented Aug 16, 2023

opensearch-trigger-bot bot commented Aug 16, 2023

github-actions bot commented Aug 16, 2023

[Backport-2.x] Add remote segment transfer stats on NodesStats API #9301

[Backport-2.x] Add remote segment transfer stats on NodesStats API #9301

Conversation

shourya035 commented Aug 14, 2023

Description

Related Issues

Check List

opensearch-trigger-bot bot commented Aug 14, 2023

github-actions bot commented Aug 14, 2023

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Aug 14, 2023

github-actions bot commented Aug 14, 2023

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Aug 14, 2023

github-actions bot commented Aug 14, 2023

Gradle Check (Jenkins) Run Completed with:

codecov bot commented Aug 14, 2023 • edited Loading

Codecov Report

opensearch-trigger-bot bot commented Aug 14, 2023

github-actions bot commented Aug 14, 2023

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Aug 16, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 16, 2023

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Aug 16, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 16, 2023

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Aug 16, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 16, 2023

Gradle Check (Jenkins) Run Completed with:

codecov bot commented Aug 14, 2023 •

edited

Loading