-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Disable MockTelemetryPlugin by default for integ tests #12877
Conversation
With this plugin enabled we see significant increase in memory usage. This is due to a static map that collects Span entries per write/replication request. Tests with higher counts of independent writes are at risk of running oom or experiencing slow down / warnings related to this. Disabled the plugin by default until this is resolved given it is still under FeatureFlag. Signed-off-by: Marc Handalian <[email protected]>
Compatibility status:Checks if related components are compatible with change 6a8dbd7 Incompatible componentsIncompatible components: [https://github.com/opensearch-project/security.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/performance-analyzer.git] |
❌ Gradle check result for 6a8dbd7: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
❕ Gradle check result for 6a8dbd7: UNSTABLE
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #12877 +/- ##
============================================
- Coverage 71.42% 71.31% -0.11%
- Complexity 59978 60183 +205
============================================
Files 4985 5011 +26
Lines 282275 283675 +1400
Branches 40946 41117 +171
============================================
+ Hits 201603 202294 +691
- Misses 63999 64597 +598
- Partials 16673 16784 +111 ☔ View full report in Codecov by Sentry. |
test/framework/src/main/java/org/opensearch/test/OpenSearchIntegTestCase.java
Show resolved
Hide resolved
@Gaganjuneja fyi |
Can we hold on this PR. I will try to fix within this week only. Otherwise we can go ahead with this PR. |
I would prefer not to wait. This could incorrectly flag a lot of tests as flaky that would take more time to analyze. We can re-enable once the fix is ready. |
|
With this plugin enabled we see significant increase in memory usage. This is due to a static map that collects Span entries per write/replication request. Tests with higher counts of independent writes are at risk of running oom or experiencing slow down / warnings related to this. Disabled the plugin by default until this is resolved given it is still under FeatureFlag. Signed-off-by: Marc Handalian <[email protected]> (cherry picked from commit 52ae79d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
With this plugin enabled we see significant increase in memory usage. This is due to a static map that collects Span entries per write/replication request. Tests with higher counts of independent writes are at risk of running oom or experiencing slow down / warnings related to this. Disabled the plugin by default until this is resolved given it is still under FeatureFlag. (cherry picked from commit 52ae79d) Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…oject#12877) With this plugin enabled we see significant increase in memory usage. This is due to a static map that collects Span entries per write/replication request. Tests with higher counts of independent writes are at risk of running oom or experiencing slow down / warnings related to this. Disabled the plugin by default until this is resolved given it is still under FeatureFlag. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>
…oject#12877) With this plugin enabled we see significant increase in memory usage. This is due to a static map that collects Span entries per write/replication request. Tests with higher counts of independent writes are at risk of running oom or experiencing slow down / warnings related to this. Disabled the plugin by default until this is resolved given it is still under FeatureFlag. Signed-off-by: Marc Handalian <[email protected]>
Description
With this plugin enabled I see significant increase in memory usage during ITs. This is due to a static map that collects Span entries per write/replication request and is not cleared until after the test execution. Tests with higher counts of independent writes are at risk of running oom or experiencing slow down / warnings related to this.
I've run two reported flaky tests on repeat with this disabled #12801 and #12836 and do not see failures.
Disabled the plugin by default until this is resolved given it is still under FeatureFlag.
Related Issues
related to #12615
Resolves #12801
Resolves #12836
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.