Change Remote state read thread pool to Fixed type #16850
+33
−5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
soosinha
requested review from
jed326,
peternied,
anasalkouz,
andrross,
ashking94,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
gbbafna,
jainankitk,
kotwanikunal,
linuxpi,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami and
VachaShah
as code owners
Signed-off-by: Sooraj Sinha <[email protected]>
soosinha
force-pushed
the
threadpool_configure
branch
from
5b0843e to
1c55c53
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #16850 +/- ##
============================================
+ Coverage 72.08% 72.21% +0.12%
- Complexity 65279 65323 +44
============================================
Files 5318 5318
Lines 304056 304068 +12
Branches 43992 43992
============================================
+ Hits 219188 219571 +383
+ Misses 66932 66503 -429
- Partials 17936 17994 +58 ☔ View full report in Codecov by Sentry. |
Bukhtawar
reviewed
Dec 16, 2024
Bukhtawar
approved these changes
Dec 17, 2024
Signed-off-by: Sooraj Sinha <[email protected]>
|
❕ Gradle check result for ab77523: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Remote state read thread pool is of Scaling type. So if for any reason multiple full cluster state downloads are attempted, it will keep filling up the queue and can cause other issues. So changing Remote state read thread pool to Fixed type with limit as queue size as 120,000. The queue size is chosen asssuming that there can be max of 50k indices. So total number of files will be around 100k (50k for index metadata and 50k for routing table). Adding some buffer to 100k and making it 120k.
Also, adding some more logs to understand when full state is downloaded.
Related Issues
NA
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.