AvgShedder #20
Closed
AvgShedder #20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…es for UniformLoadShedder (apache#16401) * Fixed the issue that newly added brokers could not be assigned bundles * skip bundle==1 * add test UniformLoadShedderTest * check style (cherry picked from commit 6ca9ee9)
…ache#16391) (cherry picked from commit 4d64e2e) Signed-off-by: Zixuan Liu <[email protected]>
…mLoadShedder (apache#16409) (cherry picked from commit 0f2060d) Signed-off-by: Zixuan Liu <[email protected]>
…geRate (apache#16096) (cherry picked from commit e6ef5ec)
…tation based on the least resource usage with weight (apache#16281) email discussion thread: https://lists.apache.org/thread/36vyyhndr4og175k2bz3mfdf5ctd2xky ### Motivation See PIP-182 apache#16274 ### Modifications See apache#16274 The main idea of the new strategy is to unify the requirement of load shedding strategy and bundle placement strategy, which consider the resource usage with weight, including historical observations. How to calculate a score for a broker ? - use its historical load and short-term load data with weight, which can solve the case of load jitter in a broker. How to select a broker for assignning bundle ? - select a broker based on which one has the least average resource usage with weight. - the random selection algorithm is better than the `minScore` among low load brokers, and use `loadBalancerAverageResourceUsageDifferenceThresholdShedderPercentage ` to adjust the size of the `randomization pool` (cherry picked from commit d5088e3) Signed-off-by: Zixuan Liu <[email protected]>
(cherry picked from commit fa8ec29) Signed-off-by: Zixuan Liu <[email protected]>
…nagerImpl.cleanupDeadBrokersData() (apache#16690) (cherry picked from commit 21a25bf)
…n doLoadShedding (apache#16476) * add NamespacePolicies check before unload bundle when doLoadShedding * fix typos Co-authored-by: nicklixinyang <[email protected]> (cherry picked from commit 0b315eb)
…apache#16953) (cherry picked from commit b516ed9)
(cherry picked from commit eab2bb5) Signed-off-by: Zixuan Liu <[email protected]>
… in ModularLoadManagerImpl (apache#19656) (cherry picked from commit 145e985) Signed-off-by: Zixuan Liu <[email protected]>
…est (apache#18664) (cherry picked from commit 58ad3d0)
When a broker in the cluster is not active, its load still remain in the memory of the Leader. If deployed in a container, brokers will scale up and down frequently, and the memory of the Leader gradually increase (cherry picked from commit 7e0a6c4) Signed-off-by: Zixuan Liu <[email protected]>
…ion when selectBroker use LeastResourceUsageWithWeight (apache#16573) * fix the zero value as denominator cause ArithmeticException * fix the zero value as denominator cause ArithmeticException Co-authored-by: nicklixinyang <[email protected]> (cherry picked from commit 99fbe22)
(cherry picked from commit e973388)
…eData access (apache#19889) Co-authored-by: lordcheng10 <[email protected]> (cherry picked from commit 19c8497)
…data. sec ver. (apache#20620) Co-authored-by: wangjinlong <[email protected]> (cherry picked from commit f2f0bf4)
apache#22786) (cherry picked from commit b0e8fe9) Signed-off-by: Zixuan Liu <[email protected]>
…ategy instance (apache#22827) (cherry picked from commit 1770cbc)
…edder (apache#22949) Co-authored-by: Kai Wang <[email protected]> Co-authored-by: Yunze Xu <[email protected]> (cherry picked from commit c160cc9) Signed-off-by: Zixuan Liu <[email protected]>
Signed-off-by: Zixuan Liu <[email protected]>
Signed-off-by: Zixuan Liu <[email protected]>
Signed-off-by: Zixuan Liu <[email protected]>
|
PASSED |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #xyz
Main Issue: #xyz
PIP: #xyz
Motivation
Modifications
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: