Initialize rate_limit_quota token buckets with a partial fill #37624
+130
−62
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Commit Message: When replacing a token bucket with another in a rate_limit_quota filter's bucket cache, initialize the replacement in a partially filled state, based on the ratio of available and max tokens of the existing token bucket.
Additional Description: This enables more frequent updates in RateLimitQuotaResponses when giving assignments for token buckets. Previously, updates to an assignment had a side effect of resetting the available tokens in the token bucket. This implementation doesn't perfectly account for asynchronous flows but is better than the implementation.
Risk Level: Low - the AtomicTokenBucketImpl class isn't widely used yet & changes are minimal. The rate_limit_quota filter is still in development.
Testing: Integration testing
Docs Changes:
Release Notes:
Platform Specific Features: