[TT-10749] New rate limiters (Leaky Bucket for 5.3) #5856
Conversation
|
API Changes --- prev.txt 2024-01-08 11:29:12.881178953 +0000
+++ current.txt 2024-01-08 11:29:09.637168090 +0000
@@ -4587,29 +4587,8 @@
// This is used as part of the RPC / Hybrid back-end configuration in a Tyk Enterprise installation and isn’t used anywhere else.
AuthOverride AuthOverrideConf `json:"auth_override"`
- // Redis based rate limiter with fixed window. Provides 100% rate limiting accuracy, but require two additional Redis roundtrip for each request.
- EnableRedisRollingLimiter bool `json:"enable_redis_rolling_limiter"`
-
- // To enable, set to `true`. The sentinel-based rate limiter delivers a smoother performance curve as rate-limit calculations happen off-thread, but a stricter time-out based cool-down for clients. For example, when a throttling action is triggered, they are required to cool-down for the period of the rate limit.
- // Disabling the sentinel based rate limiter will make rate-limit calculations happen on-thread and therefore offers a staggered cool-down and a smoother rate-limit experience for the client.
- // For example, you can slow your connection throughput to regain entry into your rate limit. This is more of a “throttle” than a “block”.
- // The standard rate limiter offers similar performance as the sentinel-based limiter. This is disabled by default.
- EnableSentinelRateLimiter bool `json:"enable_sentinel_rate_limiter"`
-
- // An enhancement for the Redis and Sentinel rate limiters, that offers a significant improvement in performance by not using transactions on Redis rate-limit buckets.
- EnableNonTransactionalRateLimiter bool `json:"enable_non_transactional_rate_limiter"`
-
- // How frequently a distributed rate limiter synchronises information between the Gateway nodes. Default: 2 seconds.
- DRLNotificationFrequency int `json:"drl_notification_frequency"`
-
- // A distributed rate limiter is inaccurate on small rate limits, and it will fallback to a Redis or Sentinel rate limiter on an individual user basis, if its rate limiter lower then threshold.
- // A Rate limiter threshold calculated using the following formula: `rate_threshold = drl_threshold * number_of_gateways`.
- // So you have 2 Gateways, and your threshold is set to 5, if a user rate limit is larger than 10, it will use the distributed rate limiter algorithm.
- // Default: 5
- DRLThreshold float64 `json:"drl_threshold"`
-
- // Controls which algorthm to use as a fallback when your distributed rate limiter can't be used.
- DRLEnableSentinelRateLimiter bool `json:"drl_enable_sentinel_rate_limiter"`
+ // RateLimit encapsulates rate limit configuration definitions.
+ RateLimit
// Allows you to dynamically configure analytics expiration on a per organisation level
EnforceOrgDataAge bool `json:"enforce_org_data_age"`
@@ -4928,7 +4907,11 @@
// ResourceSync configures mitigation strategy in case sync fails.
ResourceSync ResourceSyncConfig `json:"resource_sync"`
+ // Private contains configuration fields for internal app usage.
Private Private `json:"-"`
+
+ // DevelopmentConfig struct extends configuration for development builds.
+ DevelopmentConfig
}
Config is the configuration object used by Tyk to set up various parameters.
@@ -4936,10 +4919,18 @@
New produces a new config object by parsing the default configuration for
the values.
+func NewDefaultWithEnv() (*Config, error)
+ NewDefaultWithEnv gives a deep clone of the Default configuration and fills
+ it from environment provided.
+
func (c Config) GetEventTriggers() map[apidef.TykEvent][]TykEventHandler
GetEventTriggers returns event triggers. There was a typo in the json tag.
To maintain backward compatibility, this solution is chosen.
+func (c *Config) GetRateLimiterStorage() *StorageOptionsConf
+ GetRateLimiterStorage will return the storage configuration to use for rate
+ limiters.
+
func (c *Config) LoadIgnoredIPs()
func (c *Config) SetEventTriggers(eventTriggers map[apidef.TykEvent][]TykEventHandler)
@@ -5007,6 +4998,9 @@
Tags []string `json:"tags"`
}
+type DevelopmentConfig struct{}
+ DevelopmentConfig should contain no flags for official release builds.
+
type DnsCacheConfig struct {
// Setting this value to `true` will enable caching of DNS queries responses used for API endpoint’s host names. By default caching is disabled.
Enabled bool `json:"enabled"`
@@ -5247,6 +5241,43 @@
func (p Private) GetOAuthTokensPurgeInterval() time.Duration
GetOAuthTokensPurgeInterval returns purge interval for lapsed OAuth tokens.
+type RateLimit struct {
+ // EnableLeakyBucketRateLimiter enables leaky bucket rate limiting.
+ //
+ // LeakyBucket will delay requests so they are processed in a FIFO
+ // style queue, ensuring a constant request rate and smoothing out
+ // traffic spikes. This comes at some cost to gateway instances, as
+ // the connections would be held for a longer time, instead of
+ // blocking the requests when they go over the defined rate limits.
+ EnableLeakyBucketRateLimiter bool `json:"enable_leaky_bucket_rate_limiter"`
+
+ // Redis based rate limiter with fixed window. Provides 100% rate limiting accuracy, but require two additional Redis roundtrip for each request.
+ EnableRedisRollingLimiter bool `json:"enable_redis_rolling_limiter"`
+
+ // To enable, set to `true`. The sentinel-based rate limiter delivers a smoother performance curve as rate-limit calculations happen off-thread, but a stricter time-out based cool-down for clients. For example, when a throttling action is triggered, they are required to cool-down for the period of the rate limit.
+ // Disabling the sentinel based rate limiter will make rate-limit calculations happen on-thread and therefore offers a staggered cool-down and a smoother rate-limit experience for the client.
+ // For example, you can slow your connection throughput to regain entry into your rate limit. This is more of a “throttle” than a “block”.
+ // The standard rate limiter offers similar performance as the sentinel-based limiter. This is disabled by default.
+ EnableSentinelRateLimiter bool `json:"enable_sentinel_rate_limiter"`
+
+ // An enhancement for the Redis and Sentinel rate limiters, that offers a significant improvement in performance by not using transactions on Redis rate-limit buckets.
+ EnableNonTransactionalRateLimiter bool `json:"enable_non_transactional_rate_limiter"`
+
+ // How frequently a distributed rate limiter synchronises information between the Gateway nodes. Default: 2 seconds.
+ DRLNotificationFrequency int `json:"drl_notification_frequency"`
+
+ // A distributed rate limiter is inaccurate on small rate limits, and it will fallback to a Redis or Sentinel rate limiter on an individual user basis, if its rate limiter lower then threshold.
+ // A Rate limiter threshold calculated using the following formula: `rate_threshold = drl_threshold * number_of_gateways`.
+ // So you have 2 Gateways, and your threshold is set to 5, if a user rate limit is larger than 10, it will use the distributed rate limiter algorithm.
+ // Default: 5
+ DRLThreshold float64 `json:"drl_threshold"`
+
+ // Controls which algorthm to use as a fallback when your distributed rate limiter can't be used.
+ DRLEnableSentinelRateLimiter bool `json:"drl_enable_sentinel_rate_limiter"`
+}
+ RateLimit contains flags and configuration for controlling rate limiting
+ behaviour. It is embedded in the main config structure.
+
type Reporter struct {
// URL connection url to the zipkin server
URL string `json:"url"`
@@ -9079,12 +9110,20 @@
exclusively with user.SessionState objects, not identity
type SessionLimiter struct {
- Gw *Gateway `json:"-"`
// Has unexported fields.
}
SessionLimiter is the rate limiter for the API, use ForwardMessage() to
check if a message should pass through or not
+func NewSessionLimiter(ctx context.Context, conf *config.Config, drlManager *drl.DRL) SessionLimiter
+ NewSessionLimiter initializes the session limiter.
+
+ The session limiter initializes the storage required for rate limiters.
+ It supports two storage types: `redis` and `local`. If redis storage is
+ configured, then redis will be used. If local storage is configured,
+ then in-memory counters will be used. If no storage is configured, it falls
+ back onto the default gateway storage configuration.
+
func (l *SessionLimiter) Context() context.Context
func (l *SessionLimiter) ForwardMessage(r *http.Request, currentSession *user.SessionState, key string, store storage.Handler, enableRL, enableQ bool, globalConf *config.Config, api *APISpec, dryRun bool) sessionFailReason |
Apply Sweep Rules to your PR?
|
|
API tests result - postgres15-sha256 env: success ✅ This PR implements:
The following rate limiter implementations are added behind a build tag
The Makefile/Dockerfile (uses make) and CI tests have been adjusted to https://tyktech.atlassian.net/browse/TT-10749 Co-authored-by: Tit Petric [email protected] |
|
API tests result - postgres15-murmur64 env: success ✅ This PR implements:
The following rate limiter implementations are added behind a build tag
The Makefile/Dockerfile (uses make) and CI tests have been adjusted to https://tyktech.atlassian.net/browse/TT-10749 Co-authored-by: Tit Petric [email protected] |
|
API tests result - mongo44-sha256 env: success ✅ This PR implements:
The following rate limiter implementations are added behind a build tag
The Makefile/Dockerfile (uses make) and CI tests have been adjusted to https://tyktech.atlassian.net/browse/TT-10749 Co-authored-by: Tit Petric [email protected] |
|
API tests result - mongo44-murmur64 env: success ✅ This PR implements:
The following rate limiter implementations are added behind a build tag
The Makefile/Dockerfile (uses make) and CI tests have been adjusted to https://tyktech.atlassian.net/browse/TT-10749 Co-authored-by: Tit Petric [email protected] |
titpetric
changed the base branch from
master
to
fix/tt-10838/fix-config-load-priority
titpetric
force-pushed
the
poc/new-rate-limiters
branch
from
311e454 to
e109821
Compare
titpetric
force-pushed
the
fix/tt-10838/fix-config-load-priority
branch
3 times, most recently
from
84d830b to
f3023e0
Compare
titpetric
changed the title
titpetric
force-pushed
the
fix/tt-10838/fix-config-load-priority
branch
4 times, most recently
from
01b2a27 to
f095185
Compare
titpetric
changed the base branch from
fix/tt-10838/fix-config-load-priority
to
master
titpetric
force-pushed
the
poc/new-rate-limiters
branch
from
216ee43 to
94753cd
Compare
💥 CI tests failed 🙈git-stateall okPlease look at the run or in the Checks tab. |
💥 CI tests failed 🙈git-stateall okPlease look at the run or in the Checks tab. |
|
API tests result: failure 🚫 |
|
API tests result: failure 🚫 |
|
API tests result: failure 🚫 |
|
API tests result: failure 🚫 |
|
API tests result: failure 🚫 |
|
API tests result: failure 🚫 |
titpetric
force-pushed
the
poc/new-rate-limiters
branch
from
9f4fb8d to
5e3c89e
Compare
|
API tests result: failure 🚫 |
…gs, use dev build tag
titpetric
force-pushed
the
poc/new-rate-limiters
branch
from
cf00723 to
9787a43
Compare
|
titpetric
changed the title
This PR implements:
The following rate limiter implementations are added behind a build tag
-tags dev.The Makefile/Dockerfile (uses make) and CI tests have been adjusted to include the
devtag.https://tyktech.atlassian.net/browse/TT-10749