Rate limiter

This is a generic rate limiter, which uses Redis as a backend to provide distributed rate limit.

IMP - Redis 6.2 or 6.2+ is required e.g. I have tested it on 6.2.6, and Redis should be Non-Clustered (if you are using AWS ElasticCache)

• Why 6.2 or 6.2+ => We use "getex" Redis call, which is not available before 6.2

---

Here is an example of a configuration to set up a rate limiter.

enabled: true                                   # If true rate limiter is on otherwise, any call to rate limit will be no-op
rate_limiters:                                  # List of rate limits - you can define many rate limits which will have their own 
                                                # rate limit counter
  externalCallRateLimit:                        # name of rate limit - you will refer using this name in your code
    enabled: true                               # Enable this rate limiter or now
    redis:                                      # Redis details
      host: localhost
      port: 6379
      timeout: 3000                             # Following are Redis timeouts e.g. timeout, connection timeout, etc
      connect_timeout: 10000
      idle_connection_timeout: 10000
      ping_connection_interval: 30000
    rate: 10                                    # What is the limit to apply (in per rate interval and rate interval unit)
    rate_interval: 1                            # Rate limit interval e.g. 1 sec, 1 min, etc
    rate_interval_unit: SECONDS                 # Unit of rate limit interval time 

---

DynamoDB-specific rate limiter

We sometimes want to ensure that, we do not go above some WCU. We may have provisioned the DDB which will scale, but it takes time. And we want to adjust the rate limit to increase automatically. This lib gives a way to achieve most of it

NOTE - "version: v3" only works with DDB and with rate N per/second is supported". Any other time unit will not work properly"

1. Automatically change the rate limit to match the WCU
2. ALlow rate limiting to adjust so that the burst capacity (unused WCU during last 5 min can be utilized) This means, you may have only 100 WCU, but if they are not used then DDB will allow you to consume this in a burst. In this case rate limiter will adjust and will allow you to go ahead
3. If anything goes wrong, the circuit will open and your code will not break. However, acquire call will be a no-op in this case.
   • maybe we can implement some kind of non-distributed rate limit as a fallback (TBD)

Sample config file with defaults and min configs

rate_limit_factory:
  enabled: true

  prefix: test_easy                      # This is important if you share the same Redis. This is to avoid namespace clash. Give 
                                         # different prefix to each service
                                         # all the data is stored with <prefix>-<some lib internal things>, and this prefix will avoid 
                                         # namespace collision 
  rate_limiters:
    example-config-normal:
      enabled: true

      redis:
        host: localhost
        port: 6379
        version: v3                      # v3 is where we have auto adjust rate limit. v1 is the basic rate limit which will only scale with 
                                         # WCU but will not adjust for burst and unused capacity    
      rate_type: OVERALL
      rate: 2
      rate_interval: 1                  # For the DDB use case, do not change it, it has to be 1 Sec. Not tested what will happen if this
                                        # is not the 1 sec 
      rate_interval_unit: SECONDS

      properties:
        enable-acquire-optimization: true    # This is an optimization to reduce the load in Redis. Try this if you see high CPU usage in 
                                             # redis  (recommended = set it to true)   
                                             # If most of the time you do not max out the WCU, (which is 99% use case) then setting it to 
                                             # true is very helpful. It will reduce almost all CPU in Redis 

      rate_limit_job_config:

        # If you use the AWS keys then update the following
        # AWS_ACCESS_KEY_ID: <AWS key>
        # AWS_SECRET_ACCESS_KEY: <AWS secret>

        refresh-time-in-sec: 10         # We need to check the actual DDB WCU value to adjust. This will tell how frequently we will do it                
        rate-limit-by-write: true
        rate-limit-class: io.github.devlibx.easy.ratelimit.job.ddb.DynamoDbWriteRateLimitJob

        rate-limit-factor: 1            # This is IMP. Suppose you have 100 WCU and you want the WCU to consume all then use 1. Suppose you 
                                        # want to leave some buffer e.g. even if I have 100 WCU never use > 90% capacity then you can 
                                        # adjust it by setting it 0.9
                                        # Simple logic - when I get the WCU from DDB, I just multiply that with this no and set it as the rate 
                                          limit
        region: AP_SOUTH_1              # (default=AP_SOUTH_1) Values from com.amazonaws.regions.Regions
        enabled: true                 
        table: test                     # Your table name  

Things to ensure:

1. You have Redis running at the boot (if Redis stops in between then we have circuit breakers to avoid the issue)
2. You may see higher CPU usage in Redis (use enable-acquire-optimization=true to solve it)
3. MUST - you must have DDB table describing permission (only describe is sufficient)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeTable"
                .... other read/write permissions ....
            ],
            "Resource": "*" <<<<<< Give access to your table
        }
    ]
}

Some results:

If you do not max out the WCU (which is 99+% of the time), then :

1. I was able to do 900+ writes per sec - started with 130+ to 900+ in 25+ min (over VPN which did not allow me to go above this)
2. Redis server did not cross 8% CPU

Why this happens

The rate limit is powered by a Redis Lua script which takes most of the Redis CPU. Buf if enable-acquire-optimization=true, then we do not run this script every time, instead use simple Redis command to get permits (which is a very low CPU process). That is why it is recommended to turn enable-acquire-optimization=true