Server side caching (#193)

* #128 cache wip implementation Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 adjust to patron new version Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 refine requirements Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 first iteration of server cache functionality Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 server route cache implementation Signed-off-by: Vangelis Katikaridis <[email protected]> * Add validation for brokers in kafka.NewBuilder (#191) Signed-off-by: Stanislav Afanasev <[email protected]> * Add ActiveBrokers() method to Kafka AsyncProducer (#192) Signed-off-by: Giuseppe Mazzotta <[email protected]> * #128 revert sixth example main Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 update readme Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix linting Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix more linting Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix tests Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix tests Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 add vendor file Signed-off-by: Vangelis Katikaridis <[email protected]> * Add Route struct getters (#195) Signed-off-by: Alex Demin <[email protected]> * Introduce dockertest to integration tests (#182) Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 wrap up implementation Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix tests Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 add readme details on cache metrics Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 readme adjustments Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 finalise implementation Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 increase test timeout Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 decouple tests Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix tests Signed-off-by: Vangelis Katikaridis <[email protected]> * #193 fix tests Signed-off-by: Vangelis Katikaridis <[email protected]> * Update tracing and metrics dependencies (#190) Signed-off-by: Paschalis Tsilias <[email protected]> * #128 implement review comments Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 implement review comments Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix linting Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 update vendors Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 implement review comments Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 implement review comments Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 implement review comments Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 implement review comments Signed-off-by: Vangelis Katikaridis <[email protected]> * Upgraded github actions v2 (#198) Signed-off-by: Sotirios Mantziaris <[email protected]> * #128 make abstraction simpler and easier to use Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 add comment Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix linting Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 trigger the build Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 unexport response read writer Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 check for min < max Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 update vendor Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 remove timeinstant and make use of ttl cache only for route caching Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 update readme after latest changes Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix redis cache interaction and create cached route example Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 remove the processor specific logic for the cache Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 use the patron header alias instead of a raw map Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix linting Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 expose cache functionality as a middleware Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix linting Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 refactor cache logic to separate package Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 remove timeinstant abstraction Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 avoid defining the prometheus registerer Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix linting and exported objects Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 make middleware cache abstraction more concrete Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 make the naming conventions and structure more go like Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix imports Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 move timing request example to user service Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 fix typo Signed-off-by: Vangelis Katikaridis <[email protected]> * #128 add package doc Signed-off-by: Vangelis Katikaridis <[email protected]> Co-authored-by: Stanislav Afanasev <[email protected]> Co-authored-by: Giuseppe <[email protected]> Co-authored-by: Alexander Demin <[email protected]> Co-authored-by: Paschalis Tsilias <[email protected]> Co-authored-by: Sotirios Mantziaris <[email protected]>
beatlabs · Jul 24, 2020 · 7471d01 · 7471d01
1 parent f8ebb41
commit 7471d01
Show file tree

Hide file tree

Showing 20 changed files with 3,532 additions and 40 deletions.
diff --git a/README.md b/README.md
@@ -195,6 +195,131 @@ route := NewRoute("/index", "GET" ProcessorFunc, true, ...MiddlewareFunc)
 routeWithAuth := NewAuthRoute("/index", "GET" ProcessorFunc, true, Authendicator, ...MiddlewareFunc)
 ```
 
+### HTTP Caching
+
+The caching layer for HTTP routes is specified per Route.
+
+```go
+// RouteCache is the builder needed to build a cache for the corresponding route
+type RouteCache struct {
+	// cache is the ttl cache implementation to be used
+	cache cache.TTLCache
+	// age specifies the minimum and maximum amount for max-age and min-fresh header values respectively
+	// regarding the client cache-control requests in seconds
+	age age
+}
+
+func NewRouteCache(ttlCache cache.TTLCache, age Age) *RouteCache
+```
+
+#### server cache
+- The **cache key** is based on the route path and the url request parameters.
+- The server caches only **GET requests**.
+- The server implementation must specify an **Age** parameters upon construction.
+- Age with **Min=0** and **Max=0** effectively disables caching
+- The route should return always the most fresh object instance.
+- An **ETag header** must be always in responses that are part of the cache, representing the hash of the response.
+- Requests within the time-to-live threshold, will be served from the cache. 
+Otherwise the request will be handled as usual by the route processor function. 
+The resulting response will be cached for future requests.
+- Requests where the client control header requirements cannot be met i.e. **very low max-age** or **very high min-fresh** parameters,
+will be returned to the client with a `Warning` header present in the response. 
+
+```
+Note : When a cache is used, the handler execution might be skipped.
+That implies that all generic handler functionalities MUST be delegated to a custom middleware.
+i.e. counting number of server client requests etc ... 
+```
+
+### Usage
+
+- provide the cache in the route builder
+```go
+NewRouteBuilder("/", handler).
+	WithRouteCache(cache, http.Age{
+		Min: 30 * time.Minute,
+		Max: 1 * time.Hour,
+	}).
+    MethodGet()
+```
+
+- use the cache as a middleware
+```go
+NewRouteBuilder("/", handler).
+    WithMiddlewares(NewCachingMiddleware(NewRouteCache(cc, Age{Max: 10 * time.Second}))).
+    MethodGet()
+```
+
+#### client cache-control
+The client can control the cache with the appropriate Headers
+- `max-age=?` 
+
+returns the cached instance only if the age of the instance is lower than the max-age parameter.
+This parameter is bounded from below by the server option `minAge`.
+This is to avoid chatty clients with no cache control policy (or very aggressive max-age policy) to effectively disable the cache
+- `min-fresh=?` 
+
+returns the cached instance if the time left for expiration is lower than the provided parameter.
+This parameter is bounded from above by the server option `maxFresh`.
+This is to avoid chatty clients with no cache control policy (or very aggressive min-fresh policy) to effectively disable the cache
+
+- `no-cache` / `no-store`
+
+returns a new response to the client by executing the route processing function.
+NOTE : Except for cases where a `minAge` or `maxFresh` parameter has been specified in the server.
+This is again a safety mechanism to avoid 'aggressive' clients put unexpected load on the server.
+The server is responsible to cap the refresh time, BUT must respond with a `Warning` header in such a case.
+- `only-if-cached`
+
+expects any response that is found in the cache, otherwise returns an empty response
+
+#### metrics
+
+The http cache exposes several metrics, used to 
+- assess the state of the cache
+- help trim the optimal time-to-live policy
+- identify client control interference
+
+By default we are using prometheus as the the pre-defined metrics framework.
+
+- `additions = misses + evictions`
+
+Always , the cache addition operations (objects added to the cache), 
+must be equal to the misses (requests that were not cached) plus the evictions (expired objects).
+Otherwise we would expect to notice also an increased amount of errors or having the cache misbehaving in a different manner.
+
+- `additions ~ misses`
+
+If the additions and misses are comparable e.g. misses are almost as many as the additions, 
+it would point to some cleanup of the cache itself. In that case the cache seems to not be able to support
+the request patterns and control headers.
+
+- `hits ~ additions`
+
+The cache hit count represents how well the cache performs for the access patterns of client requests. 
+If this number is rather low e.g. comparable to the additions, 
+this would signify that probably a cache is not a good option for the access patterns at hand.
+
+- `eviction age`
+
+The age at which the objects are evicted from the cache is a very useful indicator. 
+If the vast amount of evictions are close to the time to live setting, it would indicate a nicely working cache.
+If we find that many evictions happen before the time to live threshold, clients would be making use cache-control headers.
+
+
+#### cache design reference
+- https://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
+- https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9
+
+#### improvement considerations
+- we can could the storing of the cached objects and their age counter. That way we would avoid loading the whole object in memory,
+if the object is already expired. This approach might provide considerable performance (in terms of memory utilisation) 
+improvement for big response objects. 
+- we could extend the metrics to use the key of the object as a label as well for more fine-grained tuning.
+But this has been left out for now, due to the potentially huge number of metric objects.
+We can review according to usage or make this optional in the future.
+- improve the serialization performance for the cache response objects
+
 ### Asynchronous
 
 The implementation of the async processor follows exactly the same principle as the sync processor.
@@ -370,4 +495,4 @@ GET /ready
 
 Both can return either a `200 OK` or a `503 Service Unavailable` status code (default: `200 OK`).
 
-It is possible to customize their behaviour by injecting an `http.AliveCheck` and/or an `http.ReadyCheck` `OptionFunc` to the HTTP component constructor.
+It is possible to customize their behaviour by injecting an `http.AliveCheck` and/or an `http.ReadyCheck` `OptionFunc` to the HTTP component constructor.