Merging to release-5-lts: TT-9158 prefetch org expiry when loading the apis (#5798) #6014
Conversation
<!-- Provide a general summary of your changes in the Title above --> Previously, when operating in a slave configuration, the Tyk Gateway fetched session expiry information from the master layer the first time an API was accessed for a given organization. This approach led to a significant issue: if the MDCB connection was lost, the next API consumption attempt would incur a long response time. This delay, typically around 30 seconds, was caused by the gateway waiting for the session fetching operation to time out, as it tried to communicate with the now-inaccessible master layer. Implemented Solution: To mitigate this issue, the PR introduces a proactive fetching strategy. Now, the gateway fetches the session expiry information beforehand, while there is an active connection to MDCB. By doing so, it ensures that this data is already available locally in the event of an MDCB disconnection. Outcome: This change significantly improves the API response time under MDCB disconnection scenarios. It eliminates the need for the gateway to wait for a timeout when attempting to fetch session information from the master layer, thus avoiding the previous 30-second delay. This optimization enhances the resilience and efficiency of the Tyk Gateway in distributed environments. This option comes to complement the config option `slave_options.call_timeout` which defaults to 30sec, users can set a lower value. TT-9158 <!-- Why is this change required? What problem does it solve? --> <!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests --> <!-- you ran to see how your change affects other areas of the code, etc. --> <!-- This information is helpful for reviewers and QA. --> <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality) <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If there are no documentation updates required, mark the item as checked. --> <!-- Raise up any additional concerns not covered by the checklist. --> - [ ] I ensured that the documentation is up to date - [ ] I explained why this PR updates go.mod in detail with reasoning why it's required - [ ] I would like a code coverage CI quality gate exception and have explained why (cherry picked from commit 76b4847)
|
API Changes --- prev.txt 2024-02-05 13:11:26.804942065 +0000
+++ current.txt 2024-02-05 13:11:23.420968904 +0000
@@ -9919,6 +9919,157 @@
GetExp(string) (int64, error) // Returns expiry of a key
}
+type DummyStorage struct {
+ Data map[string]string
+ IndexList map[string][]string
+}
+ DummyStorage is a simple in-memory storage structure used for testing or
+ demonstration purposes. It simulates a storage system.
+
+func NewDummyStorage() *DummyStorage
+ NewDummyStorage creates and returns a new instance of DummyStorage.
+
+func (s *DummyStorage) AddToSet(string, string)
+ AddToSet adds a value to a set associated with a key in DummyStorage;
+ implementation pending.
+
+func (s *DummyStorage) AddToSortedSet(string, string, float64)
+ AddToSortedSet inserts a value with a score into a sorted set in
+ DummyStorage; implementation pending.
+
+func (s *DummyStorage) AppendToSet(keyName string, value string)
+ AppendToSet adds a new value to the end of a list associated with a key in
+ DummyStorage.
+
+func (s *DummyStorage) Connect() bool
+ Connect establishes a connection to the storage backend; not currently
+ implemented.
+
+func (s *DummyStorage) Decrement(string)
+ Decrement reduces the value of a specified key in DummyStorage;
+ implementation pending.
+
+func (s *DummyStorage) DeleteAllKeys() bool
+ DeleteAllKeys removes all keys and their associated data from the
+ DummyStorage. This method is intended to provide a way to clear the entire
+ storage, which can be particularly useful in testing scenarios to ensure a
+ clean state before tests.
+
+func (s *DummyStorage) DeleteKey(key string) bool
+ DeleteKey removes a specified key from DummyStorage, returning true if
+ successful.
+
+func (s *DummyStorage) DeleteKeys([]string) bool
+ DeleteKeys removes a list of keys from DummyStorage, returning a success
+ status; not yet implemented.
+
+func (s *DummyStorage) DeleteRawKey(string) bool
+ DeleteRawKey removes a specified key from DummyStorage, returning success
+ status; not yet implemented.
+
+func (s *DummyStorage) DeleteScanMatch(pattern string) bool
+ DeleteScanMatch deletes keys matching a pattern from DummyStorage, returning
+ true if successful.
+
+func (s *DummyStorage) Exists(keyName string) (bool, error)
+ Exists checks if a key exists in either the IndexList or Data in
+ DummyStorage; returns true if found.
+
+func (s *DummyStorage) GetAndDeleteSet(string) []interface{}
+ GetAndDeleteSet retrieves and then deletes a set associated with a key in
+ DummyStorage; not implemented.
+
+func (s *DummyStorage) GetExp(string) (int64, error)
+ GetExp retrieves the expiration time of a specific key from the
+ DummyStorage. This method accepts a string parameter representing the key
+ and returns an int64 which is the expiration time associated with that key,
+ along with an error.
+
+func (s *DummyStorage) GetKey(key string) (string, error)
+ GetKey retrieves the value for a given key from DummyStorage, or an error if
+ not found.
+
+func (s *DummyStorage) GetKeyPrefix() string
+ GetKeyPrefix returns the prefix used for keys in DummyStorage; not yet
+ implemented.
+
+func (s *DummyStorage) GetKeys(pattern string) (keys []string)
+ GetKeys retrieves all keys matching a specified pattern from DummyStorage;
+ currently supports only '*'.
+
+func (s *DummyStorage) GetKeysAndValues() map[string]string
+ GetKeysAndValues retrieves all key-value pairs from DummyStorage; currently
+ not implemented.
+
+func (s *DummyStorage) GetKeysAndValuesWithFilter(string) map[string]string
+ GetKeysAndValuesWithFilter fetches key-value pairs matching a filter from
+ DummyStorage; not implemented.
+
+func (s *DummyStorage) GetListRange(keyName string, _, _ int64) ([]string, error)
+ GetListRange retrieves a range of list elements from DummyStorage for a
+ specified key; returns an error if not found.
+
+func (s *DummyStorage) GetMultiKey(keys []string) ([]string, error)
+ GetMultiKey retrieves multiple values from the DummyStorage based on a slice
+ of keys. It returns a slice of strings containing the values corresponding
+ to each provided key, and an error if the operation cannot be completed.
+
+func (s *DummyStorage) GetRawKey(key string) (string, error)
+ GetRawKey retrieves the value associated with a given key from the
+ DummyStorage. The method accepts a single string as the key and returns the
+ corresponding string value. An error is also returned to indicate whether
+ the retrieval was successful. Currently, this method is not implemented and
+ will cause a panic if invoked.
+
+func (s *DummyStorage) GetRollingWindow(string, int64, bool) (int, []interface{})
+ GetRollingWindow retrieves data for a specified rolling window; currently
+ not implemented.
+
+func (s *DummyStorage) GetSet(string) (map[string]string, error)
+ GetSet retrieves a set of values associated with a key in DummyStorage;
+ not yet implemented.
+
+func (s *DummyStorage) GetSortedSetRange(string, string, string) ([]string, []float64, error)
+ GetSortedSetRange retrieves a range of values and scores from a sorted set
+ in DummyStorage; not implemented.
+
+func (s *DummyStorage) IncrememntWithExpire(string, int64) int64
+ IncrememntWithExpire increments the value of a key and sets an expiry;
+ not yet implemented.
+
+func (s *DummyStorage) RemoveFromList(keyName, value string) error
+ RemoveFromList eliminates a specific value from a list within DummyStorage;
+ always returns nil.
+
+func (s *DummyStorage) RemoveFromSet(string, string)
+ RemoveFromSet deletes a specific value from a set in DummyStorage; currently
+ not implemented.
+
+func (s *DummyStorage) RemoveSortedSetRange(string, string, string) error
+ RemoveSortedSetRange deletes a range of values from a sorted set in
+ DummyStorage; yet to be implemented.
+
+func (s *DummyStorage) SetExp(string, int64) error
+ SetExp updates the expiration time of a specific key in the DummyStorage.
+ This method accepts two parameters: a string representing the key, and an
+ int64 indicating the new expiration time.
+
+func (s *DummyStorage) SetKey(key, value string, _ int64) error
+ SetKey assigns a value to a key in DummyStorage with an expiration time;
+ returns nil for success.
+
+func (s *DummyStorage) SetRawKey(string, string, int64) error
+ SetRawKey stores a value with a specified key in the DummyStorage. It takes
+ three parameters: the key and value as strings, and an expiry time as int64.
+ The expiry time could be used to simulate time-sensitive data storage or
+ caching behavior. Currently, this method is not implemented and will trigger
+ a panic if it is called. TODO: Proper implementation is needed for this
+ method to handle data storage, or manage
+
+func (s *DummyStorage) SetRollingWindow(string, int64, string, bool) (int, []interface{})
+ SetRollingWindow sets a rolling window for a key with specified parameters;
+ implementation pending.
+
type Handler interface {
GetKey(string) (string, error) // Returned string is expected to be a JSON object (user.SessionState)
GetMultiKey([]string) ([]string, error)
@@ -10001,7 +10152,8 @@
func (m MdcbStorage) GetListRange(key string, from int64, to int64) ([]string, error)
-func (m MdcbStorage) GetMultiKey([]string) ([]string, error)
+func (m MdcbStorage) GetMultiKey(keyNames []string) ([]string, error)
+ GetMultiKey gets multiple keys from the MDCB layer
func (m MdcbStorage) GetRawKey(string) (string, error)
|
💥 CI tests failed 🙈git-stateall okPlease look at the run or in the Checks tab. |
|
API tests result: failure 🚫 |
|
API tests result: failure 🚫 |
|
API tests result: failure 🚫 |
…961997742c6a6b7086a02ad14
|
API tests result: failure 🚫 |
|
API tests result - mongo44-sha256 env: success ✅ TT-9158 prefetch org expiry when loading the apis (#5798) DescriptionPreviously, when operating in a slave configuration, the Tyk Gateway Implemented Solution: To mitigate this issue, the PR introduces a Outcome: This change significantly improves the API response time under This option comes to complement the config option Related IssueTT-9158 Motivation and ContextHow This Has Been TestedScreenshots (if appropriate)Types of changes
Checklist
Co-authored-by: Sredny M [email protected] |
|
API tests result - postgres15-sha256 env: success ✅ TT-9158 prefetch org expiry when loading the apis (#5798) DescriptionPreviously, when operating in a slave configuration, the Tyk Gateway Implemented Solution: To mitigate this issue, the PR introduces a Outcome: This change significantly improves the API response time under This option comes to complement the config option Related IssueTT-9158 Motivation and ContextHow This Has Been TestedScreenshots (if appropriate)Types of changes
Checklist
Co-authored-by: Sredny M [email protected] |
…961997742c6a6b7086a02ad14
|
sredxny
deleted the
merge/release-5-lts/76b4847da1a6588961997742c6a6b7086a02ad14
branch
TT-9158 prefetch org expiry when loading the apis (#5798)
Description
Previously, when operating in a slave configuration, the Tyk Gateway
fetched session expiry information from the master layer the first time
an API was accessed for a given organization. This approach led to a
significant issue: if the MDCB connection was lost, the next API
consumption attempt would incur a long response time. This delay,
typically around 30 seconds, was caused by the gateway waiting for the
session fetching operation to time out, as it tried to communicate with
the now-inaccessible master layer.
Implemented Solution: To mitigate this issue, the PR introduces a
proactive fetching strategy. Now, the gateway fetches the session expiry
information beforehand, while there is an active connection to MDCB. By
doing so, it ensures that this data is already available locally in the
event of an MDCB disconnection.
Outcome: This change significantly improves the API response time under
MDCB disconnection scenarios. It eliminates the need for the gateway to
wait for a timeout when attempting to fetch session information from the
master layer, thus avoiding the previous 30-second delay. This
optimization enhances the resilience and efficiency of the Tyk Gateway
in distributed environments.
This option comes to complement the config option
slave_options.call_timeoutwhich defaults to 30sec, users can set alower value.
Related Issue
TT-9158
Motivation and Context
How This Has Been Tested
Screenshots (if appropriate)
Types of changes
functionality to change)
coverage to functionality)
Checklist
why it's required
explained why