When Redis is used to cache large amounts of items, the least frequently used items can take most of the memory.
This proof of concept written in LUA allows to set 2 expiration dates for each item:
- A normal expiration date after which the item expires (traditional Redis expiration)
- A pre-expiration date after which the item expires if it has never been accessed.
This will address the situation where the long tail takes most of the memory:
A search engine uses Redis to cache query results. Each search can be cached with a normal expiration of 600 seconds and a pre-expiration of 60 seconds.
We have 2 searches: air shuttle and eat ice cream in an air shuttle.
air shuttle will be searched more than once every 60 seconds and it will expire at its normal expiration (600 seconds).
eat ice cream in an air shuttle is an uncommon search query and will not be searched again within the first 60 seconds. It will expire early.
Set an item.
set key value expiration pre_expiration
redis-cli --eval redis_elu.lua null null , set mykey myvalue 600 60
Output is similar to SET
Get an item.
get key
redis-cli --eval redis_elu.lua null null , get mykey
Output is similar to GET
Triggers expiration.
expire (limit)
-- expire all
redis-cli --eval redis_elu.lua null null , expire
-- expire 1,000 items maximum
redis-cli --eval redis_elu.lua null null , expire 1000
Count of deleted items.