Memory leak from Redis due to missing key expirations

[yqiu24]

Hi, we are using ci-queue for minitest and noticed that the memory usage in our Redis cluster grows slowly over time, despite using the default TTL of 8 hours. It appears that this is because not all keys have a corresponding expire call. For example, the requeues-count key is never expired. Same thing for the running key. Is there any reason that only certain keys are marked as expired?

[casperisfine]

This is because ci-queue has been built around the idea of Redis being configued with maxmmemory-policy = allkeys-lru, as documented here: https://github.com/Shopify/ci-queue?tab=readme-ov-file#redis-requirements.

Later on some support was added to add TTL to keys, but some must have been missed: #184.