bumping staleness

[orenmazor]

Hey folks,

I'm toying around with the idea of being able to update staleness on a semaphore.

The particular use case is long running jobs (lets say a timeout of a day at most). Sometimes these jobs can get killed externally with no cleanup time, which leaves a lock laying around, and nothing can run until it expires.

I could use staleness, but I'm still in the same spot, because something can sit for a day before it hits the stale limit.

I'm thinking it might make sense to initially set my semaphore to a value that covers 95% of my runs (lets say 10 minutes) and then have the jobs themselves bump the staleness (lets say by a minute) for every piece of work they do. This way, if a job dies, the longest I have to wait before a stale cleanup would be a much shorter time.

I can happily PR this if there is interest.