Don't delete lambda versions if the stack is in UPDATE_ROLLBACK_FAILED state

[AndrejMitrovic]

Hi,

If the "keep most recent N versions" is set to 1 then it's possible the janitor will clean up lambda versions even if a cloudformation stack is stuck in UPDATE_ROLLBACK_FAILED state. In this case if the janitor happens to run before continuing rollback in cloudformation then this can cause the stack to be stuck as it won't be able to revert to the previous lambda versions (as they've since been deleted).

There's various ways to get it back to a sensible state, but it's rather cumbersome with the AWS console as it often fails to do so properly.

Normally the best solution is to keep a sensible value of at least 2 for the number of versions to keep.

Unfortunately in our case we were reaching the limits of Lambda storage and had to force ourselves to keep 1 lambda version instead of more.

---

Would it be possible to add a condition in the janitor to keep at least 2 lambda versions if the associated stack is in a failed state, or otherwise respect the configured value of 1 if the stack is in UPDATE_ROLLBACK_COMPLETE or UPDATE_COMPLETE?

I can understand if this is a big request, we can come up with alternatives in the meantime.

Cheers!

[AndrejMitrovic]

In the meantime it helped to only select Aliases under the Advanced option in the Continue update rollback action. That reverts the stack back to a state where it can be re-deployed by our CD.

Feel free to close this issue if the original idea isn't sensible.

Thanks!