Make blue/green more robust when cf routes fails

[severin]

If the cf routes command fails (e.g. when the pivotal API is having problems) then blue/green deployments always go to the blue host. This is bad when the blue host is the current live host...

Reproducing/how to fix:

• The live_color method may return nil (see https://github.com/madetech/cf-deploy/blob/master/lib/cf/deploy/commands.rb#L42), e.g. when cf routes fails...
• If the live_color is nil, then the idle_color is always "blue" (see https://github.com/madetech/cf-deploy/blob/master/lib/cf/deploy/blue_green.rb#L80-L86)
• Suggestion 1: raise an error and abort if live color cannot be determined
• Suggestion 2: change the implementation of idle_color to more thoroughly check the return value of live_color

[rrpff]

Good catch, thanks for raising this.

Can you think of many cases where cf routes would fail aside from problems with the Pivotal API? I wonder if it be worth retrying the request a few times before raising an error.

[severin]

It fails of course when you/cf is not logged in. But then everything fails and it is no problem ;) Retrying a couple of times might be worth it.

To share a bit more context: pivotal has/had problems this morning with their API and we ran into this problem: route lookup somehow failed, cf deploy targeted the blue host that was actually the live one, and cf push partially worked (it took the blue host offline to update it, but never brought it back up again)...