[WIP] OpenShift Mirror Registry Upgrade Proposal

[HammerMeetNail]

Work in progress for detailing how to perform upgrades with OMR

[wking]

Are these and the other non-update enhancement files attached to this pull request accidentally? I don't see how they are relevant to the mirror-registry-upgrade.md proposal.

[wking]

I'm not sure if you want to consider it in-scope for this particular proposal or not, but another maintenance task folks will want to perform with a long-running registry is pruning old images to avoid unbounded storage growth. In the OpenShift-mirror context, this looks like:

1. User decides to install OpenShift release oA.
2. User installs a new mirror registry, which happens to be version mA.
3. New OpenShift releases oB, oC, ... come out and get mirrored into the mirror registry.
4. New mirror-registry releases mB, mC, ... come out, and the admin updates their mirror to use those versions.
5. User realizes that they really don't need oA and oB in the mirror registry anymore, so they unpin them.
6. Registry reaper removes all the layers and manifests and whatnot that were only needed for those unpinned OpenShift releases.

(4) is definitely in-scope for this enhancement. I'm wondering whether (5) is in-scope. I assume (6) would work out of the box, but if not, I'm curious about whether that's in-scope here too.

(5) isn't strictly unique to the mirror-registry workflow. Folks might theoretically want that with highly-available registry implementations as well. But while a few GiB of stale content on a highly-available registry isn't that big a deal, that same amount of cruft on the single-node mirror-registry deployment seems like it might be something that folks would care about. And that's especially likely once we're throwing around TiBs.