docs: Attempt to flesh out the different release responsibilities #20851

RyanTheOptimist · 2022-04-15T17:49:23Z

Signed-off-by: Ryan Hamilton [email protected]

docs: Attempt to flesh out the different release responsibilitiesRisk Level:
Testing: N/A
Docs Changes: RELEASES.md
Release Notes: N/A
Platform Specific Features: N/A

Signed-off-by: Ryan Hamilton <[email protected]>

RyanTheOptimist · 2022-04-15T17:56:21Z

/assign @mattklein123

RyanTheOptimist · 2022-04-15T17:56:45Z

I have a few questions which I've noted as "TODO" in the doc. Hopefully we can sort them out :)

RELEASES.md

phlax · 2022-04-16T08:07:44Z

couple of general points re release/patches

i think its important that we keep the stable branches' CI working. If we dont it makes it harder to land patches and see actual breakages etc. It also makes it harder to backport anything as every PR requires a senior maintainer to push past breakages.

~related to this point. As release managers are not maintainers they generally cant land PRs - so even with a non-broken release branch, PRs need still to be assigned/reviewed.

Im wondering if it is worth having a rotation for a "release maintainer" that shadows the release manager, and can land PRs etc

Ideally we need to also account for non-security-patch releases on supported branches - if for no other reason than so we can address eg container issues in a timely fashion.

I think it would be a good thing to make it easier to land/test/release on the stable branches, outside of sec patches, and encouraging more bug-fix backports

wrt to the first point - i think we can/should add some scheduled CI to make sure the stable branches are passing

mattklein123

Thanks for working on this. I left one major comment about the stability/security split, and I think @phlax also left a lot of good comments. Not all of them are actionable in this PR but we could use them to guide some of the text and future work.

/wait

mattklein123 · 2022-04-18T15:56:37Z

RELEASES.md

+TODO(RyanTheOptimist): It seems to me that we have two kinds of releases and it would be helpful for
+this doc to make a clear distinction between them. For the sake of argument, I've called them
+"Major Release" (v1.22) and "Security Releases" (v1.22.1, etc). Does this terminology work?


I think we effectively have 3 (well 2 with 1 having 2 subtypes):

Major release

Minor release (security)

Minor release (stability)

The release mechanics for 2 and 3 are very similar, there is just more private coordination required for 2.

Oh, interesting. I wasn't aware that we did minor releases for non-security reasons. Are these ad-hoc releases, or are they scheduled in some way? Presumably we should talk about these releases in this doc somewhere?

They are basically ad-hoc and best effort. It's essentially if someone wants to do the work. I agree we should discuss this option and make it clear that it's best effort and requires people to volunteer for release work.

Ok, I've updated this section to hopefully make this point clear. WDYT?

RELEASES.md

mattklein123 · 2022-04-18T15:57:40Z

RELEASES.md

+Major releases are handled by Matt Klein ([mattklein123](https://github.com/mattklein123))
+or Alyssa Wilk ([alyssawilk](https://github.com/alyssawilk)) and do not involve any backports.


It would be nice if other people did this. :)

Completely agree :) Should we set up a rotation? I'd be happy to add a column to the schedule below.

Are there any steps in the stable release process with require a repo admin? (I seem to recall needing repo admin help from repo admins in the security release process).

Sure a rotation would be great. I think it might require an admin to actually make the release (I'm not sure) but that is a tiny step.

I talked to @alyssawilk about this and she proposed making this the responsibility of the on-call maintainer, which makes a lot of sense to me. It's not much work in the grand scheme of things so it should fit nicely on that person's plate.

RELEASES.md

Signed-off-by: Ryan Hamilton <[email protected]>

RyanTheOptimist

Thanks for the great comments, both of you!

RyanTheOptimist · 2022-04-18T18:46:50Z

RELEASES.md

+TODO(RyanTheOptimist): It seems to me that we have two kinds of releases and it would be helpful for
+this doc to make a clear distinction between them. For the sake of argument, I've called them
+"Major Release" (v1.22) and "Security Releases" (v1.22.1, etc). Does this terminology work?


Oh, interesting. I wasn't aware that we did minor releases for non-security reasons. Are these ad-hoc releases, or are they scheduled in some way? Presumably we should talk about these releases in this doc somewhere?

RELEASES.md

RyanTheOptimist · 2022-04-18T18:57:00Z

RELEASES.md

+Major releases are handled by Matt Klein ([mattklein123](https://github.com/mattklein123))
+or Alyssa Wilk ([alyssawilk](https://github.com/alyssawilk)) and do not involve any backports.


Completely agree :) Should we set up a rotation? I'd be happy to add a column to the schedule below.

Are there any steps in the stable release process with require a repo admin? (I seem to recall needing repo admin help from repo admins in the security release process).

RELEASES.md

phlax · 2022-04-25T07:52:35Z

Another issue i want to flag is around release versioning

Historically we had a ~bitrot issue where version histories contained links to removed code that was causing docs builds to fail (and a ton of workarounds to make links point to vaguely relevant current code)

As a result, i added "intersphinx mappings" which allows us to point to specific published versions in version histories, so the links should theoretically remain correct - they are essentially permalinked

This also however, complexifies the release process for a few reasons

we cant link to historical docs until they have been published
we want the current version history - ie the current changelog - to point to the current doctree now but use version links in the future
the mappings in docs/conf.py need to be maintained

all in all, it means that quite specific sets of steps are required for releasing either from main or from a release branch, and that the steps have to be done in roughly the correct order.

some of this can most likely be automated, but one difficulty is automating updating ref links in the version history file as its not easily parsable/editable programatically

i will try to figure out a smoother workflow and automate what i can. For now im going to manually clean up the version histories on main and current stable branches, and i guess take responsibility for making any preparations before releases, and cleaning up after

related (#20961 #20963 #20965 )

RyanTheOptimist · 2022-04-25T21:04:16Z

Another issue i want to flag is around release versioning

Historically we had a ~bitrot issue where version histories contained links to removed code that was causing docs builds to fail (and a ton of workarounds to make links point to vaguely relevant current code)

As a result, i added "intersphinx mappings" which allows us to point to specific published versions in version histories, so the links should theoretically remain correct - they are essentially permalinked

This also however, complexifies the release process for a few reasons

we cant link to historical docs until they have been published

we want the current version history - ie the current changelog - to point to the current doctree now but use version links in the future

the mappings in docs/conf.py need to be maintained

all in all, it means that quite specific sets of steps are required for releasing either from main or from a release branch, and that the steps have to be done in roughly the correct order.

some of this can most likely be automated, but one difficulty is automating updating ref links in the version history file as its not easily parsable/editable programatically

i will try to figure out a smoother workflow and automate what i can. For now im going to manually clean up the version histories on main and current stable branches, and i guess take responsibility for making any preparations before releases, and cleaning up after

related (#20961 #20963 #20965 )

Sounds great. It'd be great to write this up when you get this worked out (which I assume is already your plan :>)

RyanTheOptimist · 2022-04-25T21:05:07Z

~related to this point. As release managers are not maintainers they generally cant land PRs - so even with a non-broken release branch, PRs need still to be assigned/reviewed.

Im wondering if it is worth having a rotation for a "release maintainer" that shadows the release manager, and can land PRs etc

I think we can probably use the on-call maintainer for this role. WDYT?

Signed-off-by: Ryan Hamilton <[email protected]>

RyanTheOptimist · 2022-04-25T21:18:14Z

/assign @alyssawilk

phlax · 2022-04-26T12:28:13Z

Release steps

Posting some notes and steps i have taken re release

Release, versions and version mappings

we always want/need the current.rst to point to the "current" docs (for any branch)

on current main 1.23 is not yet published - likewise in a release branch, the repo doesnt have the .inv file/link for mapping that minor version, so links in all of the patch version eg v1.19.x version histories should only point to the docs for that release (or lesser versions where required)

Steps required

using the release branch that i have just worked on (v1.19.4), these are ~the steps i had to take

before release (on release branch):

Edit current.rst - "Pending" -> "date"
Edit VERSION(.txt) to remove -dev

after release (on release branch):

Copy current.rst -> docs/root/version_history/v1.19.4.rst
Add 1.19.4 to TOC in docs/root/version_history/version_history.rst
Blank current.rst and edit - "date" -> "Pending"
Edit VERSION(.txt) to readd -dev and increment version

after release (on main , or possibly on any newer branches)

Copy the added v1.19.4.rst file into main version_histories editing all ref:s to use v1.19: prefix
Update version_history.rst TOC
Update intersphinx_mapping in docs/conf.py
Sync .inv files (i have code to do this locally which i will publish asap - cf Add inventory fetcher to sphinx_runner for intersphinx_mappings toolshed#462)

(to be comprehensive we may wish to do ^^ step on all newer release branches not just main)

after release (on envoy-website)

Edit _data/versions.yaml to include 1.19.4 (in the case of new releases/EOL etc we may wish to cycle stable/archived here at this point)

regarding editing the ref: links the only time they need to be edited is when a release has been cut (and published) and only when copying the version history files into the newer (release) branch (ie main)

almost certainly a bunch of this can be automated altho there are some issues, eg:

editing the rst is non-trivial programatically
if we dont human edit/cycle versions etc then we need some "source of authority" as to what are the current stable versions etc

phlax · 2022-04-26T12:46:26Z

re "release maintainer" mho is that it would be better working on same cycle as release for a few reasons

provides the release manager (who may not know expectations, or how to do things) someone to ping
requires more of a release-cycle focus
- releases tend to be planned across weeks/months rather than a week
- backporting a single patch often takes longer than a week

just my 2p tho - i think anything we can do to make it clearer/easier to contribute is a good thing

we may also want to make more use of milestones for tracking (patch) releases and related tickets/blockers/etc

alyssawilk

Thanks for this clean up!

RELEASES.md

alyssawilk · 2022-04-26T12:53:25Z

RELEASES.md

+responsible for approving and merging backports. The Fix Lead is a member of the security
+team and is responsible for coordinating the overall release. This includes identifying
+issues to be fixed in the release, communications with the Envoy community, and the
+actual mechanics of the release itself.


do we want to link to the docs here? is there anything private in the doc?

AFIK the release manager generally responsible for doing backports for releases.

Also somewhere maybe we should outline all the responsibilities for the role? Right now they're docced up here https://docs.google.com/document/d/1AnIqmJlGlN0nZaxDme2uMjcO9VJxIokGDMYsq2IZM98/edit but we could fold the content in here and delete that doc?

Oh, I didn't realize that doc existed. I added a link to it. But I'd be happy to inline it here, if you think that would be better?

yeah I think inlining would be better but you're already voluntering for enough improvements so your call :-)

Will do! But I'll do it in a follow up so as not to drag this PR out any longer.

RELEASES.md

alyssawilk · 2022-04-26T13:01:01Z

RELEASES.md

+* Begin marshalling the ongoing PR flow in this repo. Ask maintainers to hold off merging any
+  particularly risky PRs until after the release is tagged. This is because we aim for main to be
+  at release candidate quality at all times.
+* Do a final check of the [release notes](docs/root/version_history/current.rst):


cc @phlax I assume we should update links to version-specific links in this step right?
Ryan maybe link to one of @phlax recent PRs for an example?

no - not here - this branch wants to keep links to its own version - we only update the ref: links after a release

in the case of a release from main (described as "major release" here) we want to update the ref: links for the v1.X.Y.rst files after the release has happened

this is because main is now on a new (pending) version so the just released version has now become historical - any future branches will also carry the updated ref: s

to use a concrete example - at the point of just happened v1.22.0:

when release happens current.rst has links with no version prefix

after release v1.22.0 is now historical so all links need to be updated on main

on the release/v1.22 branch (and patches cut from it) the links from all v1.22.x.rst files will always not have the link prefix as the version is contemporaneous

@phlax Can you suggest text to be added here?

i think the text here is fine - its below that we need to update....

phlax · 2022-04-26T16:48:50Z

re release process im experimenting with some CI which will at least flag some of the issues - like the recent one we had with a -dev release (#21022 )

as i work on it ill try and figure out any autofix/mate opportunities

Signed-off-by: Ryan Hamilton <[email protected]>

RELEASES.md

Signed-off-by: Ryan Hamilton <[email protected]>

phlax · 2022-04-26T18:04:58Z

RELEASES.md

+* Make sure we tweet the new release: either have Matt do it or email [email protected] and ask them to do an Envoy account
+  post.
+* Do a new PR to setup the next version
+  * Update [VERSION.txt](VERSION.txt) to the next development release. E.g., "1.7.0-dev".


* Edit `docs/conf.py` adding a version to `intersphinx_mapping`, eg: \```python 'v1.16': ('https://www.envoyproxy.io/docs/envoy/v1.16.0', None), \```

this should work for now - there is another step required to sync inventories and use a local copy - but some missing railway track for that

lets not worry about this for now and i will either try to automate or add relevant docs

if we dont resolve this, it will cause issues down the road, and has mostly been forgotten, so ill try to prioritize automating this update, or making it unnecessary

RELEASES.md

phlax · 2022-04-27T09:04:49Z

So these are examples of the PRs required before/after a patch release (in this case v1.20.3):

before

Close the release branch (make non-dev, fix version and update current)

#20824

after

this is not exactly the same as a main branch release but the principles are

also this example doesnt really have any :ref: links to update so also doesnt show that v well

alyssawilk

cool LGTM with optional addition mentioned. I'll defer to phlax for the rest :-)

alyssawilk · 2022-04-27T14:05:31Z

RELEASES.md

+responsible for approving and merging backports. The Fix Lead is a member of the security
+team and is responsible for coordinating the overall release. This includes identifying
+issues to be fixed in the release, communications with the Envoy community, and the
+actual mechanics of the release itself.


yeah I think inlining would be better but you're already voluntering for enough improvements so your call :-)

phlax

@RyanTheOptimist thanks for working on this

mostly, im feeling lets land and iterate

would be good to remove the instructions for updating the website now i think, as these will be almost immediately obsolete

i can update/remove anything else after as we change stuff

phlax · 2022-04-27T17:43:41Z

RELEASES.md

+  tag build to make sure that the final docker images get pushed along with
+  the final docs. The final documentation will end up in the
+  [envoy-website repository](https://github.com/envoyproxy/envoy-website/tree/main/docs/envoy).
+* Update the website ([example PR](https://github.com/envoyproxy/envoy-website/pull/148)) for the new release.


lets just remove this - its not quite correct, and is about to be obsolete

phlax · 2022-04-27T17:44:39Z

RELEASES.md

+* Make sure we tweet the new release: either have Matt do it or email [email protected] and ask them to do an Envoy account
+  post.
+* Do a new PR to setup the next version
+  * Update [VERSION.txt](VERSION.txt) to the next development release. E.g., "1.7.0-dev".


lets not worry about this for now and i will either try to automate or add relevant docs

if we dont resolve this, it will cause issues down the road, and has mostly been forgotten, so ill try to prioritize automating this update, or making it unnecessary

RELEASES.md

RyanTheOptimist

Thanks for the review @phlax
If this work for you, I'm happy to land it and we can iterate form there...

RyanTheOptimist · 2022-04-27T19:16:39Z

RELEASES.md

+responsible for approving and merging backports. The Fix Lead is a member of the security
+team and is responsible for coordinating the overall release. This includes identifying
+issues to be fixed in the release, communications with the Envoy community, and the
+actual mechanics of the release itself.


Will do! But I'll do it in a follow up so as not to drag this PR out any longer.

RyanTheOptimist · 2022-04-27T19:17:22Z

RELEASES.md

+  tag build to make sure that the final docker images get pushed along with
+  the final docs. The final documentation will end up in the
+  [envoy-website repository](https://github.com/envoyproxy/envoy-website/tree/main/docs/envoy).
+* Update the website ([example PR](https://github.com/envoyproxy/envoy-website/pull/148)) for the new release.


RELEASES.md

RyanTheOptimist · 2022-04-29T15:58:03Z

@mattklein123 Are you good with this merging and we can continue to iterate?

mattklein123

LGTM to ship and iterate! Thank you!

…voyproxy#20851) Signed-off-by: Ryan Hamilton <[email protected]>

RyanTheOptimist added 2 commits April 15, 2022 17:39

docs: Attempt to flesh out the different release responsibilities

71a5d41

Signed-off-by: Ryan Hamilton <[email protected]>

Reference GOVERNANCE.md

257b3a3

Signed-off-by: Ryan Hamilton <[email protected]>

repokitteh-read-only bot assigned mattklein123 Apr 15, 2022

phlax reviewed Apr 16, 2022

View reviewed changes

RELEASES.md Outdated Show resolved Hide resolved

mattklein123 requested changes Apr 18, 2022

View reviewed changes

repokitteh-read-only bot added the waiting label Apr 18, 2022

Address comments

a887c45

Signed-off-by: Ryan Hamilton <[email protected]>

repokitteh-read-only bot removed the waiting label Apr 18, 2022

RyanTheOptimist commented Apr 18, 2022

View reviewed changes

mattklein123 added the waiting label Apr 19, 2022

phlax added the area/release label Apr 25, 2022

Move stable release process from GOVERNANCE.md to RELEASES.md

32aeb51

Signed-off-by: Ryan Hamilton <[email protected]>

repokitteh-read-only bot removed the waiting label Apr 25, 2022

Merge branch 'main' into ReleaseDocs

34f3d3f

repokitteh-read-only bot assigned alyssawilk Apr 25, 2022

phlax mentioned this pull request Apr 26, 2022

docs: Cleanup version histories #20961

Merged

alyssawilk reviewed Apr 26, 2022

View reviewed changes

Address comments

58837c9

Signed-off-by: Ryan Hamilton <[email protected]>

phlax reviewed Apr 26, 2022

View reviewed changes

RELEASES.md Show resolved Hide resolved

New version steps

1db0a72

Signed-off-by: Ryan Hamilton <[email protected]>

phlax reviewed Apr 26, 2022

View reviewed changes

RELEASES.md Show resolved Hide resolved

alyssawilk approved these changes Apr 27, 2022

View reviewed changes

phlax reviewed Apr 27, 2022

View reviewed changes

RyanTheOptimist commented Apr 27, 2022

View reviewed changes

alyssawilk approved these changes Apr 29, 2022

View reviewed changes

mattklein123 approved these changes Apr 29, 2022

View reviewed changes

mattklein123 merged commit dba39ba into envoyproxy:main Apr 29, 2022

ravenblackx pushed a commit to ravenblackx/envoy that referenced this pull request Jun 8, 2022

docs: Attempt to flesh out the different release responsibilities (en…

3e05252

…voyproxy#20851) Signed-off-by: Ryan Hamilton <[email protected]>

		Major releases are handled by Matt Klein ([mattklein123](https://github.com/mattklein123))
		or Alyssa Wilk ([alyssawilk](https://github.com/alyssawilk)) and do not involve any backports.

docs: Attempt to flesh out the different release responsibilities #20851

docs: Attempt to flesh out the different release responsibilities #20851

Conversation

RyanTheOptimist commented Apr 15, 2022

RyanTheOptimist commented Apr 15, 2022

RyanTheOptimist commented Apr 15, 2022

phlax commented Apr 16, 2022 • edited Loading

mattklein123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RyanTheOptimist left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

phlax commented Apr 25, 2022 • edited Loading

RyanTheOptimist commented Apr 25, 2022

RyanTheOptimist commented Apr 25, 2022

RyanTheOptimist commented Apr 25, 2022

phlax commented Apr 26, 2022 • edited Loading

Release steps

Release, versions and version mappings

Steps required

phlax commented Apr 26, 2022 • edited Loading

alyssawilk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

phlax Apr 26, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

phlax commented Apr 26, 2022 • edited Loading

phlax Apr 26, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

phlax commented Apr 27, 2022 • edited Loading

before

Close the release branch (make non-dev, fix version and update current)

after

Update version_history/inventory on main

Update website docs versions

Re-open the release branch (make dev)

alyssawilk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

phlax left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RyanTheOptimist left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RyanTheOptimist commented Apr 29, 2022

mattklein123 left a comment

Choose a reason for hiding this comment

phlax commented Apr 16, 2022 •

edited

Loading

phlax commented Apr 25, 2022 •

edited

Loading

phlax commented Apr 26, 2022 •

edited

Loading

phlax commented Apr 26, 2022 •

edited

Loading

phlax Apr 26, 2022 •

edited

Loading

phlax commented Apr 26, 2022 •

edited

Loading

phlax Apr 26, 2022 •

edited

Loading

phlax commented Apr 27, 2022 •

edited

Loading

Update version_history/inventory on `main`