diff --git a/docs/process/release.adoc b/docs/process/release.adoc deleted file mode 100644 index c3e0d85991..0000000000 --- a/docs/process/release.adoc +++ /dev/null @@ -1,259 +0,0 @@ -= Release process - -This document describes the release process for `dfx`, with step-by-step instructions, information about automation, and a checklist. - -== Overview - -Before starting the release process, the team should conduct a brief Go/No-Go release review to evaluate the current state of fixes and features ready to be included in a release candidate. -After reviewing the list of fixes and features, the team will decide whether to proceed with staging a build. - -For now, our release process is driven by the DFINITY foundation SDK team. Future work needs to be done to separate the build and release process to run fully on open systems. - -If the new release is given the Go green light, two people who are familiar with the process—a *driver* and a *validator*—use the steps in this document to stage or promote a release candidate. - -The *validator* should be the person most familiar with the process and be able to assist with debugging or resolving issues if the *driver* building the release runs into trouble. - -A successful release is the result of coordination between automation, manual steps performed by team members, and a validation process. -Our goal is to update this document with the latest information as we iterate to improve the release process. - -=== Participants -- Driver / Primary -- Validator / Secondary - -=== Prerequisites - -Before you begin, verify the following prerequisites for the release process *driver*: - -- You must have a GitHub account and access to the `+dfinity+` organization and core repositories like the `sdk` repository -+ -For information about setting up a GitHub account, see link:https://docs.github.com/en/github/getting-started-with-github/signing-up-for-a-new-github-account[Signin up for a GitHub account] -To get permission for your account to access `+dfinity-lab+` and `+dfinity+` repositories, see link:https://www.notion.so/How-to-Get-Github-Access-68c9ad72b5974fa9bbec003592677d02[How to get GitHub access). -If you run into any issues accessing repositories, you can contact IT using the `#help-it` Slack channel and sending a message to `@firstresponder`. - -- You must have VPN access to the DFINITY VPN. -+ -If you don't have a VPN connection set up for your computer, you'll need to follow the instructions in link:https://www.notion.so/How-to-get-VPN-access-to-our-data-center-services-1c9b123152d740508eec25e7ac982259[How to get VPN access to our data center services]. - -- You must have link:https://github.com/dfinity-lab/dfinity/blob/master/CONTRIBUTING.adoc#install-and-configure-nix[Nix installed and configured]. -+ -NOTE: If you are installing Nix for the first time, building the cache can take hours and might timeout depending on the reliability of your network connection. - -- You must have a registered NPM account and `npm` installed on your system. -+ -If you don't have an account, you can go to the link:https://www.npmjs.com/[NPMJS] website and click *Sign Up* to create one using your `firstname.lastname@dfinity.org` email address. -+ -You will need to verify your email address to complete the registration process. -For your account to be added to the `dfinity` organization, contact IT - -=== Preliminary validation - -Verify the general stability of the master branch before attempting to create a release candidate. - -. Use this link:https://github.com/dfinity-lab/sdk/commits/master[link] -to verify: -.. Is `master` green? -.. Was `master` red recently or flaky? -+ - -image:is-master-green.png[] - -=== Preparation - -. Connect to VPN. -. Open a terminal and `cd` into your local copy of the `sdk` repo. - -=== Creating a New Release Branch - -. Check out the `master` branch and pull merged commits from the remote in your working directory by running the following command: -+ -[source, bash] ----- -git checkout master && git pull ----- - -. Create the release branch. Note that the branch name never includes alpha, beta, and so forth. All of these will be released on the same release branch. -+ -[source, bash] ----- -git switch -c release- && git push ----- -+ -For example, if creating the release branch for 0.7.0, you would run the following command: -+ -[source, bash] ----- -git switch -c release-0.7.0 && git push ----- - -. Edit CHANGELOG.md to remove the "UNRELEASED" note from the version to be released. Commit this to change to the release branch. - -=== Resuming on an Existing Release Branch - -. Check out the `master` branch and pull merged commits from the remote in your working directory by running the following command: -+ -[source, bash] ----- -git checkout release- && git pull ----- - -=== Ready - -At this point, you are ready to build a release candidate. -This is a script-based process. - -== SCRIPT-BASED release process - -To use the release script to automate building the release candidate: - -. Run the following command and substitute `` with the version number for this release candidate: -+ -[source, bash] ----- -./scripts/release.sh ----- -+ -For example, if releasing 0.7.0, you would run the following command: -+ -.... -./scripts/release.sh 0.7.0 -.... -. Follow the prompts displayed to complete the release process. - -After running this command to build the release candidate, follow the steps in <> to complete the release process. - - -[[post-build]] -== Notification and post-build validation - -. Wait for the publish workflow to complete. -. Install the build using the `DFX_VERSION=` environment variable. -. Run through the link:https://sdk.dfinity.org/docs/quickstart/local-quickstart.html[_Quick start - Local development_] steps. -. Run through link:https://sdk.dfinity.org/docs/quickstart/network-quickstart.html#ping-the-network[Check the connection to the network] and link:https://sdk.dfinity.org/docs/quickstart/network-quickstart.html#net-deploy[Register, build, and deploy the application] steps to deploy to the network. -. Notify link:https://app.slack.com/client/T43F9UHS5/CGA566TPV[#eng-sdk] team members that the new build is ready for manual installation and testing. -+ -Remind the SDK and Apps teams to add information about _features and fixes_ for release notes to their issues or PRs and to apply the changelog label to have the information included in the release notes. -. Notify the link:https://forum.dfinity.org[Developer Forum] community if there are breaking changes. -+ -If a release is known to have changes that are not backward-compatible, create a forum post to describe the change and any instructions for migrating to the new release. -+ -Depending on the change, the notification might need to be posted in more than one topic channel. -For example, changes to the external network for onboarded developers are currently posted in link:https://forum.dfinity.org/t/network-status-updates/928[Network status and updates] on the link:https://forum.dfinity.org[DFINITY Developer Forum]. - -== Add new frontend canister hash to list of WHITELISTED_WASMS in dfinity/motoko-playground repo - -Once the release is ready to promote, you need to register the new frontent canister hash with the Motoko Playground repo. - -You can do it either by using GitHub UI (https://github.com/dfinity/sdk/actions/workflows/broadcast-frontend-hash.yml) or by running the following command: -[source, bash] ----- -gh workflow run "broadcast-frontend-hash.yml" -f dfx_version= ----- - -== Promote a release candidate to production - -. Verify that release notes and documentation are ready for public consumption. -. Open the `public/manifest.json` file in a text editor. -. Verify that `dfx deploy --playground` works with an asset canister by e.g. deploying the default project created by `dfx new`. -.. If it doesn't work, make sure the PR created on the Motoko Playground repo is merged and deployed. -. Under the `tags` key, change the version number associated with the `latest` key. -+ -For example: -+ -[source,json] ----- -{ - "tags": { - "latest": "n.n.n" - } -} ----- -. Prepare a PR for the manifest by running the following commands: -+ -[source, bash] ----- -git switch -c /update-n.n.n-latest -git add --all -git commit --message "chore: Update the manifest latest to n.n.n " -git push origin /update-n.n.n-latest ----- -. After pushing, click the link in the console to go to your new branch in GitHub, then click *Create Pull Request*. -. Have the validator review and approve the PR, then merge to `master`. -. Verify the Linux and Darwin (macOS) builds are available for download from \https://download.dfinity.systems/sdk/dfx/n.n.n/architecture/dfx-n.n.n.tar.gz. -+ -Linux—Replace _n.n.n_ with the new version number and _architecture_ with `x86_64-linux`. For example, the following link would download version 0.6.1 for Linux: -+ -\https://download.dfinity.systems/sdk/dfx/0.6.1/x86_64-linux/dfx-0.6.1.tar.gz[] -+ -Darwin (macOS)—Replace _n.n.n_ with the new version number and _architecture_ with `x86_64-darwin`. For example, the following link would download version 0.6.1 for macOS: -+ -\https://download.dfinity.systems/sdk/dfx/0.6.1/x86_64-darwin/dfx-0.6.1.tar.gz[] -+ -CI Hydra: -+ -link:https://hydra.dfinity.systems/jobset/dfinity-ci-build/sdk-release[] - -Add a description and publish the tag for the latest release -https://github.com/dfinity-lab/sdk/releases/new?tag=${NEW_DFX_VERSION} - -== Release documentation - -link:https://github.com/dfinity/docs[Documentation repo] - -. Tag the documentation using `git tag -a -m `. -. Publish the tag on the remote server using `git push origin `. -. Deploy updated documentation using Netlify. - -== Requirements and properties - - - Semi-automation - - Consistent delivery - - Validation - - Rollback - - Guardrails - - Flexibility - -== Build mechanism - -Our build process is described in the `release.nix` derivation. -The `release.nix` derivation mainly invokes the `dfx-release` derivation passing the annotated tag on HEAD (which happens right now to be the stable branch). -The `dfx-release` derivation builds the release binaries and files for each platform and generates a manifest for S3 that includes the tag name. -The release tag allows us to keep a directory structure with all past and upcoming releases in S3. - -== CI - -CI release-related operation is split into two jobsets: - - - Generation and publishing of 'install.sh' and 'manifest.json'. - - Tagging of a commit to release, building and publishing the necessary executables and files for supported platforms. - -== Manifest - -We utilize a manifest to indicate to users (and in particular to our installer and dfx executable) available and supported versions for download. -The manifest allows us to rollback a release or remove a release from the list of supported releases. -See link:../specification/version_management{outfilesuffix}[Version Management] for details on the format of the manifest. - -The manifest is generated when a patch is applied on master by the CI. - -== Installer - -The installer is generated when a patch is applied on the `master` branch by the CI. - -== Changelog - -A candidate changelog is generated automatically using the respective tool (under scripts directory). -Currently, the release notes are updated manually in github. - -== Publishing of artifacts - -We now summarize the release process. -Our first step is to ensure the proper and valid state of the `master` branch. -Next, we update `cargo` and the manifest accordingly. -We then create and push an annotated tag on the `stable` branch, generate the changelog. -The product and SDK team members can then inspect, clarify, and develop the changelog to ensure it is appropriate for public -consumption. -After ensuring the proper artifacts are available in S3, we can now publish them by updating the manifest. - -== TODOs and improvements -. version from the tag -. release stress tests -. valid json test for the manifest diff --git a/docs/process/release.md b/docs/process/release.md new file mode 100644 index 0000000000..2f99161f23 --- /dev/null +++ b/docs/process/release.md @@ -0,0 +1,119 @@ +# IC SDK Release Process + +## Overview + +1. Update the replica version +1. Create the release branch +1. Update the changelog on master +1. Create beta releases +1. Open a PR to update the Portal +1. Create the final release +1. Open a PR to promote the release +1. Open a PR to update the Motoko Playground allow-list +1. Promote the release +1. Post to the forum + +## Details + +### Update the Replica Version + +Before making a new release, try to update the replica to the latest version +by running the [update-replica] workflow. + +### Create the Release Branch + +Create a release branch from `master`, for example `release-0.15.3`. + +This branch will be used to create beta releases as well as the final release. + +### Update the changelog + +Open a PR to master. Roll the changelog by adding a new header for the +new dfx version underneath the "# Unreleased" header. Further changes to dfx +should be added under the "#Unreleased" header, unless they are ported to +the release branch. + +### Create Beta Releases + +1. Check out the release branch. +1. Run the release script, for example `./scripts/release.sh 0.15.3-beta.1` +1. Open a PR from the branch pushed by the previous to the release branch, +obtain CR approval, and merge the PR. + - The release script will wait for you to do this + - It will then create and push a tag + - This triggers the [publish][publish-workflow] workflow +1. Wait for the [publish][publish-workflow] workflow to create the GitHub release +from the last commit on the release branch. +1. Update the GitHub release + - Copy/paste the changelog section for the new version into the release notes + - Set the "Prerelease" flag +1. Announce the release to #eng-sdk + - Post a message like this, linking to the GitHub release notes: + > dfx 0.15.3-beta.1 is available for manual installation and testing. + > + > ```bash + > DFX_VERSION=0.15.3-beta.1 sh -ci "$(curl -fsSL https://internetcomputer.org/install.sh)" + > ``` + > + > See also release notes. +1. Repeat the above steps until ready to promote the latest beta. + +### Open a Draft PR to update the Portal + +You can do this step while the beta releases are being tested. + +- Add a link to the [release-notes-table] +- Update the sdk submodule + +Obtain approval, but do not merge the PR yet. + +This PR is a draft in order to help remind the reviewer not to merge it. + +### Create the Final Release + +Once the beta releases are ready to be promoted: + +1. Check out the release branch +2. Run the release script, for example `./scripts/release.sh 0.15.3` +3. Follow the same steps as for the beta releases + +### Open a PR to promote the release + +1. Create a new branch from the release branch, for example `release-0.15.3-promote`. +1. Update the [version manifest][public-manifest]: + - Set `.tags.latest` to the new dfx version + - Remove the beta releases from the `versions` array +1. Open a PR from this branch to master + +Obtain approval, but do not merge this PR yet. + +### Open a PR to Update the Motoko Playground allow-list + +You can do it either by using GitHub UI ([broadcast-frontend-hash-workflow]) +or by running the following command: + +```bash +gh workflow run "broadcast-frontend-hash.yml" -f dfx_version= +``` + +Obtain approval, but do not merge this PR yet. + +### Promote the release + +You should now have three open, approved PRs: + +- Update the portal +- Promote the release +- Update the Motoko Playground allow-list + +Merge all three PRs. + +### Post to the forum + +Post a message to the forum, linking to the GitHub release notes. + +[broadcast-frontend-hash-workflow]: https://github.com/dfinity/sdk/actions/workflows/broadcast-frontend-hash.yml +[public-manifest]: https://github.com/dfinity/sdk/blob/master/public/manifest.json +[publish-workflow]: https://github.com/dfinity/sdk/blob/master/.github/workflows/publish.yml +[release-notes-table]: https://github.com/dfinity/portal/blob/master/docs/other/updates/release-notes/release-notes.md +[update-replica]: https://github.com/dfinity/sdk/actions/workflows/update-replica-version.yml