From 9970015353af8daa32a42dd8bf6f5b4a8ff534af Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Thu, 21 Nov 2024 11:35:47 -0800 Subject: [PATCH 01/11] initial update --- mkdocs/docs/how-to-release.md | 147 +++++++++++++++++++++------------- 1 file changed, 90 insertions(+), 57 deletions(-) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index f79f18ca8..b4b0a8cd6 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -17,13 +17,29 @@ - under the License. --> -# How to release +# How to Release -The guide to release PyIceberg. +This guide documents the process for releasing PyIceberg following the [Apache Release Process](https://infra.apache.org/release-publishing.html). The process involves: -The first step is to publish a release candidate (RC) and publish it to the public for testing and validation. Once the vote has passed on the RC, the RC turns into the new release. +1. Prepare for a release +2. Publishing a release candidate (RC) +3. Community voting and validation +4. Publishing the final release if the vote passes +5. Post release -## Preparing for a release +## Requirements + +* GPG key registered and published in [Apache Iceberg KEYS file](https://downloads.apache.org/iceberg/KEYS) +* PyPI + * `twine` installed for uploading to PyPi + * PyPI account with access to publish to the [pyiceberg project](https://pypi.org/project/pyiceberg/) +* SVN + * Access to Apache SVN for uploading artifacts to [Apache Iceberg dev/ dist](https://dist.apache.org/repos/dist/dev/iceberg/) (requires Apache Commmitter) + * Access to Apache SVN for uploading artifacts to [Apache Iceberg release/ dist](https://dist.apache.org/repos/dist/release/iceberg/) (requires Apache PMC) + +## Preparing for a Release + +### Remove Deprecated APIs Before running the release candidate, we want to remove any APIs that were marked for removal under the @deprecated tag for this release. @@ -48,23 +64,20 @@ deprecation_message( ) ``` -## Running a release candidate +### Update Library Version -Make sure that the version is correct in `pyproject.toml` and `pyiceberg/__init__.py`. Correct means that it reflects the version that you want to release. +Update the version in `pyproject.toml` and `pyiceberg/__init__.py` to match the release version. -### Setting the tag +## Release -Make sure that you're on the right branch, and the latest branch: +### Create Tag -For a Major/Minor release, make sure that you're on `main`, for patch versions the branch corresponding to the version that you want to patch, i.e. `pyiceberg-0.6.x`. +Make sure you are on the correct branch: -```bash -git checkout -git fetch --all -git reset --hard apache/ -``` +* For a Major/Minor release, make sure that you're on `main` +* For a Patch release, make sure that you're on the branch corresponding to the pathch version, i.e. `pyiceberg-0.6.x`. -Set the tag on the last commit: +Then, create a signed tag: ```bash export RC=rc1 @@ -74,47 +87,42 @@ export VERSION_BRANCH=${VERSION_WITHOUT_RC//./-} export GIT_TAG=pyiceberg-${VERSION} git tag -s ${GIT_TAG} -m "PyIceberg ${VERSION}" -git push apache ${GIT_TAG} - -export GIT_TAG_REF=$(git show-ref ${GIT_TAG}) -export GIT_TAG_HASH=${GIT_TAG_REF:0:40} -export LAST_COMMIT_ID=$(git rev-list ${GIT_TAG} 2> /dev/null | head -n 1) +git push git@github.com:apache/iceberg-python.git ${GIT_TAG} ``` -The `-s` option will sign the commit. If you don't have a key yet, you can find the instructions [here](http://www.apache.org/dev/openpgp.html#key-gen-generate-key). To install gpg on a M1 based Mac, a couple of additional steps are required: . -If you have not published your GPG key in [KEYS](https://downloads.apache.org/iceberg/KEYS) yet, you must publish it before sending the vote email by doing: - -```bash -svn co https://dist.apache.org/repos/dist/release/iceberg icebergsvn -cd icebergsvn -echo "" >> KEYS # append a newline -gpg --list-sigs >> KEYS # append signatures -gpg --armor --export >> KEYS # append public key block -svn commit -m "add key for " -``` +### Publish RC -### Upload to Apache SVN +#### Upload to Apache Dev SVN -Both the source distribution (`sdist`) and the binary distributions (`wheels`) need to be published for the RC. The wheels are convenient to avoid having people to install compilers locally. The downside is that each architecture requires its own wheel. [use `cibuildwheel`](https://github.com/pypa/cibuildwheel) runs in Github actions to create a wheel for each of the architectures. +##### Create Artifacts for SVN -Before committing the files to the Apache SVN artifact distribution SVN hashes need to be generated, and those need to be signed with gpg to make sure that they are authentic. +Run the [`Python release` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-release.yml). -Go to [Github Actions and run the `Python release` action](https://github.com/apache/iceberg-python/actions/workflows/python-release.yml). **Set the version to main, since we cannot modify the source**. +* Use the newly created tag. +* Set the `version` to `main`, since we cannot modify the source. ![Github Actions Run Workflow for SVN Upload](assets/images/ghactions-run-workflow-svn-upload.png) -Download the zip, and sign the files: +This will create the source distribution (`sdist`) and the binary distributions (`wheels`) for each architectures, using [`cibuildwheel`](https://github.com/pypa/cibuildwheel) + +##### Download Artifacts, Sign and Generate Checksums + +Before committing the files to the Apache SVN artifact distribution SVN hashes need to be generated, and those need to be signed with gpg to make sure that they are authentic. + +Download the zip file from the Github Action run, unzip, and sign the files: ```bash cd release-main/ for name in $(ls pyiceberg-*.whl pyiceberg-*.tar.gz) do - gpg --yes --armor --local-user fokko@apache.org --output "${name}.asc" --detach-sig "${name}" + gpg --yes --armor --output "${name}.asc" --detach-sig "${name}" shasum -a 512 "${name}" > "${name}.sha512" done ``` +##### Upload Artifacts to Apache Dev SVN + Now we can upload the files from the same directory: ```bash @@ -128,21 +136,40 @@ svn add $SVN_TMP_DIR_VERSIONED svn ci -m "PyIceberg ${VERSION}" ${SVN_TMP_DIR_VERSIONED} ``` -### Upload to PyPi +#### Upload to PyPi + +##### Create Artifacts for PyPi + +Run the [`Python release` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-release.yml). -Go to Github Actions and run the `Python release` action again. This time, set the **version** of the release candidate as the input: e.g. `0.7.0rc1`. Download the zip and unzip it locally. +* Use the newly created tag. +* Set the `version` to release candidate, e.g. `0.7.0rc1`. ![Github Actions Run Workflow for PyPi Upload](assets/images/ghactions-run-workflow-pypi-upload.png) -Next step is to upload them to pypi. Please keep in mind that this **won't** bump the version for everyone that hasn't pinned their version, since it is set to an RC [pre-release and those are ignored](https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#pre-release-versioning). +##### Download Artifacts + +Download the zip file from the Github Action run and unzip locally. + +##### Upload Artifacts to PyPi + +Upload release candidate to PyPi. This **won't** bump the version for everyone that hasn't pinned their version, since it is set to an RC [pre-release and those are ignored](https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#pre-release-versioning). ```bash -twine upload release-0.7.0rc1/* +twine upload release-${VERSION}/* ``` +## Vote + +### Generate Vote Email + Final step is to generate the email to the dev mail list: ```bash +export GIT_TAG_REF=$(git show-ref ${GIT_TAG}) +export GIT_TAG_HASH=${GIT_TAG_REF:0:40} +export LAST_COMMIT_ID=$(git rev-list ${GIT_TAG} 2> /dev/null | head -n 1) + cat << EOF > release-announcement-email.txt To: dev@iceberg.apache.org Subject: [VOTE] Release Apache PyIceberg $VERSION @@ -185,12 +212,19 @@ Please vote in the next 72 hours. [ ] +0 [ ] -1 Do not release this because... EOF - -cat release-announcement-email.txt ``` +### Send Vote Email + +Send the content of `release-announcement-email.txt` to `dev@iceberg.apache.org`. + +## Vote has failed + +If there are concerns with the RC, address the issues and generate another RC. + ## Vote has passed +A minimum of 3 binding +1 votes is required to pass an RC. Once the vote has been passed, you can close the vote thread by concluding it: ```text @@ -205,26 +239,21 @@ The release candidate has been accepted as PyIceberg . Thanks everyone, Kind regards, ``` -### Copy the artifacts to the release dist +### Upload the accepted RC to Apache Release SVN + -```bash -export RC=rc2 -export VERSION=0.7.0${RC} -export VERSION_WITHOUT_RC=${VERSION/rc?/} +!!! note + Only a PMC member has the permission to upload an artifact to the SVN release dist. + + +```bash export SVN_DEV_DIR_VERSIONED="https://dist.apache.org/repos/dist/dev/iceberg/pyiceberg-${VERSION}" export SVN_RELEASE_DIR_VERSIONED="https://dist.apache.org/repos/dist/release/iceberg/pyiceberg-${VERSION_WITHOUT_RC}" svn mv ${SVN_DEV_DIR_VERSIONED} ${SVN_RELEASE_DIR_VERSIONED} -m "PyIceberg: Add release ${VERSION_WITHOUT_RC}" ``` - - -!!! note - Only a PMC member has the permission to upload an artifact to the SVN release dist. - - - ### Upload the accepted release to PyPi The latest version can be pushed to PyPi. Check out the Apache SVN and make sure to publish the right version with `twine`: @@ -235,6 +264,10 @@ cd /tmp/iceberg-dist-release/pyiceberg-${VERSION_WITHOUT_RC} twine upload pyiceberg-*.whl pyiceberg-*.tar.gz ``` +## Post Release + +### Send out Release Announcement Email + Send out an announcement on the dev mail list: ```text @@ -253,9 +286,9 @@ This Python release can be downloaded from: https://pypi.org/project/pyiceberg/< Thanks to everyone for contributing! ``` -## Release the docs +### Release the docs -A committer triggers the [`Python Docs` Github Actions](https://github.com/apache/iceberg-python/actions/workflows/python-ci-docs.yml) through the UI by selecting the branch that just has been released. This will publish the new docs. +Run the [`Python Docs` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-ci-docs.yml). ## Update the Github template From e6635fb1f7d9a765e3ad9f8b502ad5466577ac2d Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Thu, 21 Nov 2024 11:46:58 -0800 Subject: [PATCH 02/11] edits --- mkdocs/docs/how-to-release.md | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index b4b0a8cd6..88e4e2dfb 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -21,7 +21,7 @@ This guide documents the process for releasing PyIceberg following the [Apache Release Process](https://infra.apache.org/release-publishing.html). The process involves: -1. Prepare for a release +1. Preparing for a release 2. Publishing a release candidate (RC) 3. Community voting and validation 4. Publishing the final release if the vote passes @@ -30,18 +30,18 @@ This guide documents the process for releasing PyIceberg following the [Apache R ## Requirements * GPG key registered and published in [Apache Iceberg KEYS file](https://downloads.apache.org/iceberg/KEYS) +* SVN + * Access to Apache SVN for uploading artifacts to [Apache Iceberg dev dist](https://dist.apache.org/repos/dist/dev/iceberg/) (requires Apache Commmitter) + * Access to Apache SVN for uploading artifacts to [Apache Iceberg release dist](https://dist.apache.org/repos/dist/release/iceberg/) (requires Apache PMC) * PyPI * `twine` installed for uploading to PyPi * PyPI account with access to publish to the [pyiceberg project](https://pypi.org/project/pyiceberg/) -* SVN - * Access to Apache SVN for uploading artifacts to [Apache Iceberg dev/ dist](https://dist.apache.org/repos/dist/dev/iceberg/) (requires Apache Commmitter) - * Access to Apache SVN for uploading artifacts to [Apache Iceberg release/ dist](https://dist.apache.org/repos/dist/release/iceberg/) (requires Apache PMC) ## Preparing for a Release ### Remove Deprecated APIs -Before running the release candidate, we want to remove any APIs that were marked for removal under the @deprecated tag for this release. +Before running the release candidate, we want to remove any APIs that were marked for removal under the `@deprecated` tag for this release. For example, the API with the following deprecation tag should be removed when preparing for the 0.2.0 release. @@ -68,17 +68,19 @@ deprecation_message( Update the version in `pyproject.toml` and `pyiceberg/__init__.py` to match the release version. -## Release +## Publishing a release candidate (RC) ### Create Tag Make sure you are on the correct branch: -* For a Major/Minor release, make sure that you're on `main` -* For a Patch release, make sure that you're on the branch corresponding to the pathch version, i.e. `pyiceberg-0.6.x`. +* For a Major/Minor release, use the `main` branch +* For a Patch release, use the branch corresponding to the pathch version, i.e. `pyiceberg-0.6.x`. Then, create a signed tag: +Change `VERSION` and `RC` to the appropriate values. + ```bash export RC=rc1 export VERSION=0.7.0${RC} @@ -123,7 +125,7 @@ done ##### Upload Artifacts to Apache Dev SVN -Now we can upload the files from the same directory: +Now, upload the files from the same directory: ```bash export SVN_TMP_DIR=/tmp/iceberg-${VERSION_BRANCH}/ @@ -216,13 +218,13 @@ EOF ### Send Vote Email -Send the content of `release-announcement-email.txt` to `dev@iceberg.apache.org`. +Verify the content of `release-announcement-email.txt` and send it to `dev@iceberg.apache.org` with the corresponding subject line. ## Vote has failed If there are concerns with the RC, address the issues and generate another RC. -## Vote has passed +## Publish the Final Release (Vote has passed) A minimum of 3 binding +1 votes is required to pass an RC. Once the vote has been passed, you can close the vote thread by concluding it: @@ -290,15 +292,15 @@ Thanks to everyone for contributing! Run the [`Python Docs` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-ci-docs.yml). -## Update the Github template +### Update the Github template Make sure to create a PR to update the [GitHub issues template](https://github.com/apache/iceberg-python/blob/main/.github/ISSUE_TEMPLATE/iceberg_bug_report.yml) with the latest version. -## Update the integration tests +### Update the integration tests Ensure to update the `PYICEBERG_VERSION` in the [Dockerfile](https://github.com/apache/iceberg-python/blob/main/dev/Dockerfile). -## Create a Github Release Note +### Create a Github Release Note Create a [new Release Note](https://github.com/apache/iceberg-python/releases/new) on the iceberg-python Github repository. From 8cc90306b15bd3572501d327f3cf9c887486a4dd Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Thu, 21 Nov 2024 11:55:13 -0800 Subject: [PATCH 03/11] add gpg instructions --- mkdocs/docs/how-to-release.md | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index 88e4e2dfb..2abcc2af5 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -29,7 +29,7 @@ This guide documents the process for releasing PyIceberg following the [Apache R ## Requirements -* GPG key registered and published in [Apache Iceberg KEYS file](https://downloads.apache.org/iceberg/KEYS) +* GPG key registered and published in [Apache Iceberg KEYS file](https://downloads.apache.org/iceberg/KEYS). See instructions at [Set up GPG key and Upload to Apache Iceberg KEYS file](#set-up-gpg-key-and-upload-to-apache-iceberg-keys-file). * SVN * Access to Apache SVN for uploading artifacts to [Apache Iceberg dev dist](https://dist.apache.org/repos/dist/dev/iceberg/) (requires Apache Commmitter) * Access to Apache SVN for uploading artifacts to [Apache Iceberg release dist](https://dist.apache.org/repos/dist/release/iceberg/) (requires Apache PMC) @@ -41,7 +41,7 @@ This guide documents the process for releasing PyIceberg following the [Apache R ### Remove Deprecated APIs -Before running the release candidate, we want to remove any APIs that were marked for removal under the `@deprecated` tag for this release. +Before running the release candidate, we want to remove any APIs that were marked for removal under the `@deprecated` tag for this release. See [#1269](https://github.com/apache/iceberg-python/pull/1269). For example, the API with the following deprecation tag should be removed when preparing for the 0.2.0 release. @@ -66,7 +66,7 @@ deprecation_message( ### Update Library Version -Update the version in `pyproject.toml` and `pyiceberg/__init__.py` to match the release version. +Update the version in `pyproject.toml` and `pyiceberg/__init__.py` to match the release version. See [#1276](https://github.com/apache/iceberg-python/pull/1276). ## Publishing a release candidate (RC) @@ -313,3 +313,22 @@ Then, select the previous release version as the **Previous tag** to use the dif **Generate release notes**. **Set as the latest release** and **Publish**. + +## Misc + +### Set up GPG key and Upload to Apache Iceberg KEYS file + +To set up GPG key locally, see the instructions [here](http://www.apache.org/dev/openpgp.html#key-gen-generate-key). + +To install gpg on a M1 based Mac, a couple of additional steps are required: . + +Then, published GPG key to the [Apache Iceberg KEYS file](https://downloads.apache.org/iceberg/KEYS): + +```bash +svn co https://dist.apache.org/repos/dist/release/iceberg icebergsvn +cd icebergsvn +echo "" >> KEYS # append a newline +gpg --list-sigs >> KEYS # append signatures +gpg --armor --export >> KEYS # append public key block +svn commit -m "add key for " +``` From 35d83013f46e1002ed921425da2276480d00597d Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Thu, 21 Nov 2024 11:58:47 -0800 Subject: [PATCH 04/11] verify artifacts --- mkdocs/docs/how-to-release.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index 2abcc2af5..cfbb01fc3 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -138,6 +138,8 @@ svn add $SVN_TMP_DIR_VERSIONED svn ci -m "PyIceberg ${VERSION}" ${SVN_TMP_DIR_VERSIONED} ``` +Verify the artifact is uploaded to [https://dist.apache.org/repos/dist/dev/iceberg](https://dist.apache.org/repos/dist/dev/iceberg/). + #### Upload to PyPi ##### Create Artifacts for PyPi @@ -161,6 +163,8 @@ Upload release candidate to PyPi. This **won't** bump the version for everyone t twine upload release-${VERSION}/* ``` +Verify the artifact is uploaded to [PyPi](https://pypi.org/project/pyiceberg/#history). + ## Vote ### Generate Vote Email @@ -256,6 +260,8 @@ export SVN_RELEASE_DIR_VERSIONED="https://dist.apache.org/repos/dist/release/ice svn mv ${SVN_DEV_DIR_VERSIONED} ${SVN_RELEASE_DIR_VERSIONED} -m "PyIceberg: Add release ${VERSION_WITHOUT_RC}" ``` +Verify the artifact is uploaded to [https://dist.apache.org/repos/dist/release/iceberg](https://dist.apache.org/repos/dist/release/iceberg/). + ### Upload the accepted release to PyPi The latest version can be pushed to PyPi. Check out the Apache SVN and make sure to publish the right version with `twine`: @@ -266,6 +272,8 @@ cd /tmp/iceberg-dist-release/pyiceberg-${VERSION_WITHOUT_RC} twine upload pyiceberg-*.whl pyiceberg-*.tar.gz ``` +Verify the artifact is uploaded to [PyPi](https://pypi.org/project/pyiceberg/#history). + ## Post Release ### Send out Release Announcement Email From ab7a6a02939888a32468b4d900a43dfcc5643c2a Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Thu, 21 Nov 2024 12:00:51 -0800 Subject: [PATCH 05/11] add twine not --- mkdocs/docs/how-to-release.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index cfbb01fc3..3d33bf5f1 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -159,6 +159,13 @@ Download the zip file from the Github Action run and unzip locally. Upload release candidate to PyPi. This **won't** bump the version for everyone that hasn't pinned their version, since it is set to an RC [pre-release and those are ignored](https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#pre-release-versioning). + + +!!! note + `twine` might require an PyPi API token. + + + ```bash twine upload release-${VERSION}/* ``` @@ -266,6 +273,13 @@ Verify the artifact is uploaded to [https://dist.apache.org/repos/dist/release/i The latest version can be pushed to PyPi. Check out the Apache SVN and make sure to publish the right version with `twine`: + + +!!! note + `twine` might require an PyPi API token. + + + ```bash svn checkout https://dist.apache.org/repos/dist/release/iceberg /tmp/iceberg-dist-release/ cd /tmp/iceberg-dist-release/pyiceberg-${VERSION_WITHOUT_RC} From d59b950ec290fa8c68fe2391e2f6623024e30b20 Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Thu, 21 Nov 2024 12:08:25 -0800 Subject: [PATCH 06/11] grammar --- mkdocs/docs/how-to-release.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index 3d33bf5f1..6cd474814 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -19,23 +19,23 @@ # How to Release -This guide documents the process for releasing PyIceberg following the [Apache Release Process](https://infra.apache.org/release-publishing.html). The process involves: +This guide outlines the process for releasing PyIceberg in accordance with the [Apache Release Process](https://infra.apache.org/release-publishing.html). The steps include: 1. Preparing for a release -2. Publishing a release candidate (RC) -3. Community voting and validation -4. Publishing the final release if the vote passes -5. Post release +2. Publishing a Release Candidate (RC) +3. Community Voting and Validation +4. Publishing the Final Release (if the vote passes) +5. Post-Release Step ## Requirements -* GPG key registered and published in [Apache Iceberg KEYS file](https://downloads.apache.org/iceberg/KEYS). See instructions at [Set up GPG key and Upload to Apache Iceberg KEYS file](#set-up-gpg-key-and-upload-to-apache-iceberg-keys-file). -* SVN - * Access to Apache SVN for uploading artifacts to [Apache Iceberg dev dist](https://dist.apache.org/repos/dist/dev/iceberg/) (requires Apache Commmitter) - * Access to Apache SVN for uploading artifacts to [Apache Iceberg release dist](https://dist.apache.org/repos/dist/release/iceberg/) (requires Apache PMC) -* PyPI - * `twine` installed for uploading to PyPi - * PyPI account with access to publish to the [pyiceberg project](https://pypi.org/project/pyiceberg/) +* A GPG key must be registered and published in the [Apache Iceberg KEYS file](https://downloads.apache.org/iceberg/KEYS). Follow [the instructions for setting up a GPG key and uploading it to the KEYS file](#set-up-gpg-key-and-upload-to-apache-iceberg-keys-file). +* SVN Access + * Permission to upload artifacts to the [Apache development distribution](https://dist.apache.org/repos/dist/dev/iceberg/) (requires Apache Commmitter access). + * Permission to upload artifacts to the [Apache release distribution](https://dist.apache.org/repos/dist/release/iceberg/) (requires Apache PMC access). +* PyPI Access + * The `twine` package must be installed for uploading releases to PyPi. + * A PyPI account with publishing permissions for the [pyiceberg project](https://pypi.org/project/pyiceberg/). ## Preparing for a Release From 91cad4d52b96819afaf7c9b7af815d65dfd7c942 Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Thu, 21 Nov 2024 12:20:36 -0800 Subject: [PATCH 07/11] edits --- mkdocs/docs/how-to-release.md | 36 ++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index 6cd474814..3d4d07366 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -68,18 +68,18 @@ deprecation_message( Update the version in `pyproject.toml` and `pyiceberg/__init__.py` to match the release version. See [#1276](https://github.com/apache/iceberg-python/pull/1276). -## Publishing a release candidate (RC) +## Publishing a Release Candidate (RC) ### Create Tag -Make sure you are on the correct branch: +Ensure you are on the correct branch: -* For a Major/Minor release, use the `main` branch -* For a Patch release, use the branch corresponding to the pathch version, i.e. `pyiceberg-0.6.x`. +* For a major/minor release, use the `main` branch +* For a patch release, use the branch corresponding to the patch version, i.e. `pyiceberg-0.6.x`. -Then, create a signed tag: +Create a signed tag: -Change `VERSION` and `RC` to the appropriate values. +Replace `VERSION` and `RC` with the appropriate values for the release. ```bash export RC=rc1 @@ -92,7 +92,7 @@ git tag -s ${GIT_TAG} -m "PyIceberg ${VERSION}" git push git@github.com:apache/iceberg-python.git ${GIT_TAG} ``` -### Publish RC +### Publish Release Candidate (RC) #### Upload to Apache Dev SVN @@ -100,18 +100,24 @@ git push git@github.com:apache/iceberg-python.git ${GIT_TAG} Run the [`Python release` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-release.yml). -* Use the newly created tag. -* Set the `version` to `main`, since we cannot modify the source. +* Tag: Use the newly created tag. +* Version: Set the `version` to `main`, as the source cannot be modified. ![Github Actions Run Workflow for SVN Upload](assets/images/ghactions-run-workflow-svn-upload.png) -This will create the source distribution (`sdist`) and the binary distributions (`wheels`) for each architectures, using [`cibuildwheel`](https://github.com/pypa/cibuildwheel) +This action will generate: -##### Download Artifacts, Sign and Generate Checksums +* Source distribution (`sdist`) +* Binary distributions (`wheels`) for each architectures. These are created using [`cibuildwheel`](https://github.com/pypa/cibuildwheel) -Before committing the files to the Apache SVN artifact distribution SVN hashes need to be generated, and those need to be signed with gpg to make sure that they are authentic. +##### Download Artifacts, Sign, and Generate Checksums -Download the zip file from the Github Action run, unzip, and sign the files: +Download the ZIP file containing the artifacts from the GitHub Actions run and unzip it. + +Navigate to the release directory. Sign the files and generate checksums: + +* `.asc` files: GPG-signed versions of each artifact to ensure authenticity. +* `.sha512` files: SHA-512 checksums for verifying file integrity. ```bash cd release-main/ @@ -146,8 +152,8 @@ Verify the artifact is uploaded to [https://dist.apache.org/repos/dist/dev/icebe Run the [`Python release` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-release.yml). -* Use the newly created tag. -* Set the `version` to release candidate, e.g. `0.7.0rc1`. +* Tag: Use the newly created tag. +* Version: Set the `version` to release candidate, e.g. `0.7.0rc1`. ![Github Actions Run Workflow for PyPi Upload](assets/images/ghactions-run-workflow-pypi-upload.png) From 2e16cddec42ccdad2983c592ad3c399ca9aed79f Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Thu, 21 Nov 2024 14:30:50 -0800 Subject: [PATCH 08/11] remove old artifacts --- mkdocs/docs/how-to-release.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index 3d4d07366..4b2ae6650 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -146,6 +146,14 @@ svn ci -m "PyIceberg ${VERSION}" ${SVN_TMP_DIR_VERSIONED} Verify the artifact is uploaded to [https://dist.apache.org/repos/dist/dev/iceberg](https://dist.apache.org/repos/dist/dev/iceberg/). +##### Remove Old Artifacts From Apache Dev SVN + +Clean up old RC artifacts: + +```bash +svn delete https://dist.apache.org/repos/dist/dev/iceberg/pyiceberg- -m "Remove old RC artifacts" +``` + #### Upload to PyPi ##### Create Artifacts for PyPi @@ -275,6 +283,14 @@ svn mv ${SVN_DEV_DIR_VERSIONED} ${SVN_RELEASE_DIR_VERSIONED} -m "PyIceberg: Add Verify the artifact is uploaded to [https://dist.apache.org/repos/dist/release/iceberg](https://dist.apache.org/repos/dist/release/iceberg/). +### Remove Old Artifacts From Apache Release SVN + +We only want to host the latest release. Clean up old release artifacts: + +```bash +svn delete https://dist.apache.org/repos/dist/release/iceberg/pyiceberg- -m "Remove old release artifacts" +``` + ### Upload the accepted release to PyPi The latest version can be pushed to PyPi. Check out the Apache SVN and make sure to publish the right version with `twine`: From 3e1016defcc0c0b02c182d31cbad16193da402c5 Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Fri, 22 Nov 2024 08:40:28 -0800 Subject: [PATCH 09/11] update doc workflow action --- mkdocs/docs/how-to-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index 4b2ae6650..c69b6d47b 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -334,7 +334,7 @@ Thanks to everyone for contributing! ### Release the docs -Run the [`Python Docs` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-ci-docs.yml). +Run the [`Python Docs` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-release-docs.yml). ### Update the Github template From 88aeeca73474cdf9bbc0fbcd338df837caaeaca2 Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Fri, 22 Nov 2024 08:41:23 -0800 Subject: [PATCH 10/11] and name --- mkdocs/docs/how-to-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index c69b6d47b..30ecb672f 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -334,7 +334,7 @@ Thanks to everyone for contributing! ### Release the docs -Run the [`Python Docs` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-release-docs.yml). +Run the [`Release Docs` Github Action](https://github.com/apache/iceberg-python/actions/workflows/python-release-docs.yml). ### Update the Github template From 647ab0e6cb462af85f782b21cafe56a0d6e38813 Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Fri, 22 Nov 2024 10:43:48 -0800 Subject: [PATCH 11/11] add docs on patch vs major/minor release --- mkdocs/docs/how-to-release.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/mkdocs/docs/how-to-release.md b/mkdocs/docs/how-to-release.md index 30ecb672f..b461e0074 100644 --- a/mkdocs/docs/how-to-release.md +++ b/mkdocs/docs/how-to-release.md @@ -70,6 +70,30 @@ Update the version in `pyproject.toml` and `pyiceberg/__init__.py` to match the ## Publishing a Release Candidate (RC) +### Release Types + +#### Major/Minor Release + +* Use the `main` branch for the release. +* Includes new features, enhancements, and any necessary backward-compatible changes. +* Examples: `0.8.0`, `0.9.0`, `1.0.0`. + +#### Patch Release + +* Use the branch corresponding to the patch version, such as `pyiceberg-0.8.x`. +* Focuses on critical bug fixes or security patches that maintain backward compatibility. +* Examples: `0.8.1`, `0.8.2`. + +To create a patch branch from the latest release tag: + +```bash +# Check out the base branch for the patch version +git checkout pyiceberg-0.8.x + +# Create a new branch for the upcoming patch release +git checkout -b pyiceberg-0.8.1 +``` + ### Create Tag Ensure you are on the correct branch: