Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation for "how to release" #1359

Merged
merged 11 commits into from
Nov 25, 2024

Conversation

kevinjqliu
Copy link
Contributor

@kevinjqliu kevinjqliu commented Nov 21, 2024

Closes #1306

Uploaded the documentation for "how to release" based on recent experience with releasing 0.8.0 (See PyIceberg 0.8.0 release notes)

Additional potential improvements

  • Use gh command to trigger GitHub Actions and download artifacts (similar to iceberg-go release script)
  • Consider triggering Python release github action on tag push. Is it possible to run the release action once instead of twice (main + tag)?

@kevinjqliu
Copy link
Contributor Author

Lets run this for 0.8.1 patch release


```bash
cd release-main/

for name in $(ls pyiceberg-*.whl pyiceberg-*.tar.gz)
do
gpg --yes --armor --local-user [email protected] --output "${name}.asc" --detach-sig "${name}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol, thanks! :D

@@ -128,21 +144,51 @@ svn add $SVN_TMP_DIR_VERSIONED
svn ci -m "PyIceberg ${VERSION}" ${SVN_TMP_DIR_VERSIONED}
```

### Upload to PyPi
Verify the artifact is uploaded to [https://dist.apache.org/repos/dist/dev/iceberg](https://dist.apache.org/repos/dist/dev/iceberg/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a section for deleting old RCs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

svn delete https://dist.apache.org/repos/dist/dev/iceberg/pyiceberg-<OLD_RC_VERSION> -m "Remove old RC artifacts"

I think this is the right command, will test during 0.8.1 release

export SVN_DEV_DIR_VERSIONED="https://dist.apache.org/repos/dist/dev/iceberg/pyiceberg-${VERSION}"
export SVN_RELEASE_DIR_VERSIONED="https://dist.apache.org/repos/dist/release/iceberg/pyiceberg-${VERSION_WITHOUT_RC}"

svn mv ${SVN_DEV_DIR_VERSIONED} ${SVN_RELEASE_DIR_VERSIONED} -m "PyIceberg: Add release ${VERSION_WITHOUT_RC}"
```

Verify the artifact is uploaded to [https://dist.apache.org/repos/dist/release/iceberg](https://dist.apache.org/repos/dist/release/iceberg/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a step here to only host the lastest release on the dist/release/. I recently cleaned it up, but it would be good to make this part of the release itself :)

@kevinjqliu kevinjqliu requested a review from Fokko November 22, 2024 16:29
@Fokko Fokko merged commit cc1ab2c into apache:main Nov 25, 2024
8 checks passed
@Fokko
Copy link
Contributor

Fokko commented Nov 25, 2024

Thanks @kevinjqliu for cleaning this up 👍

@kevinjqliu kevinjqliu deleted the kevinjqliu/release-process branch November 25, 2024 16:58
kevinjqliu added a commit to kevinjqliu/iceberg-python that referenced this pull request Nov 25, 2024
* initial update

* edits

* add gpg instructions

* verify artifacts

* add twine not

* grammar

* edits

* remove old artifacts

* update doc workflow action

* and name

* add docs on patch vs major/minor release
git checkout pyiceberg-0.8.x

# Create a new branch for the upcoming patch release
git checkout -b pyiceberg-0.8.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the added value of having this branch? Once the release is out, there will be a tag, and I don't think there is any value in keeping the branch unless we want to go for a 0.8.1.1 release

@kevinjqliu kevinjqliu added this to the PyIceberg 0.8.1 release milestone Nov 26, 2024
kevinjqliu added a commit that referenced this pull request Nov 27, 2024
* use the non-deprecated func (#1326)

* 0.8.0 post release steps (#1334)

* add

* fix mkdoc

* Drop upper bounds for fsspec and it's implementations (#1341)

* Drop upper bounds for fsspec and it's implementations

* Run poetry lock

* Ignore tables without `table_type` from Glue and Hive

* Ignore tables without table_type parameters while loading all iceberg table from Glue and Hive catalog (#1331)

* Use TABLE_TYPE

---------

Co-authored-by: Wenzhuo Zhao <[email protected]>

* Replace reference of `Table.identifier` with `Table.name` (#1346)

* fix Table.name

* replace Table.identifier with Table.name

* add warning filter

* Allow leading underscore in column name used in row filter (#1358)

* Update parser.py

Allow leading underscore in column name used in row filter.

* Update test_parser.py

* Update test_parser.py

* Update test_parser.py

* Remove Python 3.13 upper bound restriction (#1355)

* Remove Python 3.13 upper bound restriction

* Fix missing poetry.lock file

* Upgrading numpy on the poetry.lock file from v1.26.0 to v1.26.4

* Improve documentation for "how to release" (#1359)

* initial update

* edits

* add gpg instructions

* verify artifacts

* add twine not

* grammar

* edits

* remove old artifacts

* update doc workflow action

* and name

* add docs on patch vs major/minor release

* fix `KeyError` raised by `add_files` when parquet file doe not have column stats (#1354)

* fix KeyError, by switching del to pop

* added unit test

* update test

* fix python 3.9 compatibility, and refactor test

* update test

* bump to 0.8.1

* Add instruction for patch release (#1373)

* add instruction for patch release

* create branch from tag

* Write `null` when there is no parent-snapshot-id (#1383)

---------

Co-authored-by: Sumanth <[email protected]>
Co-authored-by: gitzwz <[email protected]>
Co-authored-by: Wenzhuo Zhao <[email protected]>
Co-authored-by: vincenzon <[email protected]>
Co-authored-by: Luca Bigon <[email protected]>
Co-authored-by: Binayak Dasgupta <[email protected]>
Co-authored-by: Fokko Driesprong <[email protected]>
kevinjqliu added a commit to kevinjqliu/iceberg-python that referenced this pull request Nov 27, 2024
* initial update

* edits

* add gpg instructions

* verify artifacts

* add twine not

* grammar

* edits

* remove old artifacts

* update doc workflow action

* and name

* add docs on patch vs major/minor release
kevinjqliu added a commit that referenced this pull request Nov 27, 2024
* initial update

* edits

* add gpg instructions

* verify artifacts

* add twine not

* grammar

* edits

* remove old artifacts

* update doc workflow action

* and name

* add docs on patch vs major/minor release
sungwy pushed a commit to sungwy/iceberg-python that referenced this pull request Dec 7, 2024
* initial update

* edits

* add gpg instructions

* verify artifacts

* add twine not

* grammar

* edits

* remove old artifacts

* update doc workflow action

* and name

* add docs on patch vs major/minor release
sungwy pushed a commit to sungwy/iceberg-python that referenced this pull request Dec 7, 2024
* initial update

* edits

* add gpg instructions

* verify artifacts

* add twine not

* grammar

* edits

* remove old artifacts

* update doc workflow action

* and name

* add docs on patch vs major/minor release
sungwy pushed a commit to sungwy/iceberg-python that referenced this pull request Dec 24, 2024
* initial update

* edits

* add gpg instructions

* verify artifacts

* add twine not

* grammar

* edits

* remove old artifacts

* update doc workflow action

* and name

* add docs on patch vs major/minor release
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve PyIceberg release process
2 participants