-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add building wheels using cibuildwheel #1155
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 9761a16.
PEP 621 does not allow `project.name` to be dynamic, and requires any build backend to fail if one declares it as such. Since we only need to change the name of the package when creating a nightly release, this adds a `change_name.py` script, which only runs in the CI, that can change the `project.name` entry in-place. Note that, since `tomllib` has only become a part of the standard Python library in version 3.11, we use the `tomli` (for reading) and `tomli-w` (for writing) packages for parsing the `pyproject.toml` file.
On MacOS there seems to be an issue with `scikit-build-core` when `build-dir` is unset, so now we explicitly set it when building the wheel.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
1uc
approved these changes
Mar 6, 2024
Logfiles from GitLab pipeline #198641 (:white_check_mark:) have been uploaded here! Status and direct links: |
ohm314
pushed a commit
that referenced
this pull request
May 21, 2024
* use cibuildwheel for creating redistributable wheels * simplify CI pipeline * update packaging docs --------- Co-authored-by: Nicolas Cornu <[email protected]> Co-authored-by: Luc Grosheintz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a follow-up of #1147 to greatly simplify and speed-up the whole build CI pipeline.
Overview of changes
pip install cibuildwheel && cibuildwheel
. The resulting wheel will be available in thewheelhouse
subdirectory. Some notes about this:CIBW_BUILD
to something likecp311*
(for Python 3.11) to only build a wheel for a specific Python version--platform linux
in the above, Docker is needed; if you have podman instead, setCIBW_CONTAINER_ENGINE=podman
)manylinux
Docker image used for building the container does not come with the Python shared library); if you want to test the built wheel locally on Linux, runbash packaging/test_wheel.bash $(command -v python3) WHEEL
, whereWHEEL
is the wheel you just builtpackaging/build_wheels.bash
cibuildwheel
everywhere. This comes with a big reduction in the CI build time (see for instance old vs. new, 16 minutes per Python version vs. 7 minutes). By default we build wheels for all Python versions, which is probably not optimal, and I don't have strong opinions on this, so whatever the popular vote decides is fine by me (build all/build specific version/build oldest/build newest/build oldest and newest)What works
What needs work
build time on Linux per Python version can take a while (5 minutes) on x86_64, and, due to the need for QEMU, is even longer on aarch64. If we only want x86_64 builds though, we should be fine (EDIT: added CircleCI to build aarch64 Linux since it has a native runner)EDIT: it now builds wheels much faster thanmaster
libpython.so
library is not available (as described here and in Fails to detect Python oncibuildwheel
scikit-build/scikit-build-core#173) so some workarounds may be needed for testing thenmodl
binary in a portable way. The Python module tests work like a charm though!