-
Notifications
You must be signed in to change notification settings - Fork 350
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
don't upload universal bdist to pypi #863
Conversation
In short, bdist (binary distributions) created from a pure python project are universal. They cannot be made to use a platform-specific tag (like linux, windows, macos, etc). This changes the release CI workflow to skip creating/uploading a bdist to PyPI. This library has a number of dependencies conditionally required per *machine* attributes (like CPU type). But pip (or other python package managers) will not be aware of such conditional dependencies because a bdist do not invoke a dynamic resolution of machine-specific dependencies. Universal bdists only resolve platform-specific dependencies (python version, system OS, etc) which isn't sufficient for this library. Instead, this paradigm is better left to a sdist (source distribution) which carry dependency details per machine-specific attributes in the setup.py file.
There are some binaries included for Raspberry Pi such as the libgpiod_pulsein in https://github.com/adafruit/Adafruit_Blinka/tree/main/src/adafruit_blinka/microcontroller/bcm283x/pulseio. Will switching to a non-binary distribution still include these files or will they not be included? |
You could use a MANIFEST.in file to explicitly include and/or exclude certain files when assembling the sdist. Typically I would use this to trim down the sdist by excluding the folders I noticed this repo has some submodules located in the test folder. If they're supposed to be part of the distribution, then the release CI should be checking out the repo with submodules initialized - uses: actions/checkout@v4
with:
submodules: true
# I usually let setuptools_scm have the
# full git history for creating release assets
fetch-depth: 0 |
You'll also notice that the binary files in question are included in the bdist that piwheels has built (for v8.44.4) from the sdist already present on PyPI (for v8.44.4). I can show more pictures, but I believe it'd be just repetitive. |
here's the important deps specs in the piwheels bdist (for v8.44.4) in the file
This is different from the METADATA found in the bdist on PyPI (for v8.44.4):
See how a universal bdist could be problematic for certain machines? I suspect that your expected behavior on RPi machines has been entirely dependent on the piwheels project without your knowledge. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok cool. That was my main concern, otherwise I'm good with this.
Ok, we're now getting complaints of additional requirements needed and very slow installs, so I'm going to revert this. |
In short, bdist (binary distributions) created from a pure python project are universal. They cannot be made to use a platform-specific tag (like linux, windows, macos, etc). This changes the release CI workflow to skip creating/uploading a bdist to PyPI.
This library has a number of dependencies conditionally required per machine attributes (like CPU type). But pip (or other python package managers) will not be aware of such conditional dependencies because a universal bdist do not invoke a dynamic resolution of machine-specific dependencies. Universal bdists only resolve platform-specific dependencies (python version, system OS, etc) which isn't sufficient for this library. Instead, this paradigm is better left to a sdist (source distribution) which carry dependency details per machine-specific attributes in the setup.py file.
Additional context
I discovered this library's bdist had flaws when research a solution for #858. The discussion there led to this PR (as requested). The only workaround for the flawed bdists from PyPI was to have users invoke pip's
--no-binary
arg or the env varPIP_NO_BINARY
.Fear not Raspberry Pi users! The piwheels project is hosting their own bdists that are built (from the sdist hosted on PyPI) on a network of RPi3 and RPi4 machines each using the 32-bit RPi OS; see status info at piwheels.org/project/adafruit-blinka. The RPi OS is already setup to use the piwheels index, so no special configuration is required.