Python issues on Ubuntu 24.04 - PyHSS requirements cannot be installed

[dchard]

root@testhss:/etc/pyhss# pip3 install -r requirements.txt
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.

If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.

If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.

See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
root@testhss:/etc/pyhss# pip3 install -r requirements.txt --break-system-packages
Collecting aiohttp==3.8.5 (from -r requirements.txt (line 1))
Using cached aiohttp-3.8.5.tar.gz (7.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting Flask==2.2.3 (from -r requirements.txt (line 2))
Using cached Flask-2.2.3-py3-none-any.whl.metadata (3.9 kB)
Collecting flask_restx==1.1.0 (from -r requirements.txt (line 3))
Using cached flask_restx-1.1.0-py2.py3-none-any.whl.metadata (9.0 kB)
Collecting Jinja2==3.1.2 (from -r requirements.txt (line 4))
Using cached Jinja2-3.1.2-py3-none-any.whl.metadata (3.5 kB)
Collecting mongo==0.2.0 (from -r requirements.txt (line 5))
Using cached mongo-0.2.0.tar.gz (4.6 kB)
Preparing metadata (setup.py) ... done
Collecting prometheus_client==0.16.0 (from -r requirements.txt (line 6))
Using cached prometheus_client-0.16.0-py3-none-any.whl.metadata (22 kB)
Collecting pycryptodome==3.17 (from -r requirements.txt (line 7))
Using cached pycryptodome-3.17-cp35-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.3 kB)
Collecting pymongo==4.3.3 (from -r requirements.txt (line 8))
Using cached pymongo-4.3.3.tar.gz (814 kB)
Preparing metadata (setup.py) ... done
Collecting pysctp==0.7.2 (from -r requirements.txt (line 9))
Using cached pysctp-0.7.2.tar.gz (32 kB)
Preparing metadata (setup.py) ... done
Collecting pysnmp==4.4.12 (from -r requirements.txt (line 10))
Using cached pysnmp-4.4.12-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting PyYAML==6.0 (from -r requirements.txt (line 11))
Using cached PyYAML-6.0.tar.gz (124 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [54 lines of output]
running egg_info
writing lib/PyYAML.egg-info/PKG-INFO
writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
writing top-level names to lib/PyYAML.egg-info/top_level.txt
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/vendor/pyproject_hooks/in_process/in_process.py", line 353, in
main()
File "/usr/lib/python3/dist-packages/pip/vendor/pyproject_hooks/in_process/in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pip/vendor/pyproject_hooks/in_process/in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
return self.get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 297, in get_build_requires
self.run_setup()
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 313, in run_setup
exec(code, locals())
File "", line 288, in
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/init.py", line 108, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/distutils/core.py", line 184, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/distutils/core.py", line 200, in run_commands
dist.run_commands()
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/distutils/dist.py", line 964, in run_commands
self.run_command(cmd)
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/distutils/dist.py", line 983, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/egg_info.py", line 310, in run
self.find_sources()
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/egg_info.py", line 318, in find_sources
mm.run()
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/egg_info.py", line 539, in run
self.add_defaults()
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/egg_info.py", line 577, in add_defaults
sdist.add_defaults(self)
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/sdist.py", line 102, in add_defaults
super().add_defaults()
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/_distutils/command/sdist.py", line 250, in add_defaults
self.add_defaults_ext()
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/_distutils/command/sdist.py", line 335, in add_defaults_ext
self.filelist.extend(build_ext.get_source_files())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 204, in get_source_files
File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/_distutils/cmd.py", line 107, in getattr
raise AttributeError(attr)
AttributeError: cython_sources
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[dchard]

Going back to 1.0.2 from master does resolve this issue.

[9600]

I'm getting the same error on Ubuntu 24.04 with release tag 1.0.2, when installing the dependencies inside a venv created with python3 -m venv venv.

[9600]

So I resolved the PyYAML build error on Ubuntu 24.04 / Python 3.12 by bumping the version in requirements.txt to PyYAML==6.0.1.

I then had a similar failure with aiohttp, which was resolved by bumping this to aiohttp==3.9.3. Appears that the relevant fix went into 3.9.0, but figured to go with the last 3.9.x release (it may work with current, which is 3.10.10, but I haven't tested).

Looks as though these issues stem from Python moving to Cython 3.0 and some modules needing to be updated to build with this.