Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider attempting CDLL with SONAMEs first #21

Open
SomeoneSerge opened this issue Jan 13, 2024 · 1 comment
Open

Consider attempting CDLL with SONAMEs first #21

SomeoneSerge opened this issue Jan 13, 2024 · 1 comment

Comments

@SomeoneSerge
Copy link

SomeoneSerge commented Jan 13, 2024

Hi! I noticed gpuctypes uses ctypes.util.find_library to dlopen libraries like libcuda.so by direct paths, instead of relying on the dynamic loader to choose the library by name:

gpuctypes/gpuctypes/cuda.py

Lines 146 to 148 in 77b94ad

_libraries = {}
_libraries['libcuda.so'] = ctypes.CDLL(ctypes.util.find_library('cuda'))
_libraries['libnvrtc.so'] = ctypes.CDLL(ctypes.util.find_library('nvrtc'))

Just loading libraries by name (CDLL("libcuda.so")) might often be already sufficient, because the dynamic loader is fairly configurable and the users may adjust its behaviour as is fit for their specific environment: it respects the LD_LIBRARY_PATH environment variable, it checks for the optional /etc/ld.so.conf, and certain entries in the executable's headers.

Additionally, note that the find_library routine doesn't have any clear semantics and it might return the current host's libraries, or it might randomly return a target platform's libraries accidentally found using e.g. gcc. Perhaps you could make find_library an optional "fallback mode"?

Also note that both CDLL("libcuda.so") and find_library("libcuda.so") require extra steps from the user to work, albeit hidden: on the commonly used distributions those would likely succeed and locate libcuda.so through /etc/ld.so.{cache,conf}, but these may not exist or not contain the records about all of the software available on the system. E.g. in Nixpkgs we're going to patch the referenced file(s) with our own paths. Generally, the solution is to explicitly declare the dependency in advance. Cf. e.g. https://discuss.python.org/t/pep-725-specifying-external-dependencies-in-pyproject-toml/31888 for more context

Thanks

@wozeparrot
Copy link

This should probably be done for the HIP as well since it's not guaranteed that HIP is installed in /opt/rocm, see #17.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants