import pyg_lib fails due to OSError: dlopen ... (no such file) on macOS

[ryrl9703]

🐛 Describe the bug

I have installed the torch and torch_sparse and pyg_lib packages successfully, but when I import torch_sparse just report error as folling:

   OSError: dlopen(/Users/wakala/venvs/versions/3.12.2/torch/lib/python3.12/site-packages/libpyg.so, 0x0006): Library not loaded: /Library/Frameworks/Python.framework/Versions/3.12/Python
  Referenced from: <7BF3ABB0-5875-3736-8AB2-1784D3EFB687> /Users/wakala/venvs/versions/3.12.2/torch/lib/python3.12/site-packages/libpyg.so
  Reason: tried: '/Library/Frameworks/Python.framework/Versions/3.12/Python' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/Python.framework/Versions/3.12/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.12/Python' (no such file)

Environment

• pyg-lib version: 0.4.0+pt23
• PyTorch version: 2.3.0
• OS: MacOS 14.4.1 (23E224)
• Python version: 3.12.2
• CUDA/cuDNN version:
• How you installed PyTorch and pyg-lib (conda, pip, source): pip
• Any other relevant information:

[yifanfu01]

I think you should try a conda... it's difficult for Mac to set py environment with pip.

[rusty1s]

Do you mind to clarify how you installed PyTorch and pyg-lib? It runs fine for me on macOS M2.

[haru-256]

@rusty1s

Hello, I have the same problem in macOS M2.

• pyg-lib version: 0.4.0
• PyTorch version: 2.3.0
• OS: macOS 14.5
• Python version: 3.11.9
• CUDA/cuDNN version: None(cpu only)
• How you installed PyTorch and pyg-lib (conda, pip, source): pip (via pipenv)
• Any other relevant information:

First, I installed pyg_lib via pipenv.

pipenv run pip install torch~=2.3.0 torch-geometric~=2.5.2 --no-cache-dir
# https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html
pipenv run pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv \
  -f https://data.pyg.org/whl/torch-2.3.0+cpu.html \
  --no-cache-dir

Then, I ran the following command and got same error.

python -c "import pyg_lib; print(pyg_lib.__version__)

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/haru256/repo/github.com/haru-256/Hands-On-Graph-Neural-Networks-Using-Python/sample/.venv/lib/python3.11/site-packages/pyg_lib/__init__.py", line 39, in <module>
    load_library('libpyg')
  File "/Users/haru256/repo/github.com/haru-256/Hands-On-Graph-Neural-Networks-Using-Python/sample/.venv/lib/python3.11/site-packages/pyg_lib/__init__.py", line 36, in load_library
    torch.ops.load_library(spec.origin)
  File "/Users/haru256/repo/github.com/haru-256/Hands-On-Graph-Neural-Networks-Using-Python/sample/.venv/lib/python3.11/site-packages/torch/_ops.py", line 1032, in load_library
    ctypes.CDLL(path)
  File "/Users/haru256/.asdf/installs/python/3.11.9/lib/python3.11/ctypes/__init__.py", line 376, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: dlopen(/Users/haru256/repo/github.com/haru-256/Hands-On-Graph-Neural-Networks-Using-Python/sample/.venv/lib/python3.11/site-packages/libpyg.so, 0x0006): Library not loaded: /Library/Frameworks/Python.framework/Versions/3.11/Python
  Referenced from: <75FFC412-93B5-322B-8E6D-268DA3498CF4> /Users/haru256/repo/github.com/haru-256/Hands-On-Graph-Neural-Networks-Using-Python/sample/.venv/lib/python3.11/site-packages/libpyg.so
  Reason: tried: '/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file)

Thank you,

[yasmeenhany]

Hello, I also have the same issue on macOS Intel

pyg-lib version: 0.4.0+pt22
PyTorch version: 2.2.0
OS: macOS 14.4.1
Python version: 3.10.11
CUDA/cuDNN version: None(cpu only)
How you installed PyTorch and pyg-lib (conda, pip, source): python3 -m pip install pyg-lib -f https://data.pyg.org/whl/torch-2.2.0+cpu.html
Any other relevant information:
This is the error trace

OSError: dlopen(/Users/*/*/*/*/gnn_py3.10_env/lib/python3.10/site-packages/libpyg.so, 0x0006): Library not loaded: /usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.12/Python
  Referenced from: <0DEE3AA8-A399-3240-A9A9-F75D4088C2F8> /Users/*/*/*/*/gnn_py3.10_env/lib/python3.10/site-packages/libpyg.so
  Reason: tried: '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.12/Python' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.12/Python' (no such file), '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.12/Python' (no such file), '/System/Library/Frameworks/Python.framework/Versions/3.12/Python' (no such file, not in dyld cache)```

[rusty1s]

Somehow pyg-lib on macOS expects that you have a proper system Python installed. The easiest way to unblock for now is to simply install a system Python 3.11/3.12 version (whatever your error message says) from here.

[Linnore]

Somehow pyg-lib on macOS expects that you have a proper system Python installed. The easiest way to unblock for now is to simply install a system Python 3.11/3.12 version (whatever your error message says) from here.

A less simple solution to this problem is to soft link /Library/Frameworks/Python.framework/${version} to a Python bin (e.g. the Python of the conda environment where you install pyg-lib) which has the same version. In this way, no redundant system-level Python will be installed, though the soft link now is environmentally dependent.

[Martins6]

It seems that even installing from Python official solves it, but the error message changes somewhat @rusty1s.

Traceback (most recent call last):
  File "/Users/adrielmartins/Documents/gcaptain/testingsomething.py", line 1, in <module>
    import pyg_lib
  File "/Users/adrielmartins/Documents/gcaptain/.venv/lib/python3.11/site-packages/pyg_lib/__init__.py", line 39, in <module>
    load_library('libpyg')
  File "/Users/adrielmartins/Documents/gcaptain/.venv/lib/python3.11/site-packages/pyg_lib/__init__.py", line 36, in load_library
    torch.ops.load_library(spec.origin)
  File "/Users/adrielmartins/Documents/gcaptain/.venv/lib/python3.11/site-packages/torch/_ops.py", line 1295, in load_library
    ctypes.CDLL(path)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ctypes/__init__.py", line 376, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: dlopen(/Users/adrielmartins/Documents/gcaptain/.venv/lib/python3.11/site-packages/libpyg.so, 0x0006): Symbol not found: __ZN3c1010Dispatcher17runRecordFunctionERN2at14RecordFunctionENSt3__117reference_wrapperIKNS_14FunctionSchemaEEENS_11DispatchKeyE
  Referenced from: <75FFC412-93B5-322B-8E6D-268DA3498CF4> /Users/adrielmartins/Documents/gcaptain/.venv/lib/python3.11/site-packages/libpyg.so
  Expected in:     <9C467D80-5BAC-3156-B480-F0FF7E45DEF2> /Users/adrielmartins/Documents/gcaptain/.venv/lib/python3.11/site-packages/torch/lib/libtorch_cpu.dylib

It seems that is still expecting packages elsewhere. I'm running this on M3 Pro Mac, using Python3.11.8, torch 2.3.0, torch_geometric >=2.5.3.

I'm thinking of doing @Linnore proposal for now.