Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImportError: libignition-gazebo4.so.4: cannot open shared object file: No such file or directory #288

Closed
yijionglin opened this issue Jan 23, 2021 · 9 comments

Comments

@yijionglin
Copy link

yijionglin commented Jan 23, 2021

Hi,

After installing this pkg with pip install gym-ignition, I ran the example.py from here, and got this issue:

Traceback (most recent call last): File "example.py", line 3, in <module> from scenario import gazebo as scenario_gazebo File "/home/th20988/py38ign/lib/python3.8/site-packages/scenario/__init__.py", line 93, in <module> from .bindings import core File "/home/th20988/py38ign/lib/python3.8/site-packages/scenario/bindings/core.py", line 12, in <module> import scenario.bindings.gazebo File "/home/th20988/py38ign/lib/python3.8/site-packages/scenario/bindings/gazebo.py", line 13, in <module> from . import _gazebo ImportError: libignition-gazebo4.so.4: cannot open shared object file: No such file or directory

Before meeting this error, I also came across this one :

Traceback (most recent call last): File "gymign_example.py", line 3, in <module> from scenario import gazebo as scenario_gazebo File "/home/th20988/py38ign/lib/python3.8/site-packages/scenario/__init__.py", line 93, in <module> from .bindings import core File "/home/th20988/py38ign/lib/python3.8/site-packages/scenario/bindings/core.py", line 12, in <module> import scenario.bindings.gazebo File "/home/th20988/py38ign/lib/python3.8/site-packages/scenario/bindings/gazebo.py", line 13, in <module> from . import _gazebo ImportError: libpython3.8.so.1.0: cannot open shared object file: No such file or directory

I solved this error by solutions provided here and here. I'm not sure if these solutions result in the first error.

I'm using Ubuntu 18 and just have installed python 3.8 according to this webpage, then python 3.8 was used as the interpreter to create a virtual env with this command virtualenv -p python3.8 py38ign

Any help and ideas would be highly appreciated!!

Cheers,

@yijionglin
Copy link
Author

yijionglin commented Jan 23, 2021

It seems the scenario lib is missing, or the complier cannot link to this lib.

I have checked that the scenario are in:

-$ find . -name scenario

  • ./py38ign/lib/python3.8/site-packages/scenario
  • ./py38ign/lib/python3.8/site-packages/scenario/include/scenario
  • ./py38ign/lib/python3.8/site-packages/scenario/lib/scenario
  • ./py38ign/lib/python3.8/site-packages/gym_ignition/scenario

And the python paths are:

print ('\n'.join(sys.path))
/opt/ros/melodic/lib/python2.7/dist-packages
/home/th20988/python3_ws/devel/lib/python3/dist-packages
/usr/local/lib/python38.zip
/usr/local/lib/python3.8
/usr/local/lib/python3.8/lib-dynload
/home/th20988/py38/lib/python3.8/site-packages

@diegoferigo
Copy link
Collaborator

@yijionglin How did you install Ignition Gazebo? Here you can find the requirements of both Stable and Nightly, the latest release requires Dome.

Note that Ubuntu Bionic is not officially supported. However, with minor extra steps wrt those listed in the installation guide, gym-ignition should be able to run also there.

@yijionglin
Copy link
Author

yijionglin commented Jan 24, 2021

@diegoferigo Hi, thanks for the quick reply!

I re-installed the Ignition Gazebo with the Dome version, and then installed pip install gym-ignition with the Stable one. Now another similar but different error showed up:
`

(py_test) th20988@IT080176:~$ python
Python 3.8.6 (default, Jan 24 2021, 11:16:39)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import scenario
Traceback (most recent call last):
File "", line 1, in
File "/home/th20988/py_test/lib/python3.8/site-packages/scenario/init.py", line 93, in
from .bindings import core
File "/home/th20988/py_test/lib/python3.8/site-packages/scenario/bindings/core.py", line 12, in
import scenario.bindings.gazebo
File "/home/th20988/py_test/lib/python3.8/site-packages/scenario/bindings/gazebo.py", line 13, in
from . import _gazebo
ImportError: libprotobuf.so.17: cannot open shared object file: No such file or directory

I tried pip install protobuf and install it successfully but still met this error.

I also checked the GCC version in the terminal with gcc -v and it showed:

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 8.4.0-1ubuntu1-18.04' --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-8 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 8.4.0 (Ubuntu 8.4.0-1ubuntu1~18.04)

Also, I checked clang version with clang -v

clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8.5
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8.5
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.5.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8
Candidate multilib: .;@m64
Selected multilib: .;@m64

I'm not sure what's the problem behind them, could you please give me some ideas? (One thing, which I'm not sure whether it would be the problem, is that when I imported the scenario in the Python shell with the terminal, the GCC version is 7.5.0, as shown in the very beginning.)

Cheers,

@yijionglin
Copy link
Author

yijionglin commented Jan 24, 2021

I have installed sudo apt-get install libprotobuf-dev, and used find . -name *libprotobuf* to find the library, and get these:

./usr/share/doc/libprotobuf-lite10
./usr/share/doc/libprotobuf-dev
./usr/share/doc/libprotobuf10
./usr/lib/x86_64-linux-gnu/libprotobuf.so
./usr/lib/x86_64-linux-gnu/libprotobuf.a
./usr/lib/x86_64-linux-gnu/libprotobuf-lite.a
./usr/lib/x86_64-linux-gnu/libprotobuf-lite.so
./usr/lib/x86_64-linux-gnu/libprotobuf.so.10.0.0
./usr/lib/x86_64-linux-gnu/libprotobuf.so.10
./usr/lib/x86_64-linux-gnu/libprotobuf-lite.so.10
./usr/lib/x86_64-linux-gnu/libprotobuf-lite.so.10.0.0

It seems there is no libprotobuf.so.17.

@diegoferigo
Copy link
Collaborator

diegoferigo commented Jan 24, 2021

As I said, Ubuntu Bionic is not officially supported, and the problems you're encountering are among the reasons. I'll try to explain the situation.

gym-ignition is a hybrid Python / C++ package that requires finding a number of system dependencies (we will lift some of these requirements as soon as the tools for PEP600 will be finalized). You can install gym-ignition in two ways from PyPI: using the wheel (no compilation necessary) or the sdist (requires the C++ buildchain and all the build dependencies). The former is of course more user friendly, but it has to find some system libraries installed. Among them, it looks for libprotobuf17, and bionic does not seem to have it in its repos.

You can find all the libraries that are missing by locating the file _gazebo.so in your venv and calling ldd. E.g., in my system I have the following.

ldd /path/to/scenario/bindings/_gazebo.so
❯ ldd /venv/lib/python3.8/site-packages/scenario/bindings/_gazebo.so
        linux-vdso.so.1 (0x00007ffea34e9000)
        libpython3.8.so.1.0 => /lib/x86_64-linux-gnu/libpython3.8.so.1.0 (0x00007efe4af9d000)
        libECMSingleton.so => /venv/lib/python3.8/site-packages/scenario/bindings/../lib/libECMSingleton.so (0x00007efe4af74000)
        libignition-gazebo4.so.4 => /rl/sources/workspace/install/lib/libignition-gazebo4.so.4 (0x00007efe4ace0000)
        libignition-fuel_tools5.so.5 => /rl/sources/workspace/install/lib/libignition-fuel_tools5.so.5 (0x00007efe4ac40000)
        libignition-common3-events.so.3 => /rl/sources/workspace/install/lib/libignition-common3-events.so.3 (0x00007efe4ac17000)
        libignition-common3.so.3 => /rl/sources/workspace/install/lib/libignition-common3.so.3 (0x00007efe4abcd000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007efe4abaa000)
        libignition-transport9.so.9 => /rl/sources/workspace/install/lib/libignition-transport9.so.9 (0x00007efe4aae7000)
        libignition-msgs6.so.6 => /rl/sources/workspace/install/lib/libignition-msgs6.so.6 (0x00007efe4a6c3000)
        libsdformat10.so.10 => /rl/sources/workspace/install/lib/libsdformat10.so.10 (0x00007efe4a4c6000)
        libignition-math6.so.6 => /rl/sources/workspace/install/lib/libignition-math6.so.6 (0x00007efe4a462000)
        libprotobuf.so.17 => /lib/x86_64-linux-gnu/libprotobuf.so.17 (0x00007efe4a16a000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007efe49f89000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007efe49e3a000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007efe49e1d000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efe49c2b000)
        libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007efe49bfd000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007efe49be1000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007efe49bdb000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007efe49bd6000)
        libignition-plugin1-loader.so.1 => /rl/sources/workspace/install/lib/libignition-plugin1-loader.so.1 (0x00007efe49bbe000)
        libignition-plugin1.so.1 => /rl/sources/workspace/install/lib/libignition-plugin1.so.1 (0x00007efe49bb1000)
        libQt5Core.so.5 => /lib/x86_64-linux-gnu/libQt5Core.so.5 (0x00007efe49668000)
        libtinyxml2.so.6 => /lib/x86_64-linux-gnu/libtinyxml2.so.6 (0x00007efe49651000)
        /lib64/ld-linux-x86-64.so.2 (0x00007efe4b729000)
        libjsoncpp.so.1 => /lib/x86_64-linux-gnu/libjsoncpp.so.1 (0x00007efe4961b000)
        libyaml-0.so.2 => /lib/x86_64-linux-gnu/libyaml-0.so.2 (0x00007efe495f9000)
        libzip.so.5 => /lib/x86_64-linux-gnu/libzip.so.5 (0x00007efe495dc000)
        libcurl.so.4 => /lib/x86_64-linux-gnu/libcurl.so.4 (0x00007efe4954b000)
        libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007efe49542000)
        libzmq.so.5 => /lib/x86_64-linux-gnu/libzmq.so.5 (0x00007efe4949c000)
        liburdfdom_model.so.1.0 => /lib/x86_64-linux-gnu/liburdfdom_model.so.1.0 (0x00007efe49477000)
        libicui18n.so.66 => /lib/x86_64-linux-gnu/libicui18n.so.66 (0x00007efe49176000)
        libicuuc.so.66 => /lib/x86_64-linux-gnu/libicuuc.so.66 (0x00007efe48f90000)
        libpcre2-16.so.0 => /lib/x86_64-linux-gnu/libpcre2-16.so.0 (0x00007efe48f0d000)
        libdouble-conversion.so.3 => /lib/x86_64-linux-gnu/libdouble-conversion.so.3 (0x00007efe48ef7000)
        libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007efe48dce000)
        libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007efe48db9000)
        libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007efe48ae3000)
        libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007efe48aba000)
        libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007efe48a99000)
        librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x00007efe48a79000)
        libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x00007efe48a0b000)
        libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007efe489f6000)
        libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007efe48963000)
        libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007efe48916000)
        libldap_r-2.4.so.2 => /lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007efe488c0000)
        liblber-2.4.so.2 => /lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007efe488af000)
        libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007efe488a1000)
        libsodium.so.23 => /lib/x86_64-linux-gnu/libsodium.so.23 (0x00007efe48847000)
        libpgm-5.2.so.0 => /lib/x86_64-linux-gnu/libpgm-5.2.so.0 (0x00007efe487f8000)
        libnorm.so.1 => /lib/x86_64-linux-gnu/libnorm.so.1 (0x00007efe4869d000)
        libtinyxml.so.2.6.2 => /lib/x86_64-linux-gnu/libtinyxml.so.2.6.2 (0x00007efe48685000)
        libconsole_bridge.so.0.4 => /lib/x86_64-linux-gnu/libconsole_bridge.so.0.4 (0x00007efe4867f000)
        libicudata.so.66 => /lib/x86_64-linux-gnu/libicudata.so.66 (0x00007efe46bbe000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007efe46b49000)
        libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007efe469c7000)
        libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007efe467f1000)
        libhogweed.so.5 => /lib/x86_64-linux-gnu/libhogweed.so.5 (0x00007efe467b9000)
        libnettle.so.7 => /lib/x86_64-linux-gnu/libnettle.so.7 (0x00007efe4677f000)
        libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007efe466f9000)
        libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007efe4661c000)
        libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007efe465eb000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007efe465e4000)
        libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007efe465d5000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007efe465b7000)
        libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007efe4659a000)
        libgssapi.so.3 => /lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007efe46555000)
        libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007efe46532000)
        libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007efe463fc000)
        libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007efe463e4000)
        libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007efe463dd000)
        libheimntlm.so.0 => /lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007efe463d1000)
        libkrb5.so.26 => /lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007efe4633e000)
        libasn1.so.8 => /lib/x86_64-linux-gnu/libasn1.so.8 (0x00007efe46297000)
        libhcrypto.so.4 => /lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007efe4625d000)
        libroken.so.18 => /lib/x86_64-linux-gnu/libroken.so.18 (0x00007efe46244000)
        libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007efe46238000)
        libwind.so.0 => /lib/x86_64-linux-gnu/libwind.so.0 (0x00007efe4620e000)
        libheimbase.so.1 => /lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007efe461fc000)
        libhx509.so.5 => /lib/x86_64-linux-gnu/libhx509.so.5 (0x00007efe461ac000)
        libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007efe46083000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007efe46048000)

In your case, I suspect that the dynamic loader will not find one or more libraries, including libprotobuf17. This happens because the wheel is created in a focal system (in our Continuos Integration pipeline) and waiting PEP600 we do not vendor system libraries in the wheel.

A possible workaround to make gym-ignition run in Ubuntu Bionic, or any other non-supported distribution, is to install it from sdist. You can do it with the following command:

pip3 install gym-ignition --no-binary "gym-ignition"

This could not be straighforward. In fact, beyond the runtime dependencies like Ignition Dome, installing from sdist requires the build dependencies. To mention one, Ubuntu Bionic has SWIG 3 in its repo, therefore you should install version 4 from sources.


If you cannot upgrade your system, please consider running all your code in a Docker container. It would allow you to have a focal distribution without the need to upgrade your system. If not, I can try to provide guidance for Ubuntu Bionic, but it could take some while to converge to a working setup. We removed it from the supported systems because it required too many manual steps due to some outdated packages in its repos.

@diegoferigo
Copy link
Collaborator

Closing this issue since it was referring to an unsupported platform. Feel free to comment again in case you need more advice.

@Gaoyuan-Liu
Copy link

Hey @diegoferigo,
I met the same issue and thanks for your explanation I understand what's happening.
But that means the gym-ignition only support ubuntu OS later than 18.04 which is not very old actually, a lot of ros projects are build on ubuntu 18.04 even older version.
I think it would be pretty nice to make gym-ignition work easily on Ubuntu Bionic, or give a building instruction.

Thanks.

@diegoferigo
Copy link
Collaborator

Hey @diegoferigo,
I met the same issue and thanks for your explanation I understand what's happening.
But that means the gym-ignition only support ubuntu OS later than 18.04 which is not very old actually, a lot of ros projects are build on ubuntu 18.04 even older version.
I think it would be pretty nice to make gym-ignition work easily on Ubuntu Bionic, or give a building instruction.

Thanks.

Hi @Gaoyuan-Liu, thanks for the feedback. I understand your point. We supported Ubuntu Bionic for almost 2 years and the installation instructions were quite complicated. As soon as Focal has been released, we started migrating our systems and recommend all those who cannot to rely on docker. Nowadays, it's pretty simple 1) creating a container 2) disabling network isolation and 3) enabling X11 support. In this way, the host could be your preferred and relatively old distribution.

This being said, since you're not the first asking some steps for Ubuntu Bionic, you can try the following:

  • Install Python 3.8 (you can use this ppa)
  • Install SWIG >= 4.0 from sources
  • Install Ignition Dome as you prefer (either from ppa or colcon)
  • Build the gym-ignition wheel locally:
    cd /tmp
    git clone https://github.com/robotology/gym-ignition
    cd gym-ignition
    python3 setup.py bdist_wheel
    pip3 install dist/*.whl

I went open loop with the commands above, please let me know if something is wrong.

@diegoferigo
Copy link
Collaborator

For those reading this issue in the future, I also cross-link #293 (comment). In our organization we started exploring conda and conda-forge. Such setup would provide a unified, distro-agnostic, multi-platform installation scheme. We're still in exploration phase, but on the long run it could become the preferred way to install also gym-ignition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants