Python bindings do not work on Windows when installed from source since Python 3.8

[traversaro]

In particular, for example import yarp is failing with:

>>> import yarp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\src\light_ws\rs\build\install\Lib\site-packages\yarp.py", line 15, in <module>
    import _yarp
ImportError: DLL load failed while importing _yarp: The specified module could not be found.
>>>

The reason is described in gazebosim/sdformat#1165 (comment), basically since Python 3.8:

New in version 3.8: Previous versions of CPython would resolve DLLs using the default behavior for the current process. This led to inconsistencies, such as only sometimes searching PATH or the current working directory, and OS functions such as AddDllDirectory having no effect.

In 3.8, the two primary ways DLLs are loaded now explicitly override the process-wide behavior to ensure consistency. See the porting notes for information on updating libraries.

[traversaro]

As suggested in gazebosim/sdformat#1165 (comment), running:

set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1

before using the bindings everything works as expected:

(robsublight) C:\src\light_ws\rs\build>set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1

(robsublight) C:\src\light_ws\rs\build>python
Python 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:30:19) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import yarp
>>> vec = yarp.Vector()
>>> vec.resize(10)
>>> vec.toString()
' 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000'
>>>

[traversaro]

According to @giotherobot tests, set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1 is not working, so we are using:

import os
os.add_dll_directory(os.path.join(os.environ['ROBOTOLOGY_SUPERBUILD_INSTALL_PREFIX'], bin))

and we also documented it in #1701 as a mitigation for this problem.

[traversaro]

Further mitigation is provided by robotology/idyntree#1209 . Once that is merged, import idyntree will work out of the box, and also any subsequent call to import casadi, import bipedal_locomotion_framework or import yarp, as all the .dll of the superbuild are installed in the same folder. So technically speaking this issue will be solved once all libraries have something like robotology/idyntree#1209, but in practice an alternative workaround to os.add_dll_directory(os.path.join(os.environ['ROBOTOLOGY_SUPERBUILD_INSTALL_PREFIX'], 'bin')) is to just add import idyntree.

[traversaro]

So, this was fixed in:

• idyntree : Fix loading of Python bindings on Windows when installed in arbitrary directory idyntree#1209
• human-dynamics-estimation : Fix Python bindings installation and support building bindings as standalone projects human-dynamics-estimation#398

[traversaro]

If we also fix it in yarp and blf, I think we can close the issue. Fixing the issue in YARP will remove the need of workaround in @giotherobot code, and the remaining python libraries (like manifpy) are not mantained from us, so I think it is fair to consider the problem as an upstream problem, unless we have specific problems.

[traversaro]

New fixes:

• Fix loading of Python bindings on Windows when installed in arbitrary directory ami-iit/bipedal-locomotion-framework#905
• Fix loading of Python bindings on Windows when installed in arbitrary directory yarp#3146

Let's close as there is not a lot we can do here, and anyhow we mitigated this with the documentation.