Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix pyarrow depending on incorrect version of libstdcxx-ng #359

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

cjdoris
Copy link

@cjdoris cjdoris commented Nov 23, 2022

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

Fixes conda-forge/arrow-cpp-feedstock#895. Partially fixes JuliaPy/PythonCall.jl#247.

In the above issue, I showed that some builds of pyarrow have an incorrect dependency on libstdcxx-ng (too low) meaning that it is possible to install versions of these such that pyarrow raises an error when being imported.

You can reproduce the problem (on linux-64) as follows:

$ micromamba install 'pyarrow=8.0.0=*_1_cpu' 'libstdcxx-ng=11' 'python=3.10' -y -c conda-forge
...
$ python -c 'import pyarrow'
...
ImportError: .../libstdc++.so.6: version `GLIBCXX_3.4.30` not found ...

The error message means that pyarrow was trying to load a symbol that only exists in a newer version of libstdc++ (v12 in this case).

By varying the above incantation, I discovered that there are only three version/build combinations with this issue, namely 8.0.0/1, 7.0.0/6 and 6.0.1/18. It affects all versions of Python (3.7, 3.8, 3.9, 3.10) the same.

This PR fixes the corresponding dependencies to libstdcxx-ng >=12.

This only affects Linux, but I have only looked at linux-64 and not other architectures. I have also only looked at the CPU builds of pyarrow.

I assume what caused this error in the first place is some short-term bug in the C++ toolchain you use to build pyarrow which was outputting incorrect dependencies. Just a guess. Perhaps it affects other packages too, I don't know.

The diff is:

linux-64::pyarrow-6.0.1-py310h225c066_18_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-6.0.1-py37h6cfa6f4_18_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-6.0.1-py38h9f6a473_18_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-6.0.1-py39h42d110c_18_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-7.0.0-py310h225c066_6_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-7.0.0-py37h6cfa6f4_6_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-7.0.0-py38h9f6a473_6_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-7.0.0-py39h42d110c_6_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-8.0.0-py310h225c066_1_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-8.0.0-py37h6cfa6f4_1_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-8.0.0-py38h9f6a473_1_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",
linux-64::pyarrow-8.0.0-py39h42d110c_1_cpu.tar.bz2
-    "libstdcxx-ng >=10.3.0",
+    "libstdcxx-ng >=12",

@cjdoris cjdoris requested a review from a team as a code owner November 23, 2022 17:03
@conda-forge-linter
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@jakirkham
Copy link
Member

Please see issue ( conda-forge/arrow-cpp-feedstock#895 ). Though would like to finish the discussion there before making changes here

@cjdoris
Copy link
Author

cjdoris commented Nov 23, 2022

Another solution would be to simply label these builds as broken. None of them are the latest build for the corresponding version.

@h-vetinari
Copy link
Member

@cjdoris
Is this still an issue?

@jakirkham
I think the context here should be clear enough - any reason we shouldn't be doing this?

@cjdoris
Copy link
Author

cjdoris commented Dec 4, 2022

Yep, still a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Yank older broken builds? unable to install package datasets
4 participants