rmm v22.04.01 #11

regro-cf-autotick-bot · 2022-06-03T11:34:06Z

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

Dependencies have been updated if changed: see upstream
Tests have passed
Updated license if changed and license_file is packaged

Information about this PR:

Feel free to push to the bot's branch to update this PR if needed.
The bot will almost always only open one PR per version.
The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name	Upstream Version	Current Version
cub	22.1
thrust	22.1

Dependency Analysis

We couldn't run dependency analysis due to an internal error in the bot. :( Help is very welcome!

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/autotick-bot/actions/runs/2434112231, please use this URL for debugging.}

conda-forge-linter · 2022-06-03T11:34:12Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

leofang · 2022-06-03T14:14:46Z

@conda-forge-admin, please rerender

github-actions · 2022-06-03T14:17:30Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/rmm-feedstock/actions/runs/2435142685.

leofang · 2022-06-03T14:38:39Z

@conda-forge-admin, please rerender

github-actions · 2022-06-03T14:41:19Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/rmm-feedstock/actions/runs/2435275784.

leofang · 2022-06-03T15:06:02Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2022.06.03.11.00.48

leofang · 2022-06-03T15:25:21Z

Hi @vyasr I am trying to build rmm on conda-forge, but it seems scikit-build has trouble configuring Thrust, any suggestion?

vyasr · 2022-06-03T16:56:16Z

It looks like the thrust that's getting installed is missing a version.h file (or at least, it isn't being found). That's causing all the math errors because all the version variables are empty and therefore the math expressions are ill-formed. Thrust is also failing to find CUB. That combination to me seems like the download of thrust is incomplete for some reason.

I'm not exactly sure why those things are going wrong, but I think what we really need to fix (and should make the above issues go away) is that the Python build of rmm is by default not looking for librmm (which should contain the dependencies). We need to change this command to something like

    - {{ PYTHON }} setup.py build_ext -- -DFIND_RMM_CPP=ON
    - {{ PYTHON }} -m pip install . --no-deps -vv

vyasr · 2022-06-03T16:57:56Z

Also @leofang please ping me once we get this working, I didn't think about conda packages earlier but we'll need to make similar changes to all the RAPIDS conda feedstocks after the transition I assume.

recipe/meta.yaml

leofang · 2022-06-03T17:35:40Z

Thanks, Vyas, the error is different now.

...

2022-06-03T17:28:02.7293350Z -- Found Threads: TRUE  
2022-06-03T17:28:03.8739748Z -- Found Thrust: $PREFIX/lib/cmake/thrust/thrust-config.cmake (found version "1.15.0.0") 
2022-06-03T17:28:03.8761449Z -- Found RMM: $PREFIX/lib/cmake/rmm/rmm-config.cmake (found suitable version "22.04.01", minimum required is "22.04.01") 
2022-06-03T17:28:03.8794319Z CMake Error at /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/cub/cub-config-version.cmake:1 (include):
2022-06-03T17:28:03.8796129Z   include could not find requested file:
2022-06-03T17:28:03.8796682Z 
2022-06-03T17:28:03.8798594Z     /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/cub/../../../include/rapids/thrust/dependencies/cub/cub-config-version.cmake
2022-06-03T17:28:03.8800134Z Call Stack (most recent call first):
2022-06-03T17:28:03.8802009Z   /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/thrust/thrust-config.cmake:500 (find_package)
2022-06-03T17:28:03.8804489Z   /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/thrust/thrust-config.cmake:607 (_thrust_find_CUDA)
2022-06-03T17:28:03.8806988Z   /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/thrust/thrust-config.cmake:171 (_thrust_find_backend)
2022-06-03T17:28:03.8809722Z   /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/rmm/rmm-config.cmake:136 (thrust_create_target)
2022-06-03T17:28:03.8813822Z   CMakeLists.txt:48 (find_package)
2022-06-03T17:28:03.8814183Z 
2022-06-03T17:28:03.8814571Z 
2022-06-03T17:28:03.8816116Z CMake Error at /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/cub/cub-config.cmake:1 (include):
2022-06-03T17:28:03.8817807Z   include could not find requested file:
2022-06-03T17:28:03.8818120Z 
2022-06-03T17:28:03.8819707Z     /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/cub/../../../include/rapids/thrust/dependencies/cub/cub-config.cmake
2022-06-03T17:28:03.8820815Z Call Stack (most recent call first):
2022-06-03T17:28:03.8822502Z   /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/thrust/thrust-config.cmake:500 (find_package)
2022-06-03T17:28:03.8824773Z   /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/thrust/thrust-config.cmake:607 (_thrust_find_CUDA)
2022-06-03T17:28:03.8826859Z   /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/thrust/thrust-config.cmake:171 (_thrust_find_backend)
2022-06-03T17:28:03.8829278Z   /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/rmm/rmm-config.cmake:136 (thrust_create_target)
2022-06-03T17:28:03.8830362Z   CMakeLists.txt:48 (find_package)
2022-06-03T17:28:03.8830643Z 
2022-06-03T17:28:03.8830876Z 
2022-06-03T17:28:03.8832550Z CMake Error at /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/thrust/thrust-config.cmake:218 (message):
2022-06-03T17:28:03.8833850Z   The version of CUB found by CMake is not compatible with this release of
2022-06-03T17:28:03.8834451Z   Thrust.  CUB is now included in the CUDA Toolkit, so you no longer need to
2022-06-03T17:28:03.8835025Z   use your own checkout of CUB.  Pass IGNORE_CUB_VERSION_CHECK to
2022-06-03T17:28:03.8837119Z   thrust_create_target to ignore.  (CUB , Thrust 1.15.0.0).
2022-06-03T17:28:03.8837931Z Call Stack (most recent call first):
2022-06-03T17:28:03.8839873Z   /home/conda/feedstock_root/build_artifacts/rmm_1654276632302/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/cmake/rmm/rmm-config.cmake:136 (thrust_create_target)
2022-06-03T17:28:03.8841138Z   CMakeLists.txt:48 (find_package)
2022-06-03T17:28:03.8841683Z 
2022-06-03T17:28:03.8841932Z 
2022-06-03T17:28:03.8842610Z -- Configuring incomplete, errors occurred!
2022-06-03T17:28:03.8843547Z See also "$SRC_DIR/rmm/python/_skbuild/linux-x86_64-3.9/cmake-build/CMakeFiles/CMakeOutput.log".
2022-06-03T17:28:03.8844587Z See also "$SRC_DIR/rmm/python/_skbuild/linux-x86_64-3.9/cmake-build/CMakeFiles/CMakeError.log".
2022-06-03T17:28:03.8937039Z Traceback (most recent call last):

...

vyasr · 2022-06-03T18:36:22Z

Hmm maybe librmm bundles thrust and not cub? It looks like it's finding thrust now, but not cub. What happens if we include cub in the reqs list?

leofang · 2022-06-03T18:45:33Z

So the nightmare from conda-forge/librmm-feedstock#31 continues... 😞 (cc: @kkraus14 @robertmaynard for the mess we were dealing with last week...)

CUB is bundled, the log did mention rapids/thrust/dependencies/cub/cub-config-version.cmake. The problem is again if we don't constrain the Thrust/CUB version to 1.15 as we did previously, on conda-forge 1.16 would be fetched, but RMM hates it.

If we include only - cub 1.15*, it'd indeed constrain Thrust to 1.15 too because they set the version binding to each other through run_constrained. But then we'd be back to the same situation (6af7015 -> #11 (comment)).

leofang · 2022-06-03T18:47:20Z

Perhaps I should give it a shot anyway.

leofang · 2022-06-03T19:11:32Z

@vyasr Is there a way to instruct scikit-build to use librmm's bundled Thrust/CUB, as Robert suggested here?

vyasr · 2022-06-03T20:41:20Z

Hmm I'm not sure... is this something that we want to do in general, or is this specific to conda-forge? In other words, is this logic that should live in our CMake files, or is this something that needs to be done by providing e.g. a command line parameter or something?

The Python RMM package doesn't ever look for thrust directly. It executes find_package(rmm) to find librmm and the rmm-config.cmake file in librmm should tell it what thrust to use. So we need to hook into that. Since librmm uses CPM under the hood, I think that if inside the recipe you can get the path to thrust you could do something like add -DCPM_thrust_SOURCE=/path/to/thrust to the build_ext command. You'd get the path using some combination of $CONDA_HOME and knowing librmm's layout. That seems like a hack, though, and I don't know if there is a less hacked together solution here.

leofang · 2022-06-03T20:52:33Z

Indeed I am looking for a minimally intrusive workaround specific for conda-forge. -DCPM_<package_SOURCE=... was actually what I tried earlier (conda-forge/librmm-feedstock#31 (comment)) and it worked, but Keith didn't think it's a good idea. However, given we are facing the issue of neither librmm nor rmm can be built, I say we should give it one more try. Thanks Vays.

leofang · 2022-06-03T23:29:57Z

OK we have fixed the Thrust thing and hit a new error now:

-- Set runtime path of "$PREFIX/rmm/_lib/cuda_stream.cpython-38-x86_64-linux-gnu.so" to ""
Traceback (most recent call last):
  File "setup.py", line 8, in <module>
    setup(
  File "/home/conda/feedstock_root/build_artifacts/rmm_1654293709934/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.8/site-packages/skbuild/setuptools_wrap.py", line 676, in setup
    _classify_installed_files(
  File "/home/conda/feedstock_root/build_artifacts/rmm_1654293709934/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.8/site-packages/skbuild/setuptools_wrap.py", line 816, in _classify_installed_files
    raise SKBuildError(
skbuild.exceptions.SKBuildError: 
  CMake-installed files must be within the project root.
    Project Root  : /home/conda/feedstock_root/build_artifacts/rmm_1654293709934/work/rmm/python/_skbuild/linux-x86_64-3.8/cmake-install
    Violating File: /home/conda/feedstock_root/build_artifacts/rmm_1654293709934/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/rmm/_cuda/stream.cpython-38-x86_64-linux-gnu.so

vyasr · 2022-06-04T01:10:10Z

@leofang this is the error that I just fixed in rapidsai/rmm#1053. I guess we do need to backport it to 22.06?

leofang · 2022-06-04T01:14:40Z

Ah thanks for pointer Vyas. It's up to you to backport or not, on the conda-forge side I think the patch would be much simpler (just tweak an argument after build_ext IIUC) because we just need to fix rmm, not librmm.

leofang · 2022-06-04T04:10:15Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2022.06.03.22.56.00

github-actions · 2022-06-04T05:20:58Z

Hi! This is the friendly conda-forge automerge bot!

I considered the following status checks when analyzing this PR:

linter: passed
travis: passed
azure: passed

Thus the PR was passing and merged! Have a great day!

leofang · 2022-06-04T14:12:00Z

Thanks for help @vyasr, looks like it's still straightforward to build, once we know where the problems are for conda-forge 👍

updated v22.04.01

6ee7212

regro-cf-autotick-bot requested review from alhad-deshpande and leofang as code owners June 3, 2022 11:34

leofang added 2 commits June 3, 2022 10:14

Delete py_ver.diff

dc9520c

remove patch

672d727

add scikit-build

c638f86

try this

89cd639

MNT: Re-rendered with conda-build 3.21.9, conda-smithy 3.20.0, and co…

f48e58c

…nda-forge-pinning 2022.06.03.11.00.48

try this

6af7015

vyasr reviewed Jun 3, 2022

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

recipe/meta.yaml Outdated Show resolved Hide resolved

try to use RMM's bundled Thrust/CUB

2d1ccc2

try to add cub 1.15 but leave out thrust

761750f

leofang added 2 commits June 3, 2022 17:12

try to use the bundled thrust

2c5b2bf

fix typo

0fd1088

try to use -DThrust_ROOT instead

c3c750a

leofang added 2 commits June 3, 2022 19:44

Try to set cmake prefix

bb0ff2c

where do we hit the error?

afd1473

leofang added 4 commits June 3, 2022 21:16

this should work?

44d476e

clean up

243d212

script_env seems to hate long value

dafff70

add py310 migrator

9ee4959

MNT: Re-rendered with conda-build 3.21.9, conda-smithy 3.20.0, and co…

4d06291

…nda-forge-pinning 2022.06.03.22.56.00

leofang added the automerge Merge the PR when CI passes label Jun 4, 2022

github-actions bot merged commit da2c45c into conda-forge:main Jun 4, 2022

regro-cf-autotick-bot deleted the 22.04.01_had30eb branch June 4, 2022 05:32

This was referenced Jun 7, 2022

Export Thrust/CUB version used at build time conda-forge/librmm-feedstock#31

Closed

Thrust/CUB should be run-exported conda-forge/librmm-feedstock#30

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rmm v22.04.01 #11

rmm v22.04.01 #11

regro-cf-autotick-bot commented Jun 3, 2022

conda-forge-linter commented Jun 3, 2022

leofang commented Jun 3, 2022

github-actions bot commented Jun 3, 2022

leofang commented Jun 3, 2022

github-actions bot commented Jun 3, 2022

leofang commented Jun 3, 2022

leofang commented Jun 3, 2022

vyasr commented Jun 3, 2022

vyasr commented Jun 3, 2022

leofang commented Jun 3, 2022

vyasr commented Jun 3, 2022 •

edited

Loading

leofang commented Jun 3, 2022 •

edited

Loading

leofang commented Jun 3, 2022

leofang commented Jun 3, 2022 •

edited

Loading

vyasr commented Jun 3, 2022

leofang commented Jun 3, 2022

leofang commented Jun 3, 2022

vyasr commented Jun 4, 2022

leofang commented Jun 4, 2022

leofang commented Jun 4, 2022

github-actions bot commented Jun 4, 2022

leofang commented Jun 4, 2022

rmm v22.04.01 #11

rmm v22.04.01 #11

Conversation

regro-cf-autotick-bot commented Jun 3, 2022

Pending Dependency Version Updates

Dependency Analysis

conda-forge-linter commented Jun 3, 2022

leofang commented Jun 3, 2022

github-actions bot commented Jun 3, 2022

leofang commented Jun 3, 2022

github-actions bot commented Jun 3, 2022

leofang commented Jun 3, 2022

leofang commented Jun 3, 2022

vyasr commented Jun 3, 2022

vyasr commented Jun 3, 2022

leofang commented Jun 3, 2022

vyasr commented Jun 3, 2022 • edited Loading

leofang commented Jun 3, 2022 • edited Loading

leofang commented Jun 3, 2022

leofang commented Jun 3, 2022 • edited Loading

vyasr commented Jun 3, 2022

leofang commented Jun 3, 2022

leofang commented Jun 3, 2022

vyasr commented Jun 4, 2022

leofang commented Jun 4, 2022

leofang commented Jun 4, 2022

github-actions bot commented Jun 4, 2022

leofang commented Jun 4, 2022

vyasr commented Jun 3, 2022 •

edited

Loading

leofang commented Jun 3, 2022 •

edited

Loading

leofang commented Jun 3, 2022 •

edited

Loading