Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add wheel tests to CI #1416

Merged
merged 18 commits into from
Jan 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ jobs:
- conda-python-tests
- docs-build
- wheel-build
- wheel-tests
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
checks:
Expand Down Expand Up @@ -53,3 +54,10 @@ jobs:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
matrix_filter: map(select(.ARCH == "amd64")) | max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]) | [.]
script: "ci/build_wheel.sh"
wheel-tests:
needs: wheel-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: pull-request
script: "ci/test_wheel.sh"
9 changes: 9 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,12 @@ jobs:
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
wheel-tests:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
script: ci/test_wheel.sh
2 changes: 1 addition & 1 deletion ci/build_wheel.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ rapids-generate-version > ./VERSION
python -m pip wheel . -w dist -v --no-deps --disable-pip-version-check
./ci/validate_wheel.sh dist

RAPIDS_PY_WHEEL_NAME="dask-cuda" rapids-upload-wheels-to-s3 dist
RAPIDS_PY_WHEEL_NAME="dask-cuda" RAPIDS_PY_WHEEL_PURE="1" rapids-upload-wheels-to-s3 dist
19 changes: 19 additions & 0 deletions ci/test_wheel.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/bash
# Copyright (c) 2025, NVIDIA CORPORATION.

set -eou pipefail

RAPIDS_PY_WHEEL_NAME="dask-cuda" RAPIDS_PY_WHEEL_PURE="1" rapids-download-wheels-from-s3 python ./dist

# Install cuda-suffixed dependencies b/c while `dask-cuda` has no cuda suffix, the test dependencies do
rapids-dependency-file-generator \
--output requirements \
--file-key "test_python" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};cuda_suffixed=true" \
| tee /tmp/requirements-test.txt

rapids-logger "Installing test dependencies"
# echo to expand wildcard
python -m pip install -v --prefer-binary -r /tmp/requirements-test.txt $(echo ./dist/dask_cuda*.whl)

python -m pytest ./dask_cuda/tests -k "not ucxx"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems ok to me... looks like the changes from #1406 doing the same thing in the conda Python test jobs hasn't been reverted yet.

12 changes: 2 additions & 10 deletions dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,6 @@ files:
table: project
includes:
- run_python
py_test:
output: pyproject
pyproject_dir: .
extras:
table: project.optional-dependencies
key: test
includes:
- test_python
py_docs:
output: pyproject
pyproject_dir: .
Expand Down Expand Up @@ -188,21 +180,21 @@ dependencies:
- numactl-devel-cos7-aarch64
- output_types: [requirements, pyproject]
matrices:
# kvikio should be added to the CUDA-version-specific matrices once there are wheels available
# ref: https://github.com/rapidsai/kvikio/pull/369
- matrix:
cuda: "12.*"
cuda_suffixed: "true"
packages:
- cudf-cu12==25.2.*,>=0.0.0a0
- dask-cudf-cu12==25.2.*,>=0.0.0a0
- kvikio-cu12==25.2.*,>=0.0.0a0
- ucx-py-cu12==0.42.*,>=0.0.0a0
- matrix:
cuda: "11.*"
cuda_suffixed: "true"
packages:
- cudf-cu11==25.2.*,>=0.0.0a0
- dask-cudf-cu11==25.2.*,>=0.0.0a0
- kvikio-cu11==25.2.*,>=0.0.0a0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh nice, thank you!

- ucx-py-cu11==0.42.*,>=0.0.0a0
- matrix:
packages:
Expand Down
8 changes: 0 additions & 8 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,14 +49,6 @@ docs = [
"sphinx-click>=2.7.1",
"sphinx-rtd-theme>=0.5.1",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit dependencies.yaml and run `rapids-dependency-file-generator`.
test = [
"cudf==25.2.*,>=0.0.0a0",
"dask-cudf==25.2.*,>=0.0.0a0",
"kvikio==25.2.*,>=0.0.0a0",
"pytest",
"pytest-cov",
"ucx-py==0.42.*,>=0.0.0a0",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit dependencies.yaml and run `rapids-dependency-file-generator`.

[project.urls]
Homepage = "https://github.com/rapidsai/dask-cuda"
Expand Down
Loading