Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove dask-sql #684

Merged
merged 4 commits into from
Jun 27, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 2 additions & 8 deletions .github/workflows/build-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,6 @@ on:
RAPIDS_VER:
required: true
type: string
DASK_SQL_VER:
required: true
type: string
BASE_TAG:
required: true
type: string
Expand All @@ -52,7 +49,6 @@ jobs:
LINUX_VER: ["${{ inputs.LINUX_VER }}"]
PYTHON_VER: ["${{ inputs.PYTHON_VER }}"]
RAPIDS_VER: ["${{ inputs.RAPIDS_VER }}"]
DASK_SQL_VER: ["${{ inputs.DASK_SQL_VER }}"]
fail-fast: false
runs-on: "linux-${{ matrix.ARCH }}-cpu4"
steps:
Expand All @@ -70,8 +66,8 @@ jobs:
GIT_DESCRIBE_TAG="$(git describe --tags --abbrev=0)"
GIT_DESCRIBE_TAG="${GIT_DESCRIBE_TAG:1}" # remove leading 'v'
if [[ ! $GIT_DESCRIBE_TAG =~ [a-z] ]]; then
rapids-logger "Most recent tag is for release, adding `rapidsai` channel and removing `rapidsai-nightly` and `dask/label/dev` channels."
sed -i 's|rapidsai-nightly|rapidsai|;\|dask/label/dev|d' context/condarc
rapids-logger "Most recent tag is for release, adding `rapidsai` channel and removing `rapidsai-nightly` channel."
raydouglass marked this conversation as resolved.
Show resolved Hide resolved
sed -i 's|rapidsai-nightly|rapidsai|' context/condarc
else
rapids-logger "Most recent tag is an alpha. Build will use nightly channels."
fi
Expand Down Expand Up @@ -104,7 +100,6 @@ jobs:
LINUX_VER=${{ inputs.LINUX_VER }}
PYTHON_VER=${{ inputs.PYTHON_VER }}
RAPIDS_VER=${{ inputs.RAPIDS_VER }}
DASK_SQL_VER=${{ inputs.DASK_SQL_VER }}
tags: ${{ inputs.BASE_TAG }}-${{ matrix.ARCH }}
- name: Build notebooks image
uses: docker/build-push-action@v4
Expand All @@ -121,7 +116,6 @@ jobs:
LINUX_VER=${{ inputs.LINUX_VER }}
PYTHON_VER=${{ inputs.PYTHON_VER }}
RAPIDS_VER=${{ inputs.RAPIDS_VER }}
DASK_SQL_VER=${{ inputs.DASK_SQL_VER }}
tags: ${{ inputs.NOTEBOOKS_TAG }}-${{ matrix.ARCH }}
- name: Build RAFT ANN Benchmarks GPU image
uses: docker/build-push-action@v4
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/build-test-publish-images.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,6 @@ jobs:
LINUX_VER: ${{ matrix.LINUX_VER }}
PYTHON_VER: ${{ matrix.PYTHON_VER }}
RAPIDS_VER: ${{ needs.compute-matrix.outputs.RAPIDS_VER }}
DASK_SQL_VER: ${{ matrix.DASK_SQL_VER }}
BASE_TAG:
"rapidsai/${{ needs.compute-matrix.outputs.BASE_IMAGE_REPO }}:\
${{ needs.compute-matrix.outputs.BASE_TAG_PREFIX }}\
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-to-ngc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
id: generate-matrix
run: |
#!/bin/bash
matrix=$(yq 'del(.DASK_SQL_VER)' matrix.yaml | yq -o json | jq -c)
matrix=$(yq '.' matrix.yaml | yq -o json | jq -c)
echo "matrix=${matrix}" | tee -a ${GITHUB_OUTPUT}
copy-images:
Expand Down
66 changes: 0 additions & 66 deletions .github/workflows/update-dask-sql.yml

This file was deleted.

1 change: 0 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,3 @@ To build just the `base` image with default arguments: `docker buildx build --pu
- `CUDA_VER` - Version of CUDA to use. Should be `major.minor.patch`
- `PYTHON_VER` - Version of Python to use. Should be `major.minor`
- `RAPIDS_VER` - Version of RAPIDS to use. Should be `YY.MM`
- `DASK_SQL_VER` - Version of `dask-sql` to use. Should be `YYYY.M.P`
6 changes: 1 addition & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,13 @@ ARG LINUX_DISTRO_VER=22.04
ARG LINUX_VER=${LINUX_DISTRO}${LINUX_DISTRO_VER}

ARG RAPIDS_VER=24.08
ARG DASK_SQL_VER=2024.5.0

# Gather dependency information
FROM rapidsai/ci-conda:latest AS dependencies
ARG CUDA_VER
ARG PYTHON_VER

ARG RAPIDS_VER
ARG DASK_SQL_VER

ARG RAPIDS_BRANCH="branch-${RAPIDS_VER}"

Expand All @@ -41,7 +39,6 @@ ARG CUDA_VER
ARG PYTHON_VER

ARG RAPIDS_VER
ARG DASK_SQL_VER

SHELL ["/bin/bash", "-euo", "pipefail", "-c"]

Expand All @@ -56,7 +53,6 @@ COPY condarc /opt/conda/.condarc
RUN <<EOF
mamba install -y -n base \
"rapids=${RAPIDS_VER}.*" \
"dask-sql=${DASK_SQL_VER%.*}.*" \
"python=${PYTHON_VER}.*" \
"cuda-version=${CUDA_VER%.*}.*" \
ipython
Expand Down Expand Up @@ -138,7 +134,7 @@ LABEL com.nvidia.workbench.package-manager-environment.type="conda"
LABEL com.nvidia.workbench.package-manager.apt.binary="/usr/bin/apt"
LABEL com.nvidia.workbench.package-manager.apt.installed-packages=""
LABEL com.nvidia.workbench.package-manager.conda3.binary="/opt/conda/bin/conda"
LABEL com.nvidia.workbench.package-manager.conda3.installed-packages="rapids cudf cuml cugraph rmm pylibraft cuspatial cuxfilter cucim xgboost dask-sql jupyterlab"
LABEL com.nvidia.workbench.package-manager.conda3.installed-packages="rapids cudf cuml cugraph rmm pylibraft cuspatial cuxfilter cucim xgboost jupyterlab"
LABEL com.nvidia.workbench.package-manager.pip.binary="/opt/conda/bin/pip"
LABEL com.nvidia.workbench.package-manager.pip.installed-packages="jupyterlab-nvdashboard"
LABEL com.nvidia.workbench.programming-languages="python3"
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ There are two image types: `base` ([`rapidsai/base`](https://hub.docker.com/r/ra

This image can be found here: https://hub.docker.com/r/rapidsai/base

It contains the basic installation of RAPIDS and [`dask-sql`](https://github.com/dask-contrib/dask-sql). By default it starts an `ipython` REPL.
It contains the basic installation of RAPIDS. By default it starts an `ipython` REPL.

### Notebooks image

Expand Down
10 changes: 1 addition & 9 deletions ci/compute-matrix.jq
Original file line number Diff line number Diff line change
@@ -1,13 +1,5 @@
def compute_arch($x):
["amd64"] |
if
$x.LINUX_VER != "ubuntu20.04" # Dask-sql arm64 requires glibc >=2.32
then
. + ["arm64"]
else
.
end |
$x + {ARCHES: .};
$x + {ARCHES: ["amd64", "arm64"]};

def compute_ubuntu_version($x):
if
Expand Down
1 change: 0 additions & 1 deletion context/condarc
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
auto_update_conda: False
channels:
- rapidsai-nightly
- dask/label/dev
raydouglass marked this conversation as resolved.
Show resolved Hide resolved
- pytorch
- conda-forge
- nvidia
2 changes: 0 additions & 2 deletions context/notebooks.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
# Clones repos with notebooks & compiles notebook test dependencies
# Requires environment variables:
# RAPIDS_BRANCH
# DASK_SQL_VER
# CUDA_VER
# PYTHON_VER

Expand Down Expand Up @@ -41,6 +40,5 @@ done

pushd "/dependencies"
conda-merge ./*.yaml |
yq ".dependencies += [\"dask-sql==${DASK_SQL_VER%.*}.*\"]" | # Ensure dask-sql dependency is not altered
yq '.channels = load("/condarc").channels' | # Use channels provided by CI, not repos
tee /test_notebooks_dependencies.yaml
1 change: 0 additions & 1 deletion dockerhub-readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ RAPIDS Libraries included in the images:
- `cuxfilter`
- `cuCIM`
- `xgboost`
- `dask-sql`


### Image Types
Expand Down
2 changes: 0 additions & 2 deletions matrix.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,3 @@ PYTHON_VER:
- "3.9"
- "3.10"
- "3.11"
DASK_SQL_VER:
- "2024.5.0"
Loading