Skip to content

Commit

Permalink
build cu11 for docker and C interface library
Browse files Browse the repository at this point in the history
  • Loading branch information
njzjz authored Nov 20, 2023
1 parent 2f679d2 commit ff5369e
Show file tree
Hide file tree
Showing 7 changed files with 40 additions and 13 deletions.
15 changes: 13 additions & 2 deletions .github/workflows/build_wheel.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,14 @@ jobs:
# use the already built wheels to build docker
needs: [build_wheels]
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- variant: ""
cuda_version: "12"
- variant: "_cu11"
cuda_version: "11"
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v3
Expand All @@ -142,8 +150,11 @@ jobs:
with:
context: source/install/docker
push: ${{ github.repository_owner == 'deepmodeling' && github.event_name == 'push' }}
tags: ${{ steps.meta.outputs.tags }}
tags: ${{ steps.meta.outputs.tags }}${{ matrix.variant }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
VARIANT=${{ matrix.variant }}
CUDA_VERSION=${{ matrix.cuda_version }}
build_pypi_index:
needs: [build_wheels, build_sdist]
Expand Down Expand Up @@ -182,7 +193,7 @@ jobs:

pass:
name: Pass testing build wheels
needs: [build_wheels, build_sdist]
needs: [build_wheels, build_sdist, build_docker, deploy_pypi_index]
runs-on: ubuntu-latest
if: always()
steps:
Expand Down
17 changes: 15 additions & 2 deletions .github/workflows/package_c.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,36 @@ jobs:
build_c:
name: Build C library
runs-on: ubuntu-22.04
strategy:
matrix:
include:
- tensorflow_build_version: "2.15"
tensorflow_version: ""
filename: libdeepmd_c.tar.gz
- tensorflow_build_version: "2.14"
tensorflow_version: ">=2.5.0rc0,<2.15"
filename: libdeepmd_c_cu11.tar.gz
steps:
- uses: actions/checkout@v4
- name: Package C library
run: ./source/install/docker_package_c.sh
env:
TENSORFLOW_VERSION: ${{ matrix.tensorflow_version }}
TENSORFLOW_BUILD_VERSION: ${{ matrix.tensorflow_build_version }}
- run: “cp libdeepmd_c.tar.gz ${{ matrix.filename }} || :”
# for download and debug
- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: libdeepmd_c
path: ./libdeepmd_c.tar.gz
path: ${{ matrix.filename }}
- name: Test C library
run: ./source/install/docker_test_package_c.sh
- name: Release
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/')
with:
files: libdeepmd_c.tar.gz
files: ${{ matrix.filename }}
test_c:
name: Test building from C library
needs: [build_c]
Expand Down
4 changes: 3 additions & 1 deletion doc/install/easy-install-dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@ The following is the way to install the pre-compiled packages without [building

## Install with docker

The [`devel` tag](https://github.com/deepmodeling/deepmd-kit/pkgs/container/deepmd-kit/131827568?tag=devel) is used to mark the latest development version with CUDA support:
The [`devel` tag](https://github.com/deepmodeling/deepmd-kit/pkgs/container/deepmd-kit/131827568?tag=devel) is used to mark the latest development version with CUDA 12.2 support:

```bash
docker pull ghcr.io/deepmodeling/deepmd-kit:devel
```

For CUDA 11.8 support, use the `devel_cu11` tag.

## Install with pip

Below is an one-line shell command to download the [artifact](https://nightly.link/deepmodeling/deepmd-kit/workflows/build_wheel/devel/artifact.zip) containing wheels and install it with `pip`:
Expand Down
2 changes: 1 addition & 1 deletion doc/install/install-from-c-library.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

DeePMD-kit provides pre-compiled C library package (`libdeepmd_c.tar.gz`) in each [release](https://github.com/deepmodeling/deepmd-kit/releases). It can be used to build the [LAMMPS plugin](./install-lammps.md) and [GROMACS patch](./install-gromacs.md), as well as many [third-party software packages](../third-party/out-of-deepmd-kit.md), without building TensorFlow and DeePMD-kit on one's own.

The library is built in Linux (GLIBC 2.17) with CUDA 12.2. It's noted that this package does not contain CUDA Toolkit and cuDNN, so one needs to download them from the NVIDIA website.
The library is built in Linux (GLIBC 2.17) with CUDA 12.2 (`libdeepmd_c.tar.gz`) or 11.8 (`libdeepmd_c_cu11.tar.gz`). It's noted that this package does not contain CUDA Toolkit and cuDNN, so one needs to download them from the NVIDIA website.

## Use Pre-compiled C Library to build the LAMMPS plugin and GROMACS patch

Expand Down
3 changes: 1 addition & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -135,9 +135,8 @@ build = ["cp310-*"]
skip = ["*-win32", "*-manylinux_i686", "*-musllinux*"]
# TODO: uncomment when CUDA 11 is deprecated
# manylinux-x86_64-image = "manylinux_2_28"
# manylinux-aarch64-image = "manylinux_2_28"
manylinux-x86_64-image = "quay.io/pypa/manylinux_2_28_x86_64:2022-11-19-1b19e81"
manylinux-aarch64-image = "quay.io/pypa/manylinux_2_28_aarch64:2022-11-19-1b19e81"
manylinux-aarch64-image = "manylinux_2_28"

[tool.cibuildwheel.macos]
environment = { PIP_PREFER_BINARY="1", DP_LAMMPS_VERSION="stable_2Aug2023_update1", DP_ENABLE_IPI="1" }
Expand Down
8 changes: 5 additions & 3 deletions source/install/docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
FROM python:3.10 AS compile-image
FROM python:3.11 AS compile-image
ARG VARIANT=""
ARG CUDA_VERSION="12"
RUN python -m venv /opt/deepmd-kit
# Make sure we use the virtualenv
ENV PATH="/opt/deepmd-kit/bin:$PATH"
# Install package
COPY dist /dist
RUN pip install "$(ls /dist/deepmd_kit-*manylinux*_x86_64.whl)[gpu,cu12,lmp,ipi]" \
RUN pip install "$(ls /dist/deepmd_kit${VARIANT}-*manylinux*_x86_64.whl)[gpu,cu${CUDA_VERSION},lmp,ipi]" \
&& dp -h \
&& lmp -h \
&& dp_ipi \
&& python -m deepmd -h

FROM python:3.10 AS build-image
FROM python:3.11 AS build-image
COPY --from=compile-image /opt/deepmd-kit /opt/deepmd-kit
ENV PATH="/opt/deepmd-kit/bin:$PATH"
CMD ["/bin/bash"]
4 changes: 2 additions & 2 deletions source/install/docker_package_c.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ set -e
SCRIPT_PATH=$(dirname $(realpath -s $0))

docker run --rm -v ${SCRIPT_PATH}/../..:/root/deepmd-kit -w /root/deepmd-kit \
tensorflow/build:2.15-python3.11 \
/bin/sh -c "pip install tensorflow cmake \
tensorflow/build:${TENSORFLOW_BUILD_VERSION:-2.15}-python3.11 \
/bin/sh -c "pip install tensorflow${TENSORFLOW_VERSION} cmake \
&& cd /root/deepmd-kit/source/install \
&& CC=/dt9/usr/bin/gcc \
CXX=/dt9/usr/bin/g++ \
Expand Down

0 comments on commit ff5369e

Please sign in to comment.