From 461d89ccc48aae4b5eba8cb502762ba5434d5b29 Mon Sep 17 00:00:00 2001 From: Johnny Date: Tue, 24 Dec 2024 17:23:30 +0100 Subject: [PATCH 1/7] Update amd-installation.md --- .../source/getting_started/amd-installation.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/source/getting_started/amd-installation.md b/docs/source/getting_started/amd-installation.md index b9ccbd7d6c7fc..21ef9753176f7 100644 --- a/docs/source/getting_started/amd-installation.md +++ b/docs/source/getting_started/amd-installation.md @@ -2,14 +2,14 @@ # Installation with ROCm -vLLM supports AMD GPUs with ROCm 6.2. +vLLM supports AMD GPUs with ROCm 6.3. ## Requirements - OS: Linux - Python: 3.9 -- 3.12 - GPU: MI200s (gfx90a), MI300 (gfx942), Radeon RX 7900 series (gfx1100) -- ROCm 6.2 +- ROCm 6.3 Installation options: @@ -33,7 +33,7 @@ It is important that the user kicks off the docker build using buildkit. Either } ``` -[Dockerfile.rocm](https://github.com/vllm-project/vllm/blob/main/Dockerfile.rocm) uses ROCm 6.2 by default, but also supports ROCm 5.7, 6.0 and 6.1 in older vLLM branches. +[Dockerfile.rocm](https://github.com/vllm-project/vllm/blob/main/Dockerfile.rocm) uses ROCm 6.3 by default, but also supports ROCm 5.7, 6.0 and 6.1 in older vLLM branches. It provides flexibility to customize the build of docker image using the following arguments: - `BASE_IMAGE`: specifies the base image used when running `docker build`, specifically the PyTorch on ROCm base image. @@ -44,13 +44,13 @@ It provides flexibility to customize the build of docker image using the followi Their values can be passed in when running `docker build` with `--build-arg` options. -To build vllm on ROCm 6.2 for MI200 and MI300 series, you can use the default: +To build vllm on ROCm 6.3 for MI200 and MI300 series, you can use the default: ```console $ DOCKER_BUILDKIT=1 docker build -f Dockerfile.rocm -t vllm-rocm . ``` -To build vllm on ROCm 6.2 for Radeon RX7900 series (gfx1100), you should specify `BUILD_FA` as below: +To build vllm on ROCm 6.3 for Radeon RX7900 series (gfx1100), you should specify `BUILD_FA` as below: ```console $ DOCKER_BUILDKIT=1 docker build --build-arg BUILD_FA="0" -f Dockerfile.rocm -t vllm-rocm . @@ -83,7 +83,7 @@ Where the `` is the location where the model is stored, for examp - [ROCm](https://rocm.docs.amd.com/en/latest/deploy/linux/index.html) - [PyTorch](https://pytorch.org/) -For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.2_ubuntu20.04_py3.9_pytorch_release_2.3.0`, `rocm/pytorch-nightly`. +For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.3_ubuntu20.04_py3.9_pytorch_release_2.3.0`, `rocm/pytorch-nightly`. Alternatively, you can install PyTorch using PyTorch wheels. You can check PyTorch installation guide in PyTorch [Getting Started](https://pytorch.org/get-started/locally/) @@ -111,7 +111,7 @@ $ cd ../.. Install ROCm's flash attention (v2.5.9.post1) following the instructions from [ROCm/flash-attention](https://github.com/ROCm/flash-attention/tree/ck_tile#amd-gpurocm-support) Alternatively, wheels intended for vLLM use can be accessed under the releases. -For example, for ROCm 6.2, suppose your gfx arch is `gfx90a`. To get your gfx architecture, run `rocminfo |grep gfx`. +For example, for ROCm 6.3, suppose your gfx arch is `gfx90a`. To get your gfx architecture, run `rocminfo |grep gfx`. ```console $ git clone https://github.com/ROCm/flash-attention.git @@ -126,14 +126,14 @@ $ cd .. - You might need to downgrade the "ninja" version to 1.10 it is not used when compiling flash-attention-2 (e.g. `pip install ninja==1.10.2.4`) ``` -3. Build vLLM. For example, vLLM on ROCM 6.2 can be built with the following steps: +3. Build vLLM. For example, vLLM on ROCM 6.3 can be built with the following steps: ```bash $ pip install --upgrade pip # Install PyTorch $ pip uninstall torch -y -$ pip install --no-cache-dir --pre torch==2.6.0.dev20241024 --index-url https://download.pytorch.org/whl/nightly/rocm6.2 +$ pip install --no-cache-dir --pre torch==2.6.0.dev20241024 --index-url https://download.pytorch.org/whl/nightly/rocm6.3 # Build & install AMD SMI $ pip install /opt/rocm/share/amd_smi From 8d19665a703a6ba6bbcf01e1264ecf2def8c841c Mon Sep 17 00:00:00 2001 From: Johnny Date: Tue, 24 Dec 2024 17:26:11 +0100 Subject: [PATCH 2/7] Update amd-installation.md --- docs/source/getting_started/amd-installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting_started/amd-installation.md b/docs/source/getting_started/amd-installation.md index 21ef9753176f7..b8f3563d6912b 100644 --- a/docs/source/getting_started/amd-installation.md +++ b/docs/source/getting_started/amd-installation.md @@ -133,7 +133,7 @@ $ pip install --upgrade pip # Install PyTorch $ pip uninstall torch -y -$ pip install --no-cache-dir --pre torch==2.6.0.dev20241024 --index-url https://download.pytorch.org/whl/nightly/rocm6.3 +$ pip install --no-cache-dir --pre torch==2.6.0.dev20241224 --index-url https://download.pytorch.org/whl/nightly/rocm6.3 # Build & install AMD SMI $ pip install /opt/rocm/share/amd_smi From 9e42b53f9c007fca416d5e486cbb289f79f6835d Mon Sep 17 00:00:00 2001 From: Johnny Date: Tue, 24 Dec 2024 17:28:04 +0100 Subject: [PATCH 3/7] Update amd-installation.md --- docs/source/getting_started/amd-installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting_started/amd-installation.md b/docs/source/getting_started/amd-installation.md index b8f3563d6912b..cefaa0c71787d 100644 --- a/docs/source/getting_started/amd-installation.md +++ b/docs/source/getting_started/amd-installation.md @@ -83,7 +83,7 @@ Where the `` is the location where the model is stored, for examp - [ROCm](https://rocm.docs.amd.com/en/latest/deploy/linux/index.html) - [PyTorch](https://pytorch.org/) -For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.3_ubuntu20.04_py3.9_pytorch_release_2.3.0`, `rocm/pytorch-nightly`. +For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.3_ubuntu20.04_py3.9_pytorch_release_2.4.0`, `rocm/pytorch-nightly`. Alternatively, you can install PyTorch using PyTorch wheels. You can check PyTorch installation guide in PyTorch [Getting Started](https://pytorch.org/get-started/locally/) From f853174bcdcbda6601695ae4a5761c38ef217975 Mon Sep 17 00:00:00 2001 From: Johnny Date: Tue, 24 Dec 2024 17:36:11 +0100 Subject: [PATCH 4/7] Update amd-installation.md --- docs/source/getting_started/amd-installation.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/source/getting_started/amd-installation.md b/docs/source/getting_started/amd-installation.md index cefaa0c71787d..73f1889d84e35 100644 --- a/docs/source/getting_started/amd-installation.md +++ b/docs/source/getting_started/amd-installation.md @@ -8,7 +8,7 @@ vLLM supports AMD GPUs with ROCm 6.3. - OS: Linux - Python: 3.9 -- 3.12 -- GPU: MI200s (gfx90a), MI300 (gfx942), Radeon RX 7900 series (gfx1100) +- GPU: MI200s (gfx90a), MI300 (gfx942), Radeon Pro, Radeon RX 7900 series (gfx1100) - ROCm 6.3 Installation options: @@ -83,7 +83,7 @@ Where the `` is the location where the model is stored, for examp - [ROCm](https://rocm.docs.amd.com/en/latest/deploy/linux/index.html) - [PyTorch](https://pytorch.org/) -For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.3_ubuntu20.04_py3.9_pytorch_release_2.4.0`, `rocm/pytorch-nightly`. +For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.3_ubuntu24.04_py3.12_pytorch_release_2.4.0`, `rocm/pytorch-nightly`. Alternatively, you can install PyTorch using PyTorch wheels. You can check PyTorch installation guide in PyTorch [Getting Started](https://pytorch.org/get-started/locally/) @@ -94,9 +94,9 @@ Install ROCm's Triton flash attention (the default triton-mlir branch) following ```console $ python3 -m pip install ninja cmake wheel pybind11 $ pip uninstall -y triton -$ git clone https://github.com/OpenAI/triton.git +$ git clone https://github.com/triton-lang/triton $ cd triton -$ git checkout e192dba +$ git checkout release/3.2.x $ cd python $ pip3 install . $ cd ../.. @@ -116,7 +116,6 @@ For example, for ROCm 6.3, suppose your gfx arch is `gfx90a`. To get your gfx ar ```console $ git clone https://github.com/ROCm/flash-attention.git $ cd flash-attention -$ git checkout 3cea2fb $ git submodule update --init $ GPU_ARCHS="gfx90a" python3 setup.py install $ cd .. From cb57c0fe878982b70f81ae7ade09632179e13e5d Mon Sep 17 00:00:00 2001 From: Johnny Date: Tue, 24 Dec 2024 17:50:53 +0100 Subject: [PATCH 5/7] Update Dockerfile.rocm --- Dockerfile.rocm | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Dockerfile.rocm b/Dockerfile.rocm index e733994f8c33e..851af41477309 100644 --- a/Dockerfile.rocm +++ b/Dockerfile.rocm @@ -1,5 +1,5 @@ -# Default ROCm 6.2 base image -ARG BASE_IMAGE="rocm/pytorch:rocm6.2_ubuntu20.04_py3.9_pytorch_release_2.3.0" +# Default ROCm 6.3 base image +ARG BASE_IMAGE="rocm/pytorch:rocm6.3_ubuntu24.04_py3.9_pytorch_release_2.5.0" # Default ROCm ARCHes to build vLLM for. ARG PYTORCH_ROCM_ARCH="gfx908;gfx90a;gfx942;gfx1100" @@ -48,13 +48,13 @@ RUN apt-get purge -y sccache; python3 -m pip uninstall -y sccache; rm -f "$(whic # Install torch == 2.6.0 on ROCm RUN --mount=type=cache,target=/root/.cache/pip \ case "$(ls /opt | grep -Po 'rocm-[0-9]\.[0-9]')" in \ - *"rocm-6.2"*) \ + *"rocm-6.3"*) \ python3 -m pip uninstall -y torch torchvision \ && python3 -m pip install --pre \ - torch==2.6.0.dev20241113+rocm6.2 \ + torch==2.6.0.dev20241224+rocm6.3 \ 'setuptools-scm>=8' \ - torchvision==0.20.0.dev20241113+rocm6.2 \ - --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.2;; \ + torchvision==0.20.0.dev20241224+rocm6.3 \ + --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.3;; \ *) ;; esac ENV LLVM_SYMBOLIZER_PATH=/opt/rocm/llvm/bin/llvm-symbolizer From 109ca723c9dbc8f42a1fc089dd591a72d5544e29 Mon Sep 17 00:00:00 2001 From: Johnny Date: Tue, 24 Dec 2024 17:53:19 +0100 Subject: [PATCH 6/7] Update amd-installation.md --- docs/source/getting_started/amd-installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting_started/amd-installation.md b/docs/source/getting_started/amd-installation.md index 73f1889d84e35..56df08e32ce80 100644 --- a/docs/source/getting_started/amd-installation.md +++ b/docs/source/getting_started/amd-installation.md @@ -33,7 +33,7 @@ It is important that the user kicks off the docker build using buildkit. Either } ``` -[Dockerfile.rocm](https://github.com/vllm-project/vllm/blob/main/Dockerfile.rocm) uses ROCm 6.3 by default, but also supports ROCm 5.7, 6.0 and 6.1 in older vLLM branches. +[Dockerfile.rocm](https://github.com/vllm-project/vllm/blob/main/Dockerfile.rocm) uses ROCm 6.3 by default, but also supports ROCm 5.7, 6.0, 6.1 and 6.2 in older vLLM branches. It provides flexibility to customize the build of docker image using the following arguments: - `BASE_IMAGE`: specifies the base image used when running `docker build`, specifically the PyTorch on ROCm base image. From 28fc44d53e83f03877df0fc83b68a10212876b79 Mon Sep 17 00:00:00 2001 From: Johnny Date: Tue, 24 Dec 2024 17:54:03 +0100 Subject: [PATCH 7/7] Update amd-installation.md --- docs/source/getting_started/amd-installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting_started/amd-installation.md b/docs/source/getting_started/amd-installation.md index 56df08e32ce80..d9ed67afe4213 100644 --- a/docs/source/getting_started/amd-installation.md +++ b/docs/source/getting_started/amd-installation.md @@ -83,7 +83,7 @@ Where the `` is the location where the model is stored, for examp - [ROCm](https://rocm.docs.amd.com/en/latest/deploy/linux/index.html) - [PyTorch](https://pytorch.org/) -For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.3_ubuntu24.04_py3.12_pytorch_release_2.4.0`, `rocm/pytorch-nightly`. +For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.3_ubuntu24.04_py3.12_pytorch_release_2.5.0`, `rocm/pytorch-nightly`. Alternatively, you can install PyTorch using PyTorch wheels. You can check PyTorch installation guide in PyTorch [Getting Started](https://pytorch.org/get-started/locally/)