From f1f8ed09a84a28eeb31230c81a265d9c0fbd4387 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jason=20=CE=92oxman?= <141646877+jason-seqera@users.noreply.github.com> Date: Tue, 3 Dec 2024 23:58:58 -0500 Subject: [PATCH] Use more complex example for AWS Batch UseGPU (#321) Use more complex example for AWS Batch UseGPU - https://seqera.slack.com/archives/C040MFU06AE/p1732803250849139 --- .../version-22.4/compute-envs/overview.mdx | 6 ++++-- .../version-23.1/compute-envs/overview.mdx | 6 ++++-- .../version-23.3/compute-envs/overview.mdx | 6 ++++-- .../version-23.4/compute-envs/overview.mdx | 6 ++++-- .../version-24.1/compute-envs/overview.mdx | 6 ++++-- .../version-24.2/compute-envs/overview.mdx | 6 ++++-- 6 files changed, 24 insertions(+), 12 deletions(-) diff --git a/platform_versioned_docs/version-22.4/compute-envs/overview.mdx b/platform_versioned_docs/version-22.4/compute-envs/overview.mdx index f3e0b7a3a..031f34e89 100644 --- a/platform_versioned_docs/version-22.4/compute-envs/overview.mdx +++ b/platform_versioned_docs/version-22.4/compute-envs/overview.mdx @@ -46,10 +46,12 @@ The AWS Batch compute environment creation form in Tower includes an **Enable GP - This setting can be overridden by **AMI ID** in the advanced options. -- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. For example: +- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. The `containerOptions` directive can be set inline in your process definition or via configuration. For example, to add the directive to a process named `UseGPU` via configuration: ```groovy -process UseGPU { +process { + withName: UseGPU { containerOptions '-e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all' + } } ``` diff --git a/platform_versioned_docs/version-23.1/compute-envs/overview.mdx b/platform_versioned_docs/version-23.1/compute-envs/overview.mdx index d5f591a37..76031268f 100644 --- a/platform_versioned_docs/version-23.1/compute-envs/overview.mdx +++ b/platform_versioned_docs/version-23.1/compute-envs/overview.mdx @@ -48,10 +48,12 @@ The AWS Batch compute environment creation form in Tower includes an **Enable GP - This setting can be overridden by **AMI ID** in the advanced options. -- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. For example: +- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. The `containerOptions` directive can be set inline in your process definition or via configuration. For example, to add the directive to a process named `UseGPU` via configuration: ```groovy -process UseGPU { +process { + withName: UseGPU { containerOptions '-e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all' + } } ``` diff --git a/platform_versioned_docs/version-23.3/compute-envs/overview.mdx b/platform_versioned_docs/version-23.3/compute-envs/overview.mdx index 5fc6576bb..cfc357fd6 100644 --- a/platform_versioned_docs/version-23.3/compute-envs/overview.mdx +++ b/platform_versioned_docs/version-23.3/compute-envs/overview.mdx @@ -48,10 +48,12 @@ Some important considerations: - The **Enable GPUs** setting causes Batch Forge to specify the most current [AWS-recommended GPU-optimized ECS AMI](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html) as the EC2 fleet AMI when creating the compute environment. This setting can be overridden by **AMI ID** in the advanced options. - The **Enable GPUs** setting alone does not deploy GPU instances in your compute environment. You must still specify GPU-enabled instance types in the **Advanced options > Instance types** field. - Your Nextflow script must include [accelerator directives](https://www.nextflow.io/docs/latest/process.html?highlight=accelerator#accelerator) to use the provisioned GPUs. -- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU-accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. For example: +- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. The `containerOptions` directive can be set inline in your process definition or via configuration. For example, to add the directive to a process named `UseGPU` via configuration: ```groovy -process UseGPU { +process { + withName: UseGPU { containerOptions '-e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all' + } } ``` diff --git a/platform_versioned_docs/version-23.4/compute-envs/overview.mdx b/platform_versioned_docs/version-23.4/compute-envs/overview.mdx index 5fc6576bb..cfc357fd6 100644 --- a/platform_versioned_docs/version-23.4/compute-envs/overview.mdx +++ b/platform_versioned_docs/version-23.4/compute-envs/overview.mdx @@ -48,10 +48,12 @@ Some important considerations: - The **Enable GPUs** setting causes Batch Forge to specify the most current [AWS-recommended GPU-optimized ECS AMI](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html) as the EC2 fleet AMI when creating the compute environment. This setting can be overridden by **AMI ID** in the advanced options. - The **Enable GPUs** setting alone does not deploy GPU instances in your compute environment. You must still specify GPU-enabled instance types in the **Advanced options > Instance types** field. - Your Nextflow script must include [accelerator directives](https://www.nextflow.io/docs/latest/process.html?highlight=accelerator#accelerator) to use the provisioned GPUs. -- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU-accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. For example: +- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. The `containerOptions` directive can be set inline in your process definition or via configuration. For example, to add the directive to a process named `UseGPU` via configuration: ```groovy -process UseGPU { +process { + withName: UseGPU { containerOptions '-e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all' + } } ``` diff --git a/platform_versioned_docs/version-24.1/compute-envs/overview.mdx b/platform_versioned_docs/version-24.1/compute-envs/overview.mdx index 5fc6576bb..cfc357fd6 100644 --- a/platform_versioned_docs/version-24.1/compute-envs/overview.mdx +++ b/platform_versioned_docs/version-24.1/compute-envs/overview.mdx @@ -48,10 +48,12 @@ Some important considerations: - The **Enable GPUs** setting causes Batch Forge to specify the most current [AWS-recommended GPU-optimized ECS AMI](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html) as the EC2 fleet AMI when creating the compute environment. This setting can be overridden by **AMI ID** in the advanced options. - The **Enable GPUs** setting alone does not deploy GPU instances in your compute environment. You must still specify GPU-enabled instance types in the **Advanced options > Instance types** field. - Your Nextflow script must include [accelerator directives](https://www.nextflow.io/docs/latest/process.html?highlight=accelerator#accelerator) to use the provisioned GPUs. -- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU-accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. For example: +- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. The `containerOptions` directive can be set inline in your process definition or via configuration. For example, to add the directive to a process named `UseGPU` via configuration: ```groovy -process UseGPU { +process { + withName: UseGPU { containerOptions '-e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all' + } } ``` diff --git a/platform_versioned_docs/version-24.2/compute-envs/overview.mdx b/platform_versioned_docs/version-24.2/compute-envs/overview.mdx index 5fc6576bb..cfc357fd6 100644 --- a/platform_versioned_docs/version-24.2/compute-envs/overview.mdx +++ b/platform_versioned_docs/version-24.2/compute-envs/overview.mdx @@ -48,10 +48,12 @@ Some important considerations: - The **Enable GPUs** setting causes Batch Forge to specify the most current [AWS-recommended GPU-optimized ECS AMI](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html) as the EC2 fleet AMI when creating the compute environment. This setting can be overridden by **AMI ID** in the advanced options. - The **Enable GPUs** setting alone does not deploy GPU instances in your compute environment. You must still specify GPU-enabled instance types in the **Advanced options > Instance types** field. - Your Nextflow script must include [accelerator directives](https://www.nextflow.io/docs/latest/process.html?highlight=accelerator#accelerator) to use the provisioned GPUs. -- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU-accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. For example: +- The NVIDIA Container Runtime uses [environment variables](https://github.com/NVIDIA/nvidia-container-runtime#environment-variables-oci-spec) in container images to specify a GPU accelerated container. These variables should be included in the [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#process-containeroptions) directive for each GPU-dependent process in your Nextflow script. The `containerOptions` directive can be set inline in your process definition or via configuration. For example, to add the directive to a process named `UseGPU` via configuration: ```groovy -process UseGPU { +process { + withName: UseGPU { containerOptions '-e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all' + } } ```