diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index 31734f9c..69237b29 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -1,6 +1,8 @@ { "name": "nfcore", - "image": "ghcr.io/nextflow-io/training:latest", + "build": { + "dockerfile": "../.github/gitpod.Dockerfile" + }, "remoteUser": "gitpod", "features": { "ghcr.io/devcontainers/features/docker-outside-of-docker:1": {} @@ -23,7 +25,7 @@ "ms-python.vscode-pylance", "nf-core.nf-core-extensionpack", "nextflow.nextflow", - "codezombiech.gitignore", + "codezombiech.gitignore" ] } }, diff --git a/docs/hello_nextflow/10_hello_seqera.md b/docs/hello_nextflow/10_hello_seqera.md index 367802ab..5781b383 100644 --- a/docs/hello_nextflow/10_hello_seqera.md +++ b/docs/hello_nextflow/10_hello_seqera.md @@ -12,9 +12,7 @@ In this section, we'll introduce you to Seqera Platform, a powerful cloud-based [Sign up](https://cloud.seqera.io/) to try Seqera for free or request a [demo](https://seqera.io/demo/) for deployments in your own on-premise or cloud environment. -You'll learn how to use Seqera Platform within Nextflow CLI, the Seqera Platform GUI, and the API. - -You can use Seqera Platform via either the **CLI**, through the **online GUI** or through the **API**. +You'll learn how to use Seqera Platform within Nextflow CLI command-line interface, the Seqera Platform GUI (Graphical User Interface), and the Seqera API (Application Program Interface). --8<-- "docs/hello_nextflow/seqera/01_run_with_cli.md" diff --git a/docs/hello_nextflow/seqera/01_run_with_cli.md b/docs/hello_nextflow/seqera/01_run_with_cli.md index 681dac88..c9b91000 100644 --- a/docs/hello_nextflow/seqera/01_run_with_cli.md +++ b/docs/hello_nextflow/seqera/01_run_with_cli.md @@ -33,7 +33,7 @@ Follow these steps to set up your token: Open a terminal and type: ```bash - export TOWER_ACCESS_TOKEN=eyxxxxxxxxxxxxxxxQ1ZTE= + export TOWER_ACCESS_TOKEN=eyxxxxxxxxxxxxxxxQ1ZTE= ``` Where `eyxxxxxxxxxxxxxxxQ1ZTE=` is the token you have just created. @@ -84,30 +84,31 @@ Before we set the configuration, we need to permanently store the token in Nextf nextflow secrets set tower_access_token "eyxxxxxxxxxxxxxxxQ1ZTE=" ``` -The following block of configuration will enable Seqera Platform logging by default: +We want to configure Nextflow to use Seqera Platform by default across all our pipelines, so we will open the global Nextflow configuration file (`$HOME/.nextflow/config`) for editing: -```groovy title="nextflow.config" -tower { - enabled = true - endpoint = "https://api.cloud.seqera.io" - accessToken = secrets.tower_access_token -} +```bash +code $HOME/.nextflow/config ``` -However, instead of enabling Seqera Platform for an individual pipeline, we want to enable it for ourselves globally. - -Run the following command to put the config block in your user configuration file located at `$HOME/.nextflow/config`. +Add the following configuration to the file: -```bash -cat <> $HOME/.nextflow/config +```groovy title="$HOME/.nextflow/config" tower { enabled = true - endpoint = "https://api.cloud.seqera.io" accessToken = secrets.tower_access_token + workspaceId = secrets.tower_workspace_id + endpoint = "https://api.cloud.seqera.io" } -EOF ``` +!!! hint "Workspace ID and Endpoint` + + We haven't set `secrets.tower_workspace_id` yet, and so Nextflow will fill in an empty string for this value. + This will default to the user's workspace in Seqera Platform which is what we want for now. + + The `endpoint` is the URL of the Seqera Platform API. + If your institution is running a private instance of Seqera Platform, you will want to change this to the appropriate URL. + Run your Nextflow workflows as usual: ```bash diff --git a/docs/hello_nextflow/seqera/03_add_compute_environment.md b/docs/hello_nextflow/seqera/03_add_compute_environment.md index ecc0d9ee..c1a2acc6 100644 --- a/docs/hello_nextflow/seqera/03_add_compute_environment.md +++ b/docs/hello_nextflow/seqera/03_add_compute_environment.md @@ -23,4 +23,73 @@ To practice this process, we will simulate setting up a new slurm compute enviro ### 3.1. Setup Tower Agent -### 3.2. Add a new Slurm compute environment +Most Seqera compute environments require provisioning a [credential](https://docs.seqera.io/platform/latest/credentials/overview) that grants access to those compute resources. +In this case, we will use [Tower Agent ](https://docs.seqera.io/platform/24.1/supported_software/agent), a lightweight program that can be installed on any machine to enable Seqera Platform to run Nextflow workflows on your behalf. + +Follow these instructions to configure your Tower Agent. + + 1. Create a new Token named "GitpodAgentToken" on the tokens page, following previous instructions. + 1. Export your token into your current terminal: + + ```bash + export TOWER_ACCESS_TOKEN= + ``` + + 1. Within Seqera Platform, click on your workspace name on the top left, and change back to your personal user workspace. + 1. Click on the "Credentials" tab + 1. Add the name "gitpodTowerAgent" to the name field + 1. Select "Tower Agent" from the list of providers. + 1. Copy the "Agent Connection ID" shown in the dropdown, it should look similar to `75d74f5f-9454-48b6-8967-cf20b74f6c78` + 1. In your terminal execute the command below replacing with your connection ID: + + ```bash + tw-agent 75d74f5f-9454-48b6-8967-cf20b74f6c78 --work-dir=./work + ``` + + 1. Return to Seqera Platform and click on the "Add" button. + +If you completed this successfully, you'll see "gitpodTowerAgent" in the list of credentials in Seqera Platform, and console output in your terminal similar to what's below: + +```console title="tw-agent logs" +21:47:33.531 INFO - Established active environments: [cli] +21:47:33.662 INFO - TOWER AGENT v0.5.0 +21:47:33.662 INFO - Compatible with TOWER API v1.8 +21:47:33.662 INFO - Connecting as user 'gitpod' with default work directory '/workspaces/training/work' +21:47:34.565 INFO - Connecting to Tower +21:47:34.801 INFO - Connection to Tower established +21:48:18.674 INFO - Sending heartbeat +21:48:18.755 INFO - Received heartbeat +21:49:03.680 INFO - Sending heartbeat +21:49:03.741 INFO - Received heartbeat +``` + +### 3.2. Add a _simulated_ Grid Engine compute environment to Seqera Plaform. + +Now that we have set up a credential granting us secure access to our computational resources, we will set up a Compute Environment in Seqera which is a set of configuration that allows us to launch nextflow pipelines. +Follow the steps below: + + 1. Navigate to the Compute Environments tab of Seqera Platform. + 1. Click "Add compute environment" + 1. Name your compute environment `gitpodGridEngine`. + 1. Select "Grid Engine" as the target execution platform. + 1. Make sure "gitpodTowerAgent" is selected from the list of credentials. + 1. Click "Add" + +### 3.3. Add a pipeline to the launchpad for execution on our "Compute Environment" + +Now we need to set up a pipeline in Seqera Platform to run on our simulated Grid Engine compute environment. + +Follow the steps below: + + 1. Click on the "Launchpad" tab of Seqera Platform. + 1. On the top right click on "Add Pipeline". + 1. For name enter `hello-grid-engine` + 1. For the Compute Environment select `gitpodGridEngine` + 1. For the "Pipeline to launch" enter `nextflow-io/hello`. + 1. Finally click "Add" + +You now should see "hello-grid-engine` in the list of pipelines. + +### 3.4. Launch your Nextflow pipeline + +To create that: diff --git a/mkdocs.yml b/mkdocs.yml index 3c3a5a54..bc121e9d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -155,6 +155,8 @@ markdown_extensions: preserve_tabs: true - pymdownx.tabbed: alternate_style: true + - pymdownx.tasklist: + custom_checkbox: true - tables - toc: title: On this page