diff --git a/docs/big_nextflow/01_orientation.md b/docs/big_nextflow/01_orientation.md new file mode 100644 index 00000000..d2d2aee5 --- /dev/null +++ b/docs/big_nextflow/01_orientation.md @@ -0,0 +1,46 @@ +# Orientation + +The Gitpod environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself. +However, you do need a (free) account to log in, and you should take a few minutes to familiarize yourself with the interface. + +If you have not yet done so, please follow [this link](../../envsetup/) before going any further. + +## Materials provided + +Throughout this training course, we'll be working in the `big-nextflow/` directory. +This directory contains all the code files, test data and accessory files you will need. + +Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left-hand side of the Gitpod workspace. +Alternatively, you can use the `tree` command. +Throughout the course, we use the output of `tree` to represent directory structure and contents in a readable form, sometimes with minor modifications for clarity. + +Here we generate a table of contents to the second level down: + +```bash +tree . -L 2 +``` + +If you run this inside `big-nextflow`, you should see the following output: [TODO] + +```console title="Directory contents" +. +``` + +!!!note + + Don't worry if this seems like a lot; we'll go through the relevant pieces at each step of the course. + This is just meant to give you an overview. + +**Here's a summary of what you should know to get started:** + +[TODO] + +!!!tip + + If for whatever reason you move out of this directory, you can always run this command to return to it: + + ```bash + cd /workspace/gitpod/big-nextflow + ``` + +Now, to begin the course, click on the arrow in the bottom right corner of this page. diff --git a/docs/big_nextflow/index.md b/docs/big_nextflow/index.md new file mode 100644 index 00000000..493eb17b --- /dev/null +++ b/docs/big_nextflow/index.md @@ -0,0 +1,31 @@ +--- +title: Big Nextflow +hide: + - toc +--- + +# Big Nextflow + +[TODO]] + +Let's get started! + +[![Open in Gitpod](https://img.shields.io/badge/Gitpod-%20Open%20in%20Gitpod-908a85?logo=gitpod)](https://gitpod.io/#https://github.com/nextflow-io/training) + +## Learning objectives + +In this workshop, you will learn [TODO]. + +By the end of this workshop you will be able to: + +[TODO] + +## Audience & prerequisites + +[TODO] + +**Prerequisites** + +- A GitHub account +- Experience with command line + [TODO] diff --git a/docs/hello_nextflow/03_hello_containers.md b/docs/hello_nextflow/03_hello_containers.md index ec975b17..50a710e2 100644 --- a/docs/hello_nextflow/03_hello_containers.md +++ b/docs/hello_nextflow/03_hello_containers.md @@ -174,7 +174,7 @@ You know how to pull a container and run it interactively, make your data access ### What's next? -Learn how to get a container image for any pip/conda-installable tool. +[TODO] update text (was wrong one) --- @@ -183,6 +183,8 @@ Learn how to get a container image for any pip/conda-installable tool. Nextflow has built-in support for running processes inside containers to let you run tools you don't have installed in your compute environment. This means that you can use any container image you like to run your processes, and Nextflow will take care of pulling the image, mounting the data, and running the process inside it. +[TODO] [Update this to add a cowsay step to the hello-world pipeline (just add it after the uppercase step) -- include passing in the character as a parameter] + ### 2.1. Add a container directive to your process Edit the `hello-containers.nf` script to add a `container` directive to the `cowsay` process. @@ -281,191 +283,4 @@ You know how to use containers in Nextflow to run processes. ### What's next? -You have everything you need to continue to the [next chapter](./04_hello_genomics.md) of this training series. -Optionally, continue on to learn how to get container images for tools you want to use in your Nextflow pipelines. - ---- - -## 3. Optional Topic: How to find or make container images - -Some software developers provide container images for their software that are available on container registries like Docker Hub, but many do not. -In this optional section, we'll show you to two ways to get a container image for tools you want to use in your Nextflow pipelines: using Seqera Containers and building the container image yourself. - -You'll be getting/building a container image for the `quote` pip package, which will be used in the exercise at the end of this section. - -### 3.1. Get a container image from Seqera Containers - -Seqera Containers is a free service that builds container images for pip and conda (including bioconda) installable tools. -Navigate to [Seqera Containers](https://www.seqera.io/containers/) and search for the `quote` pip package. - -![Seqera Containers](img/seqera-containers-1.png) - -Click on "+Add" and then "Get Container" to request a container image for the `quote` pip package. - -![Seqera Containers](img/seqera-containers-2.png) - -If this is the first time a community container has been built for this version of the package, it may take a few minutes to complete. -Click to copy the URI (e.g. `community.wave.seqera.io/library/pip_quote:ae07804021465ee9`) of the container image that was created for you. - -You can now use the container image to run the `quote` command and get a random saying from Grace Hopper. - -```bash -docker run --rm community.wave.seqera.io/library/pip_quote:ae07804021465ee9 quote "Grace Hopper" -``` - -Output: - -```console title="Output" -Humans are allergic to change. They love to say, 'We've always done it -this way.' I try to fight that. That's why I have a clock on my wall -that runs counter-clockwise. -``` - -### 3.2. Build the container image yourself - -Let's use some build details from the Seqera Containers website to build the container image for the `quote` pip package ourselves. -Return to the Seqera Containers website and click on the "Build Details" button. - -The first item we'll look at is the `Dockerfile`, a type of script file that contains all the commands needed to build the container image. -We've added some explanatory comments to the Dockerfile below to help you understand what each part does. - -```Dockerfile title="Dockerfile" -# Start from the micromamba base docker image -FROM mambaorg/micromamba:1.5.10-noble -# Copy the conda.yml file into the container -COPY --chown=$MAMBA_USER:$MAMBA_USER conda.yml /tmp/conda.yml -# Install various utilities for Nextflow to use and the packages in the conda.yml file -RUN micromamba install -y -n base -f /tmp/conda.yml \ - && micromamba install -y -n base conda-forge::procps-ng \ - && micromamba env export --name base --explicit > environment.lock \ - && echo ">> CONDA_LOCK_START" \ - && cat environment.lock \ - && echo "<< CONDA_LOCK_END" \ - && micromamba clean -a -y -# Run the container as the root user -USER root -# Set the PATH environment variable to include the micromamba installation directory -ENV PATH="$MAMBA_ROOT_PREFIX/bin:$PATH" -``` - -The second item we'll look at is the `conda.yml` file, which contains the list of packages that need to be installed in the container image. - -```conda.yml title="conda.yml" -channels: -- conda-forge -- bioconda -dependencies: -- pip -- pip: - - quote==3.0.0 # -``` - -Copy the contents of these files into the stubs located in the `containers/build` directory, then run the following command to build the container image yourself. - -!!! Note - - We use the `-t quote:latest` flag to tag the container image with the name `quote` and the tag `latest`. - We will be able to use this tag to refer to the container image when running it on this system. - -```bash -docker build -t quote:latest containers/build -``` - -After it has finished building, you can run the container image you just built. - -```bash -docker run --rm quote:latest quote "Margaret Oakley Dayhoff" -``` - -### Takeaway - -You've learned two different ways to get a container image for a tool you want to use in your Nextflow pipelines: using Seqera Containers and building the container image yourself. - -### What's next? - -You have everything you need to continue to the [next chapter](./04_hello_genomics.md) of this training series. -You can also continue on with an optional exercise to fetch quotes on computer/biology pioneers using the `quote` container and output them using the `cowsay` container. - ---- - -## 4. Bonus Exercise: Make the cow quote famous scientists - -This section contains some stretch exercises, to practice what you've learned so far. -Doing these exercises is _not required_ to understand later parts of the training, but provide a fun way to reinforce your learnings by figuring out how to make the cow quote famous scientists. - -```console title="cowsay-output-Grace-Hopper.txt" - _________________________________________________ - / \ -| Humans are allergic to change. They love to | -| say, 'We've always done it this way.' I try to fi | -| ght that. That's why I have a clock on my wall th | -| at runs counter-clockwise. | -| -Grace Hopper | - \ / - ================================================= - \ - \ - ^__^ - (oo)\_______ - (__)\ )\/\ - ||----w | - || || -``` - -### 4.1. Modify the `hello-containers.nf` script to use a getQuote process - -We have a list of computer and biology pioneers in the `containers/data/pioneers.csv` file. -At a high level, to complete this exercise you will need to: - -- Modify the default `params.input_file` to point to the `pioneers.csv` file. -- Create a `getQuote` process that uses the `quote` container to fetch a quote for each input. -- Connect the output of the `getQuote` process to the `cowsay` process to display the quote. - -For the `quote` container image, you can either use the one you built yourself in the previous stretch exercise or use the one you got from Seqera Containers . - -!!! Hint - - A good choice for the `script` block of your getQuote process might be: - ```groovy - script: - def safe_author = author.tokenize(' ').join('-') - """ - quote "$author" > quote-${safe_author}.txt - echo "-${author}" >> quote-${safe_author}.txt - """ - ``` - -You can find a solution to this exercise in `containers/solutions/hello-containers-4.1.nf`. - -### 4.2. Modify your Nextflow pipeline to allow it to execute in `quote` and `sayHello` modes. - -Add some branching logic using to your pipeline to allow it to accept inputs intended for both `quote` and `sayHello`. -Here's an example of how to use an `if` statement in a Nextflow workflow: - -```groovy title="hello-containers.nf" -workflow { - if (params.quote) { - ... - } - else { - ... - } - cowSay(text_ch) -} -``` - -!!! Hint - - You can use `new_ch = processName.out` to assign a name to the output channel of a process. - -You can find a solution to this exercise in `containers/solutions/hello-containers-4.2.nf`. - -### Takeaway - -You know how to use containers in Nextflow to run processes, and how to build some branching logic into your pipelines! - -### What's next? - -Celebrate, take a stretch break and drink some water! - -When you are ready, move on to Part 3 of this training series to learn how to apply what you've learned so far to a more realistic data analysis use case. +[TODO] diff --git a/docs/hello_nextflow/06_hello_config.md b/docs/hello_nextflow/04_hello_config.md similarity index 85% rename from docs/hello_nextflow/06_hello_config.md rename to docs/hello_nextflow/04_hello_config.md index d6f672f9..e3db8786 100644 --- a/docs/hello_nextflow/06_hello_config.md +++ b/docs/hello_nextflow/04_hello_config.md @@ -1,5 +1,7 @@ # Part 5: Hello Config +[TODO] SIMPLIFY / STREAMLINE THIS + This section will explore how to set up and manage the configuration of your Nextflow pipeline so that you'll be able to customize its behavior, adapt it to different environments, and optimize resource usage _without altering a single line of the workflow code itself_. We're going to cover essential components of Nextflow configuration such as config files, profiles, process directives, executors, and parameter files. @@ -7,7 +9,7 @@ By learning to utilize these configuration options effectively, you can enhance --- -## 0. Warmup: Moving to a formal project structure +## 0. Warmup: Moving to a formal project structure [TODO] GET RID OF THIS So far we've been working with a very loose structure, with just one workflow code file and a tiny configuration file that we've mostly ignored, because we were very focused on learning how to implement the workflow itself. However, we're now moving into the phase of this training series that is more focused on code development and maintenance practices. @@ -33,10 +35,10 @@ hello-config └── nextflow.config ``` -- **`main.nf`** is a workflow based on `hello-operators.nf`, the workflow produced by completing Part 4 of this training course; +- **`main.nf`** is a workflow based on `hello-operators.nf`, the workflow produced by completing Part 4 of this training course; -- **`nextflow.config`** is a copy of the original `nextflow.config` file from the `hello-nextflow` directory, one level up (where we've been working so far). - Whenever there is a file named `nextflow.config` in the current directory, Nextflow will automatically load configuration from it. The one we have been using contains the following lines: +- **`nextflow.config`** is a copy of the original `nextflow.config` file from the `hello-nextflow` directory, one level up (where we've been working so far). + Whenever there is a file named `nextflow.config` in the current directory, Nextflow will automatically load configuration from it. The one we have been using contains the following lines: ```console title="nextflow.config" linenums="1" docker.fixOwnership = true @@ -54,8 +56,8 @@ hello-config Anything you put into the `nextflow.config` can be overridden at runtime by providing the relevant process directives or parameters and values on the command line, or by importing another configuration file, according to the order of precedence described [here](https://www.nextflow.io/docs/latest/config.html). -- **`demo-params.json`** is a parameter file intended for supplying parameter values to a workflow. - We will use it in section 5 of this tutorial. +- **`demo-params.json`** is a parameter file intended for supplying parameter values to a workflow. + We will use it in section 5 of this tutorial. The one thing that's missing is a way to point to the original data without making a copy of it or updating the file paths wherever they're specified. The simplest solution is to link to the data location. @@ -117,6 +119,8 @@ Learn how to modify basic configuration properties to adapt to your compute envi ## 1. Determine what software packaging technology to use +[TODO] SIMPLIFY ALL THIS TO JUST USE THE COWSAY PACKAGE + In the very first part of this training course (Part 1: Hello World) we just used locally installed software in our workflow. Then from Part 2 onward, we've been using Docker containers. Now, let's pretend we're working on an HPC cluster and the admin doesn't allow the use of Docker for security reasons. @@ -431,12 +435,12 @@ This runs each step on the same machine that Nextflow is running on. However, for large workloads, you will typically want to use a distributed executor such as an HPC or cloud. Nextflow supports several different distributed executors, including: -- HPC (SLURM, PBS, SGE) -- AWS Batch -- Google Batch -- Azure Batch -- Kubernetes -- GA4GH TES +- HPC (SLURM, PBS, SGE) +- AWS Batch +- Google Batch +- Azure Batch +- Kubernetes +- GA4GH TES The executor is subject to a process directive called `executor`. By default it is set to `local`, so the following configuration is implied: @@ -468,6 +472,8 @@ Basically we are telling Nextflow to generate a Slurm submission script and subm ### 3.2. Launch the workflow to generate the job submission script +[TODO] THIS WAS CONFUSING — EXPLAIN BETTER OR CUT + Let's try running this; even though we know it won't execute (since we don't have Slurm set up in this Gitpod environment) we'll be able to see what the submission script looks like. ```bash @@ -714,6 +720,8 @@ Let's try that out. ### 4.3. Run the workflow to generate a resource utilization report +[TODO] SUMMARIZE AND REFER TO DEEPER DIVE IN DOMAIN-SPECIFIC EXAMPLES (not useful with toy examples) + To have Nextflow generate the report automatically, simply add `-with-report .html` to your command line. ```bash @@ -849,7 +857,7 @@ Configuring the parameters destined for the tools and operations wrapped within --- -## 5. Configure workflow parameters +## 5. Configure workflow parameters [TODO] STREAMLINE THIS (no point ping-ponging so much) So far we've been exploring options for configuring how Nextflow behaves in terms of executing the work. That's all well and good, but how do we manage the parameters that are meant for the workflow itself, and the tools it calls within the processes? @@ -857,99 +865,11 @@ That is also something we should be able to do without editing code files every As it turns out, there's a lot of overlap between this kind of configuration and the infrastructure configuration, starting with the `nextflow.config` file, which can also house default values for command line parameters. -### 5.1. Move the default parameter declarations to the configuration file - -We originally stored all our default parameter values in the workflow script itself, but we can move them out into the `nextflow.config` file if we prefer. - -So let's cut this set of params out of `main.nf`: - -```groovy title="main.nf" linenums="3" -/* - * Pipeline parameters - */ - -// Primary input (file of input files, one per line) -params.reads_bam = "${projectDir}/data/sample_bams.txt" - -// Output directory -params.outdir = 'results_genomics' - -// Accessory files -params.reference = "${projectDir}/data/ref/ref.fasta" -params.reference_index = "${projectDir}/data/ref/ref.fasta.fai" -params.reference_dict = "${projectDir}/data/ref/ref.dict" -params.intervals = "${projectDir}/data/ref/intervals.bed" - -// Base name for final output file -params.cohort_name = "family_trio" -``` - -And let's stick it into the `nextflow.config` file. - -!!!note - - It doesn't really matter where we put it into the file, as long as we keep the params together and avoid mixing them in with the infrastructure configuration, for the sake of readability. - So putting it at the end will do just fine. - -### 5.2. Run the workflow with `-resume` to verify that it still works - -Let's check that we haven't broken anything, and let's include the `-resume` flag. - -```bash -nextflow run main.nf -profile my_laptop -resume -``` - -Not only does everything work, but all of the process calls are recognized as having been run previously. - -```console title="Output" - N E X T F L O W ~ version 24.10.0 - - ┃ Launching `main.nf` [modest_kay] DSL2 - revision: 328869237b - -[d6/353bb0] SAMTOOLS_INDEX (3) [100%] 3 of 3, cached: 3 ✔ -[dc/2a9e3f] GATK_HAPLOTYPECALLER (2) [100%] 3 of 3, cached: 3 ✔ -[fe/a940b2] GATK_JOINTGENOTYPING [100%] 1 of 1, cached: 1 ✔ -``` - -Indeed, having moved the parameter values to a different file changes nothing to the command submission that Nextflow generates. The resumability of the pipeline is preserved. - -### 5.3. Streamline the syntax of the parameter defaults - -Now that our default parameter declarations are in `nextflow.config`, we can switch to using a more structured syntax using a `params` block. That allows us to remove the repeated `params.`. +[TODO] NO JUST STICK IT STRAIGHT INTO A PARAMS JSON -```groovy title="nextflow.config" linenums="35" -/* - * Pipeline parameters - */ +### 5.4. Using a parameter file -params { - // Primary input (file of input files, one per line) - reads_bam = "${projectDir}/data/sample_bams.txt" - - // Output directory - outdir = 'results_genomics' - - // Accessory files - reference = "${projectDir}/data/ref/ref.fasta" - reference_index = "${projectDir}/data/ref/ref.fasta.fai" - reference_dict = "${projectDir}/data/ref/ref.dict" - intervals = "${projectDir}/data/ref/intervals.bed" - - // Base name for final output file - cohort_name = "family_trio" -} -``` - -Feel free to re-run this with the same command as above to verify that it works and still preserves the resumability of the pipeline. - -At this point, you may be wondering how to provide actual data and reference files to run this workflow for real, since what we've put in here is just a tiny test set. - -There are several options. -As we mentioned earlier (see note at the start of this page), you can override the defaults specified in the `nextflow.config` file by providing directives or parameter values on the command line, or by importing other configuration files. - -In this particular case, the best solution is to use a parameter file, which is a JSON file containing key-value pairs for all of the parameters you want to supply values for. - -### 5.4. Using a parameter file to override defaults +[TODO] UPDATE CODE We provide a parameter file in the current directory, called `demo-params.json`, which contains key-value pairs for all of the parameters our workflow expects. The values are the same input files and reference files we've been using so far. @@ -985,63 +905,14 @@ executor > local (7) [35/bda5eb] GATK_JOINTGENOTYPING [100%] 1 of 1 ✔ ``` -However, you may be thinking, well, did we really override the configuration? How would we know, since those were the same files? - -### 5.5. Remove or generalize default values from `nextflow.config` - -Let's strip out all the file paths from the `params` block in `nextflow.config`, replacing them with `null`, and replace the `cohort_name` value with something more generic. - -_Before:_ - -```groovy title="nextflow.config" linenums="39" -params { - // Primary input (file of input files, one per line) - reads_bam = "${projectDir}/data/sample_bams.txt" - - // Output directory - outdir = 'results_genomics' - - // Accessory files - reference = "${projectDir}/data/ref/ref.fasta" - reference_index = "${projectDir}/data/ref/ref.fasta.fai" - reference_dict = "${projectDir}/data/ref/ref.dict" - intervals = "${projectDir}/data/ref/intervals.bed" - - // Base name for final output file - cohort_name = "family_trio" -} -``` - -_After:_ - -```groovy title="nextflow.config" linenums="39" -params { - // Primary input (file of input files, one per line) - reads_bam = null - - // Output directory - outdir = null - - // Accessory files - reference = null - reference_index = null - reference_dict = null - intervals = null - - // Base name for final output file - cohort_name = "my_cohort" -} -``` - -Now, if you run the same command again, it will still work. -So yes, we're definitely able to pull those parameter values from the parameter file. - This is great because, with the parameter file in hand, we'll now be able to provide parameter values at runtime without having to type massive command lines **and** without modifying the workflow nor the default configuration. That being said, it was nice to be able to demo the workflow without having to keep track of filenames and such. Let's see if we can use a profile to replicate that behavior. ### 5.6. Create a demo profile +[TODO] DECIDE IF WE KEEP THIS; IF SO UPDATE TEXT + Yes we can! We just need to retrieve the default parameter declarations as they were written in the original workflow (with the `params.*` syntax) and copy them into a new profile that we'll call `demo`. _Before:_ diff --git a/docs/hello_nextflow/07_hello_modules.md b/docs/hello_nextflow/05_hello_modules.md similarity index 99% rename from docs/hello_nextflow/07_hello_modules.md rename to docs/hello_nextflow/05_hello_modules.md index 5dfdd78a..a9cec9ff 100644 --- a/docs/hello_nextflow/07_hello_modules.md +++ b/docs/hello_nextflow/05_hello_modules.md @@ -15,7 +15,7 @@ This makes the code more shareable, flexible and maintainable. --- -## 0. Warmup +## 0. Warmup [TODO] GET RID OF MOST OF THIS When we started developing our workflow, we put everything in one single code file. In Part 5 (Hello Config), we started turning our one-file workflow into a proper pipeline project. diff --git a/docs/nf4-genomics/01_orientation.md b/docs/nf4-genomics/01_orientation.md new file mode 100644 index 00000000..79d7e0dd --- /dev/null +++ b/docs/nf4-genomics/01_orientation.md @@ -0,0 +1,46 @@ +# Orientation + +The Gitpod environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself. +However, you do need a (free) account to log in, and you should take a few minutes to familiarize yourself with the interface. + +If you have not yet done so, please follow [this link](../../envsetup/) before going any further. + +## Materials provided + +Throughout this training course, we'll be working in the `nf4-genomics/` directory. +This directory contains all the code files, test data and accessory files you will need. + +Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left-hand side of the Gitpod workspace. +Alternatively, you can use the `tree` command. +Throughout the course, we use the output of `tree` to represent directory structure and contents in a readable form, sometimes with minor modifications for clarity. + +Here we generate a table of contents to the second level down: + +```bash +tree . -L 2 +``` + +If you run this inside `nf4-genomics`, you should see the following output: [TODO] + +```console title="Directory contents" +. +``` + +!!!note + + Don't worry if this seems like a lot; we'll go through the relevant pieces at each step of the course. + This is just meant to give you an overview. + +**Here's a summary of what you should know to get started:** + +[TODO] + +!!!tip + + If for whatever reason you move out of this directory, you can always run this command to return to it: + + ```bash + cd /workspace/gitpod/nf4-genomics + ``` + +Now, to begin the course, click on the arrow in the bottom right corner of this page. diff --git a/docs/hello_nextflow/04_hello_genomics.md b/docs/nf4-genomics/02_linear.md similarity index 98% rename from docs/hello_nextflow/04_hello_genomics.md rename to docs/nf4-genomics/02_linear.md index b1b3e522..fe495dfa 100644 --- a/docs/hello_nextflow/04_hello_genomics.md +++ b/docs/nf4-genomics/02_linear.md @@ -40,10 +40,10 @@ So to recap, we're going to develop a workflow that does the following: ### Dataset -- **A reference genome** consisting of a small region of the human chromosome 20 (from hg19/b37) and its accessory files (index and sequence dictionary). -- **Three whole genome sequencing samples** corresponding to a family trio (mother, father and son), which have been subset to a small slice of data on chromosome 20 to keep the file sizes small. - The sequencing data is in (Binary Alignment Map) format, i.e. genome sequencing reads that have already been mapped to the reference genome. -- **A list of genomic intervals**, i.e. coordinates on the genome where our samples have data suitable for calling variants, provided in BED format. +- **A reference genome** consisting of a small region of the human chromosome 20 (from hg19/b37) and its accessory files (index and sequence dictionary). +- **Three whole genome sequencing samples** corresponding to a family trio (mother, father and son), which have been subset to a small slice of data on chromosome 20 to keep the file sizes small. + The sequencing data is in (Binary Alignment Map) format, i.e. genome sequencing reads that have already been mapped to the reference genome. +- **A list of genomic intervals**, i.e. coordinates on the genome where our samples have data suitable for calling variants, provided in BED format. --- diff --git a/docs/hello_nextflow/05_hello_operators.md b/docs/nf4-genomics/03_collect.md similarity index 97% rename from docs/hello_nextflow/05_hello_operators.md rename to docs/nf4-genomics/03_collect.md index 946c6640..35e1e797 100644 --- a/docs/hello_nextflow/05_hello_operators.md +++ b/docs/nf4-genomics/03_collect.md @@ -39,10 +39,10 @@ So to recap, we're going to develop a workflow that does the following: ### Dataset -- **A reference genome** consisting of a small region of the human chromosome 20 (from hg19/b37) and its accessory files (index and sequence dictionary). -- **Three whole genome sequencing samples** corresponding to a family trio (mother, father and son), which have been subset to a small portion on chromosome 20 to keep the file sizes small. - The sequencing data is in [BAM](https://samtools.github.io/hts-specs/SAMv1.pdf) (Binary Alignment Map) format, _i.e._ genome sequencing reads that have already been mapped to the reference genome. -- **A list of genomic intervals**, _i.e._ coordinates on the genome where our samples have data suitable for calling variants, provided in BED format. +- **A reference genome** consisting of a small region of the human chromosome 20 (from hg19/b37) and its accessory files (index and sequence dictionary). +- **Three whole genome sequencing samples** corresponding to a family trio (mother, father and son), which have been subset to a small portion on chromosome 20 to keep the file sizes small. + The sequencing data is in [BAM](https://samtools.github.io/hts-specs/SAMv1.pdf) (Binary Alignment Map) format, _i.e._ genome sequencing reads that have already been mapped to the reference genome. +- **A list of genomic intervals**, _i.e._ coordinates on the genome where our samples have data suitable for calling variants, provided in BED format. --- @@ -257,8 +257,8 @@ It should look very familiar, but feel free to run it if you want to satisfy you We're going to start by making two changes: -- Add the `-ERC GVCF` parameter to the GATK HaplotypeCaller command; -- Update the output file path to use the corresponding `.g.vcf` extension, as per GATK convention. +- Add the `-ERC GVCF` parameter to the GATK HaplotypeCaller command; +- Update the output file path to use the corresponding `.g.vcf` extension, as per GATK convention. Make sure you add a backslash (`\`) at the end of the previous line when you add `-ERC GVCF`. diff --git a/docs/nf4-genomics/04_config.md b/docs/nf4-genomics/04_config.md new file mode 100644 index 00000000..3e4793b8 --- /dev/null +++ b/docs/nf4-genomics/04_config.md @@ -0,0 +1,85 @@ +# Part 3: Resource profiling and optimization + +[TODO] + +--- + +[TODO] + +### 4.3. Run the workflow to generate a resource utilization report + +To have Nextflow generate the report automatically, simply add `-with-report .html` to your command line. + +```bash +nextflow run main.nf -profile my_laptop -with-report report-config-1.html +``` + +The report is an html file, which you can download and open in your browser. You can also right click it in the file explorer on the left and click on `Show preview` in order to view it on Gitpod. + +Take a few minutes to look through the report and see if you can identify some opportunities for adjusting resources. +Make sure to click on the tabs that show the utilization results as a percentage of what was allocated. +There is some [documentation](https://www.nextflow.io/docs/latest/reports.html) describing all the available features. + + + +One observation is that the `GATK_JOINTGENOTYPING` seems to be very hungry for CPU, which makes sense since it performs a lot of complex calculations. +So we could try boosting that and see if it cuts down on runtime. + +However, we seem to have overshot the mark with the memory allocations; all processes are only using a fraction of what we're giving them. +We should dial that back down and save some resources. + +### 4.4. Adjust resource allocations for a specific process + +We can specify resource allocations for a given process using the `withName` process selector. +The syntax looks like this when it's by itself in a process block: + +```groovy title="Syntax" +process { + withName: 'GATK_JOINTGENOTYPING' { + cpus = 4 + } +} +``` + +Let's add that to the existing process block in the `nextflow.config` file. + +```groovy title="nextflow.config" linenums="11" +process { + // defaults for all processes + cpus = 2 + memory = 2.GB + // allocations for a specific process + withName: 'GATK_JOINTGENOTYPING' { + cpus = 4 + } +} +``` + +With that specified, the default settings will apply to all processes **except** the `GATK_JOINTGENOTYPING` process, which is a special snowflake that gets a lot more CPU. +Hopefully that should have an effect. + +### 4.5. Run again with the modified configuration + +Let's run the workflow again with the modified configuration and with the reporting flag turned on, but notice we're giving the report a different name so we can differentiate them. + +```bash +nextflow run main.nf -profile my_laptop -with-report report-config-2.html +``` + +Once again, you probably won't notice a substantial difference in runtime, because this is such a small workload and the tools spend more time in ancillary tasks than in performing the 'real' work. + +However, the second report shows that our resource utilization is more balanced now. + + + +As you can see, this approach is useful when your processes have different resource requirements. It empowers you to right-size the resource allocations you set up for each process based on actual data, not guesswork. + +!!!note + + This is just a tiny taster of what you can do to optimize your use of resources. + Nextflow itself has some really neat [dynamic retry logic](https://training.nextflow.io/basic_training/debugging/#dynamic-resources-allocation) built in to retry jobs that fail due to resource limitations. + Additionally, the Seqera Platform offers AI-driven tooling for optimizing your resource allocations automatically as well. + + We'll cover both of those approaches in an upcoming part of this training course. + +That being said, there may be some constraints on what you can (or must) allocate depending on what computing executor and compute infrastructure you're using. For example, your cluster may require you to stay within certain limits that don't apply when you're running elsewhere. diff --git a/docs/hello_nextflow/08_hello_nf-test.md b/docs/nf4-genomics/05_testing.md similarity index 96% rename from docs/hello_nextflow/08_hello_nf-test.md rename to docs/nf4-genomics/05_testing.md index 1665d181..096a83fa 100644 --- a/docs/hello_nextflow/08_hello_nf-test.md +++ b/docs/nf4-genomics/05_testing.md @@ -1,4 +1,8 @@ -# Part 7: Hello nf-test +# Part N: Hello nf-test + +!!!note + + This training module is under redevelopment. Being able to systematically test that every part of your workflow is doing what it's supposed to do is critical for reproducibility and long-term maintenance. And it's also helpful during the development process! @@ -179,9 +183,9 @@ In plain English, the logic of the test reads as follows: The expected results are formulated as `assert` statements. -- `assert process.success` states that we expect the process to run successfully and complete without any failures. -- `snapshot(process.out).match()` states that we expect the result of the run to be identical to the result obtained in a previous run (if applicable). - We discuss this in more detail later. +- `assert process.success` states that we expect the process to run successfully and complete without any failures. +- `snapshot(process.out).match()` states that we expect the result of the run to be identical to the result obtained in a previous run (if applicable). + We discuss this in more detail later. For most real-world modules (which usually require some kind of input), this is not yet a functional test. We need to add the inputs that will be fed to the process, and any parameters if applicable. @@ -293,9 +297,9 @@ params { Finally, it's time to run our test! Let's break down the syntax. -- The basic command is `nf-test test`. -- To that, we add `--profile docker_on` to specify that we want Nextflow to run the test with Docker enabled. -- Then the test file that we want to run. +- The basic command is `nf-test test`. +- To that, we add `--profile docker_on` to specify that we want Nextflow to run the test with Docker enabled. +- Then the test file that we want to run. !!!note @@ -345,8 +349,8 @@ If we re-run the test, the program will check that the new output matches the ou If, in the course of future development, something in the code changes that causes the output to be different, the test will fail and we will have to determine whether the change is expected or not. -- If it turns out that something in the code broke, we will have to fix it, with the expectation that the fixed code will pass the test. -- If it is an expected change (e.g., the tool has been improved and the results are better) then we will need to update the snapshot to accept the new output as the reference to match, using the parameter `--update-snapshot` when we run the test command. +- If it turns out that something in the code broke, we will have to fix it, with the expectation that the fixed code will pass the test. +- If it is an expected change (e.g., the tool has been improved and the results are better) then we will need to update the snapshot to accept the new output as the reference to match, using the parameter `--update-snapshot` when we run the test command. ### 1.7. Add more tests to `SAMTOOLS_INDEX` @@ -460,8 +464,8 @@ Now that we know how to handle the simplest case, we're going to kick things up As the second step in our pipeline, its input depends on the output of another process. We can deal with this in two ways: -- Manually generate some static test data that is suitable as intermediate input to the process; -- Use a special [setup method](https://www.nf-test.com/docs/testcases/setup/) to handle it dynamically for us. +- Manually generate some static test data that is suitable as intermediate input to the process; +- Use a special [setup method](https://www.nf-test.com/docs/testcases/setup/) to handle it dynamically for us. **Spoiler:** We're going to use the setup method. diff --git a/docs/hello_nextflow/img/gatk-pipeline.png b/docs/nf4-genomics/img/gatk-pipeline.png similarity index 100% rename from docs/hello_nextflow/img/gatk-pipeline.png rename to docs/nf4-genomics/img/gatk-pipeline.png diff --git a/docs/hello_nextflow/img/haplotype-caller.excalidraw.svg b/docs/nf4-genomics/img/haplotype-caller.excalidraw.svg similarity index 100% rename from docs/hello_nextflow/img/haplotype-caller.excalidraw.svg rename to docs/nf4-genomics/img/haplotype-caller.excalidraw.svg diff --git a/docs/hello_nextflow/img/hello-gatk-1.svg b/docs/nf4-genomics/img/hello-gatk-1.svg similarity index 100% rename from docs/hello_nextflow/img/hello-gatk-1.svg rename to docs/nf4-genomics/img/hello-gatk-1.svg diff --git a/docs/hello_nextflow/img/hello-gatk-2.svg b/docs/nf4-genomics/img/hello-gatk-2.svg similarity index 100% rename from docs/hello_nextflow/img/hello-gatk-2.svg rename to docs/nf4-genomics/img/hello-gatk-2.svg diff --git a/docs/hello_nextflow/img/joint-calling.png b/docs/nf4-genomics/img/joint-calling.png similarity index 100% rename from docs/hello_nextflow/img/joint-calling.png rename to docs/nf4-genomics/img/joint-calling.png diff --git a/docs/nf4-genomics/img/memory-after.png b/docs/nf4-genomics/img/memory-after.png new file mode 100644 index 00000000..d61b4a7c Binary files /dev/null and b/docs/nf4-genomics/img/memory-after.png differ diff --git a/docs/nf4-genomics/img/memory-before.png b/docs/nf4-genomics/img/memory-before.png new file mode 100644 index 00000000..ce0f7ac2 Binary files /dev/null and b/docs/nf4-genomics/img/memory-before.png differ diff --git a/docs/hello_nextflow/img/variants.png b/docs/nf4-genomics/img/variants.png similarity index 100% rename from docs/hello_nextflow/img/variants.png rename to docs/nf4-genomics/img/variants.png diff --git a/docs/nf4-genomics/index.md b/docs/nf4-genomics/index.md new file mode 100644 index 00000000..e00b77fa --- /dev/null +++ b/docs/nf4-genomics/index.md @@ -0,0 +1,31 @@ +--- +title: Nextflow for Genomics +hide: + - toc +--- + +# Nextflow for Genomics + +[TODO]] + +Let's get started! + +[![Open in Gitpod](https://img.shields.io/badge/Gitpod-%20Open%20in%20Gitpod-908a85?logo=gitpod)](https://gitpod.io/#https://github.com/nextflow-io/training) + +## Learning objectives + +In this workshop, you will learn [TODO]. + +By the end of this workshop you will be able to: + +[TODO] + +## Audience & prerequisites + +[TODO] + +**Prerequisites** + +- A GitHub account +- Experience with command line + [TODO] diff --git a/docs/run_nextflow/01_orientation.md b/docs/run_nextflow/01_orientation.md new file mode 100644 index 00000000..4353bdd7 --- /dev/null +++ b/docs/run_nextflow/01_orientation.md @@ -0,0 +1,46 @@ +# Orientation + +The Gitpod environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself. +However, you do need a (free) account to log in, and you should take a few minutes to familiarize yourself with the interface. + +If you have not yet done so, please follow [this link](../../envsetup/) before going any further. + +## Materials provided + +Throughout this training course, we'll be working in the `run-nextflow/` directory. +This directory contains all the code files, test data and accessory files you will need. + +Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left-hand side of the Gitpod workspace. +Alternatively, you can use the `tree` command. +Throughout the course, we use the output of `tree` to represent directory structure and contents in a readable form, sometimes with minor modifications for clarity. + +Here we generate a table of contents to the second level down: + +```bash +tree . -L 2 +``` + +If you run this inside `run-nextflow`, you should see the following output: [TODO] + +```console title="Directory contents" +. +``` + +!!!note + + Don't worry if this seems like a lot; we'll go through the relevant pieces at each step of the course. + This is just meant to give you an overview. + +**Here's a summary of what you should know to get started:** + +[TODO] + +!!!tip + + If for whatever reason you move out of this directory, you can always run this command to return to it: + + ```bash + cd /workspace/gitpod/run-nextflow + ``` + +Now, to begin the course, click on the arrow in the bottom right corner of this page. diff --git a/docs/run_nextflow/02_run_basics.md b/docs/run_nextflow/02_run_basics.md new file mode 100644 index 00000000..f926e96f --- /dev/null +++ b/docs/run_nextflow/02_run_basics.md @@ -0,0 +1,10 @@ +# Part 1: Run Basics + +[TODO] + +Should cover: + +- basic project structure (main.nf, modules, nextflow.config) +- run from CLI (re-use from Hello-World) +- basic config elements (refer to hello-config) including profiles +- running resource profiling and adapting the config diff --git a/docs/hello_nextflow/09_hello_nf-core.md b/docs/run_nextflow/03_run_nf-core.md similarity index 97% rename from docs/hello_nextflow/09_hello_nf-core.md rename to docs/run_nextflow/03_run_nf-core.md index 0f8b0186..3d26c019 100644 --- a/docs/hello_nextflow/09_hello_nf-core.md +++ b/docs/run_nextflow/03_run_nf-core.md @@ -1,4 +1,4 @@ -# Part 8: Hello nf-core +# Part 3: Run nf-core nf-core is a community effort to develop and maintain a curated set of analysis pipelines built using Nextflow. @@ -18,12 +18,12 @@ The nf-core collection currently offers [over 100 pipelines](https://nf-co.re/pi Each released pipeline has a dedicated page that includes 6 documentation sections: -- **Introduction:** An introduction and overview of the pipeline -- **Usage:** Descriptions of how to execute the pipeline -- **Parameters:** Grouped pipeline parameters with descriptions -- **Output:** Descriptions and examples of the expected output files -- **Results:** Example output files generated from the full test dataset -- **Releases & Statistics:** Pipeline version history and statistics +- **Introduction:** An introduction and overview of the pipeline +- **Usage:** Descriptions of how to execute the pipeline +- **Parameters:** Grouped pipeline parameters with descriptions +- **Output:** Descriptions and examples of the expected output files +- **Results:** Example output files generated from the full test dataset +- **Releases & Statistics:** Pipeline version history and statistics You should read the pipeline documentation carefully to understand what a given pipeline does and how it can be configured before attempting to run it. @@ -384,8 +384,8 @@ The nf-core pipeline template has a `main.nf` script that calls `myfirstpipeline Instead of having one large monolithic pipeline script, it's broken up into smaller script components, namely, modules and subworkflows: -- **Modules:** Wrappers around a single process -- **Subworkflows:** Two or more modules that are packaged together as a mini workflow +- **Modules:** Wrappers around a single process +- **Subworkflows:** Two or more modules that are packaged together as a mini workflow
--8<-- "docs/hello_nextflow/img/nested.excalidraw.svg" @@ -442,10 +442,10 @@ In the template, the `nextflow.config` file is a central configuration file and There are several configuration files that are stored in the `conf` folder and are added to the configuration by default or optionally as profiles: -- `base.config`: A 'blank slate' config file, appropriate for general use on most high-performance computing environments. This defines broad bins of resource usage, for example, which are convenient to apply to modules. -- `modules.config`: Additional module directives and arguments. -- `test.config`: A profile to run the pipeline with minimal test data. -- `test_full.config`: A profile to run the pipeline with a full-sized test dataset. +- `base.config`: A 'blank slate' config file, appropriate for general use on most high-performance computing environments. This defines broad bins of resource usage, for example, which are convenient to apply to modules. +- `modules.config`: Additional module directives and arguments. +- `test.config`: A profile to run the pipeline with minimal test data. +- `test_full.config`: A profile to run the pipeline with a full-sized test dataset. #### `nextflow_schema.json` @@ -642,8 +642,8 @@ Using this module information you can work out what inputs are required for the 1. `tuple val(meta), path(reads)` - - A tuple with a meta _map_ and a list of FASTQ _files_ - - The channel `ch_samplesheet` used by the `FASTQC` process can be used as the reads input. + - A tuple with a meta _map_ and a list of FASTQ _files_ + - The channel `ch_samplesheet` used by the `FASTQC` process can be used as the reads input. Only one input channel is required, and it already exists, so it can be added to your `firstpipeline.nf` file without any additional channel creation or modifications. @@ -1009,9 +1009,9 @@ In the next step we will take a look how we can add a new key to the `meta` map nf-core pipelines typically use samplesheets as inputs to the pipelines. This allows us to: -- validate each entry and print specific error messages. -- attach information to each input file. -- track which datasets are processed. +- validate each entry and print specific error messages. +- attach information to each input file. +- track which datasets are processed. Samplesheets are comma-separated text files with a header row specifying the column names, followed by one entry per row. For example, the samplesheet that we have been using during this teaching module looks like this: diff --git a/docs/hello_nextflow/10_hello_seqera.md b/docs/run_nextflow/04_run_seqera.md similarity index 100% rename from docs/hello_nextflow/10_hello_seqera.md rename to docs/run_nextflow/04_run_seqera.md diff --git a/docs/run_nextflow/img/cpu-after.png b/docs/run_nextflow/img/cpu-after.png new file mode 100644 index 00000000..f173ef02 Binary files /dev/null and b/docs/run_nextflow/img/cpu-after.png differ diff --git a/docs/run_nextflow/img/cpu-before.png b/docs/run_nextflow/img/cpu-before.png new file mode 100644 index 00000000..e0b8c76c Binary files /dev/null and b/docs/run_nextflow/img/cpu-before.png differ diff --git a/docs/run_nextflow/img/memory-after.png b/docs/run_nextflow/img/memory-after.png new file mode 100644 index 00000000..d61b4a7c Binary files /dev/null and b/docs/run_nextflow/img/memory-after.png differ diff --git a/docs/run_nextflow/img/memory-before.png b/docs/run_nextflow/img/memory-before.png new file mode 100644 index 00000000..ce0f7ac2 Binary files /dev/null and b/docs/run_nextflow/img/memory-before.png differ diff --git a/docs/hello_nextflow/img/nested.excalidraw.svg b/docs/run_nextflow/img/nested.excalidraw.svg similarity index 100% rename from docs/hello_nextflow/img/nested.excalidraw.svg rename to docs/run_nextflow/img/nested.excalidraw.svg diff --git a/docs/hello_nextflow/img/nf-core-modules.png b/docs/run_nextflow/img/nf-core-modules.png similarity index 100% rename from docs/hello_nextflow/img/nf-core-modules.png rename to docs/run_nextflow/img/nf-core-modules.png diff --git a/docs/hello_nextflow/img/pipeline.excalidraw.svg b/docs/run_nextflow/img/pipeline.excalidraw.svg similarity index 100% rename from docs/hello_nextflow/img/pipeline.excalidraw.svg rename to docs/run_nextflow/img/pipeline.excalidraw.svg diff --git a/docs/hello_nextflow/img/pipeline_schema.png b/docs/run_nextflow/img/pipeline_schema.png similarity index 100% rename from docs/hello_nextflow/img/pipeline_schema.png rename to docs/run_nextflow/img/pipeline_schema.png diff --git a/docs/run_nextflow/img/report_cover.png b/docs/run_nextflow/img/report_cover.png new file mode 100644 index 00000000..9feb1792 Binary files /dev/null and b/docs/run_nextflow/img/report_cover.png differ diff --git a/docs/hello_nextflow/img/seqera-containers-1.png b/docs/run_nextflow/img/seqera-containers-1.png similarity index 100% rename from docs/hello_nextflow/img/seqera-containers-1.png rename to docs/run_nextflow/img/seqera-containers-1.png diff --git a/docs/hello_nextflow/img/seqera-containers-2.png b/docs/run_nextflow/img/seqera-containers-2.png similarity index 100% rename from docs/hello_nextflow/img/seqera-containers-2.png rename to docs/run_nextflow/img/seqera-containers-2.png diff --git a/docs/run_nextflow/index.md b/docs/run_nextflow/index.md new file mode 100644 index 00000000..e6cd6ad4 --- /dev/null +++ b/docs/run_nextflow/index.md @@ -0,0 +1,39 @@ +--- +title: Run Nextflow +hide: + - toc +--- + +# Run Nextflow + +Hello! You are now on the path to running reproducible and scalable scientific workflows using Nextflow. + +[TODO] NEED TO DISTINGUISH CLEARLY FROM HELLO NEXTFLOW + +The rise of big data has made it increasingly necessary to be able to analyze and perform experiments on large datasets in a portable and reproducible manner. Parallelization and distributed computing are the best ways to tackle this challenge, but the tools commonly available to computational scientists often lack good support for these techniques, or they provide a model that fits poorly with the needs of computational scientists. Nextflow was particularly created to address these challenges. + +During this training, you will be introduced to Nextflow in a series of complementary hands-on workshops. + +Let's get started! + +[![Open in Gitpod](https://img.shields.io/badge/Gitpod-%20Open%20in%20Gitpod-908a85?logo=gitpod)](https://gitpod.io/#https://github.com/nextflow-io/training) + +## Learning objectives + +In this workshop, you will learn foundational concepts for building pipelines. + +By the end of this workshop you will be able to: + +- Launch a Nextflow workflow locally +- Find and interpret outputs (results) and log files generated by Nextflow +- Troubleshoot basic issues +- [TODO] + +## Audience & prerequisites + +This is a workshop for those who are completely new to Nextflow. Some basic familiarity with the command line, and common file formats is assumed. + +**Prerequisites** + +- A GitHub account +- Experience with command line diff --git a/docs/hello_nextflow/seqera/01_run_with_cli.md b/docs/run_nextflow/seqera/01_run_with_cli.md similarity index 88% rename from docs/hello_nextflow/seqera/01_run_with_cli.md rename to docs/run_nextflow/seqera/01_run_with_cli.md index f2e4349c..d9a321db 100644 --- a/docs/hello_nextflow/seqera/01_run_with_cli.md +++ b/docs/run_nextflow/seqera/01_run_with_cli.md @@ -147,9 +147,9 @@ If you ran your pipeline from the `hello_nextflow` directory, you'll see somethi Notice that configuration for our pipeline run is being run pulled from three separate files: -- `/home/gitpod/.nextflow/config` - This is the global configuration file we just added. -- `/home/gitpod/.nextflow/assets/nextflow-io/hello/nextflow.config` - This is the `nextflow.config` file from the `nextflow-io/hello` repository. -- `/workspace/gitpod/nf-training/hello-nextflow/nextflow.config` - This is the `nextflow.config` file from our current working directory. +- `/home/gitpod/.nextflow/config` - This is the global configuration file we just added. +- `/home/gitpod/.nextflow/assets/nextflow-io/hello/nextflow.config` - This is the `nextflow.config` file from the `nextflow-io/hello` repository. +- `/workspace/gitpod/nf-training/hello-nextflow/nextflow.config` - This is the `nextflow.config` file from our current working directory. Nextflow resolves these configurations at runtime with a [specific order of precedence](https://www.nextflow.io/docs/latest/config.html#configuration-file). The general rule, however, is that more specific configurations override less specific ones, and config/params specified on the CLI will override defaults in the config files. @@ -160,10 +160,10 @@ Helpfully, Seqera Platform shows us the final output of this configuration resol You have learned how to: -- Set up your Seqera Platform token by exporting it to your environment. -- Run Nextflow CLI with Seqera Platform visualizing and capturing logs. -- Set up Seqera Platform logging by default. -- Use Seqera Platform to explore the resolved configuration of a Nextflow pipeline. +- Set up your Seqera Platform token by exporting it to your environment. +- Run Nextflow CLI with Seqera Platform visualizing and capturing logs. +- Set up Seqera Platform logging by default. +- Use Seqera Platform to explore the resolved configuration of a Nextflow pipeline. ### What's next? diff --git a/docs/hello_nextflow/seqera/02_run_with_launchpad.md b/docs/run_nextflow/seqera/02_run_with_launchpad.md similarity index 94% rename from docs/hello_nextflow/seqera/02_run_with_launchpad.md rename to docs/run_nextflow/seqera/02_run_with_launchpad.md index 4b2f41fb..c3080d72 100644 --- a/docs/hello_nextflow/seqera/02_run_with_launchpad.md +++ b/docs/run_nextflow/seqera/02_run_with_launchpad.md @@ -75,10 +75,10 @@ Click on the task to see the task details: You have learned how to: -- Switch between organizations and workspaces in Seqera Platform. -- Launch a Nextflow pipeline that ran in the cloud using Seqera Platform. -- Monitor the progress of the pipeline run. -- Inspect the details of a task that was executed as part of the pipeline. +- Switch between organizations and workspaces in Seqera Platform. +- Launch a Nextflow pipeline that ran in the cloud using Seqera Platform. +- Monitor the progress of the pipeline run. +- Inspect the details of a task that was executed as part of the pipeline. ### Next steps diff --git a/docs/hello_nextflow/seqera/img/compute_env_platforms.png b/docs/run_nextflow/seqera/img/compute_env_platforms.png similarity index 100% rename from docs/hello_nextflow/seqera/img/compute_env_platforms.png rename to docs/run_nextflow/seqera/img/compute_env_platforms.png diff --git a/docs/hello_nextflow/seqera/img/launchpad.gif b/docs/run_nextflow/seqera/img/launchpad.gif similarity index 100% rename from docs/hello_nextflow/seqera/img/launchpad.gif rename to docs/run_nextflow/seqera/img/launchpad.gif diff --git a/docs/hello_nextflow/seqera/img/resolved_configuration.png b/docs/run_nextflow/seqera/img/resolved_configuration.png similarity index 100% rename from docs/hello_nextflow/seqera/img/resolved_configuration.png rename to docs/run_nextflow/seqera/img/resolved_configuration.png diff --git a/docs/hello_nextflow/seqera/img/run_with_tower.png b/docs/run_nextflow/seqera/img/run_with_tower.png similarity index 100% rename from docs/hello_nextflow/seqera/img/run_with_tower.png rename to docs/run_nextflow/seqera/img/run_with_tower.png diff --git a/docs/hello_nextflow/seqera/img/running_pipeline.png b/docs/run_nextflow/seqera/img/running_pipeline.png similarity index 100% rename from docs/hello_nextflow/seqera/img/running_pipeline.png rename to docs/run_nextflow/seqera/img/running_pipeline.png diff --git a/docs/hello_nextflow/seqera/img/task_details.png b/docs/run_nextflow/seqera/img/task_details.png similarity index 100% rename from docs/hello_nextflow/seqera/img/task_details.png rename to docs/run_nextflow/seqera/img/task_details.png diff --git a/docs/hello_nextflow/seqera/img/usage_create_token.png b/docs/run_nextflow/seqera/img/usage_create_token.png similarity index 100% rename from docs/hello_nextflow/seqera/img/usage_create_token.png rename to docs/run_nextflow/seqera/img/usage_create_token.png diff --git a/docs/hello_nextflow/seqera/img/usage_name_token.png b/docs/run_nextflow/seqera/img/usage_name_token.png similarity index 100% rename from docs/hello_nextflow/seqera/img/usage_name_token.png rename to docs/run_nextflow/seqera/img/usage_name_token.png diff --git a/docs/hello_nextflow/seqera/img/usage_token.png b/docs/run_nextflow/seqera/img/usage_token.png similarity index 100% rename from docs/hello_nextflow/seqera/img/usage_token.png rename to docs/run_nextflow/seqera/img/usage_token.png diff --git a/docs/side_quests/containers.md b/docs/side_quests/containers.md new file mode 100644 index 00000000..d015abf2 --- /dev/null +++ b/docs/side_quests/containers.md @@ -0,0 +1,189 @@ +# Part 1: More Containers + +[TODO] + +--- + +## 1. How to find or make container images + +Some software developers provide container images for their software that are available on container registries like Docker Hub, but many do not. +In this optional section, we'll show you to two ways to get a container image for tools you want to use in your Nextflow pipelines: using Seqera Containers and building the container image yourself. + +You'll be getting/building a container image for the `quote` pip package, which will be used in the exercise at the end of this section. + +### 1.1. Get a container image from Seqera Containers + +Seqera Containers is a free service that builds container images for pip and conda (including bioconda) installable tools. +Navigate to [Seqera Containers](https://www.seqera.io/containers/) and search for the `quote` pip package. + +![Seqera Containers](img/seqera-containers-1.png) + +Click on "+Add" and then "Get Container" to request a container image for the `quote` pip package. + +![Seqera Containers](img/seqera-containers-2.png) + +If this is the first time a community container has been built for this version of the package, it may take a few minutes to complete. +Click to copy the URI (e.g. `community.wave.seqera.io/library/pip_quote:ae07804021465ee9`) of the container image that was created for you. + +You can now use the container image to run the `quote` command and get a random saying from Grace Hopper. + +```bash +docker run --rm community.wave.seqera.io/library/pip_quote:ae07804021465ee9 quote "Grace Hopper" +``` + +Output: + +```console title="Output" +Humans are allergic to change. They love to say, 'We've always done it +this way.' I try to fight that. That's why I have a clock on my wall +that runs counter-clockwise. +``` + +### 1.2. Build the container image yourself + +Let's use some build details from the Seqera Containers website to build the container image for the `quote` pip package ourselves. +Return to the Seqera Containers website and click on the "Build Details" button. + +The first item we'll look at is the `Dockerfile`, a type of script file that contains all the commands needed to build the container image. +We've added some explanatory comments to the Dockerfile below to help you understand what each part does. + +```Dockerfile title="Dockerfile" +# Start from the micromamba base docker image +FROM mambaorg/micromamba:1.5.10-noble +# Copy the conda.yml file into the container +COPY --chown=$MAMBA_USER:$MAMBA_USER conda.yml /tmp/conda.yml +# Install various utilities for Nextflow to use and the packages in the conda.yml file +RUN micromamba install -y -n base -f /tmp/conda.yml \ + && micromamba install -y -n base conda-forge::procps-ng \ + && micromamba env export --name base --explicit > environment.lock \ + && echo ">> CONDA_LOCK_START" \ + && cat environment.lock \ + && echo "<< CONDA_LOCK_END" \ + && micromamba clean -a -y +# Run the container as the root user +USER root +# Set the PATH environment variable to include the micromamba installation directory +ENV PATH="$MAMBA_ROOT_PREFIX/bin:$PATH" +``` + +The second item we'll look at is the `conda.yml` file, which contains the list of packages that need to be installed in the container image. + +```conda.yml title="conda.yml" +channels: +- conda-forge +- bioconda +dependencies: +- pip +- pip: + - quote==3.0.0 # +``` + +Copy the contents of these files into the stubs located in the `containers/build` directory, then run the following command to build the container image yourself. + +!!! Note + + We use the `-t quote:latest` flag to tag the container image with the name `quote` and the tag `latest`. + We will be able to use this tag to refer to the container image when running it on this system. + +```bash +docker build -t quote:latest containers/build +``` + +After it has finished building, you can run the container image you just built. + +```bash +docker run --rm quote:latest quote "Margaret Oakley Dayhoff" +``` + +### Takeaway + +You've learned two different ways to get a container image for a tool you want to use in your Nextflow pipelines: using Seqera Containers and building the container image yourself. + +### What's next? + +You have everything you need to continue to the [next chapter](./04_hello_genomics.md) of this training series. +You can also continue on with an optional exercise to fetch quotes on computer/biology pioneers using the `quote` container and output them using the `cowsay` container. + +--- + +## 2. Make the cow quote famous scientists + +This section contains some stretch exercises, to practice what you've learned so far. +Doing these exercises is _not required_ to understand later parts of the training, but provide a fun way to reinforce your learnings by figuring out how to make the cow quote famous scientists. + +```console title="cowsay-output-Grace-Hopper.txt" + _________________________________________________ + / \ +| Humans are allergic to change. They love to | +| say, 'We've always done it this way.' I try to fi | +| ght that. That's why I have a clock on my wall th | +| at runs counter-clockwise. | +| -Grace Hopper | + \ / + ================================================= + \ + \ + ^__^ + (oo)\_______ + (__)\ )\/\ + ||----w | + || || +``` + +### 2.1. Modify the `hello-containers.nf` script to use a getQuote process + +We have a list of computer and biology pioneers in the `containers/data/pioneers.csv` file. +At a high level, to complete this exercise you will need to: + +- Modify the default `params.input_file` to point to the `pioneers.csv` file. +- Create a `getQuote` process that uses the `quote` container to fetch a quote for each input. +- Connect the output of the `getQuote` process to the `cowsay` process to display the quote. + +For the `quote` container image, you can either use the one you built yourself in the previous stretch exercise or use the one you got from Seqera Containers . + +!!! Hint + + A good choice for the `script` block of your getQuote process might be: + ```groovy + script: + def safe_author = author.tokenize(' ').join('-') + """ + quote "$author" > quote-${safe_author}.txt + echo "-${author}" >> quote-${safe_author}.txt + """ + ``` + +You can find a solution to this exercise in `containers/solutions/hello-containers-4.1.nf`. + +### 2.2. Modify your Nextflow pipeline to allow it to execute in `quote` and `sayHello` modes. + +Add some branching logic using to your pipeline to allow it to accept inputs intended for both `quote` and `sayHello`. +Here's an example of how to use an `if` statement in a Nextflow workflow: + +```groovy title="hello-containers.nf" +workflow { + if (params.quote) { + ... + } + else { + ... + } + cowSay(text_ch) +} +``` + +!!! Hint + + You can use `new_ch = processName.out` to assign a name to the output channel of a process. + +You can find a solution to this exercise in `containers/solutions/hello-containers-4.2.nf`. + +### Takeaway + +You know how to use containers in Nextflow to run processes, and how to build some branching logic into your pipelines! + +### What's next? + +Celebrate, take a stretch break and drink some water! + +When you are ready, move on to Part 3 of this training series to learn how to apply what you've learned so far to a more realistic data analysis use case. diff --git a/docs/side_quests/if_else.md b/docs/side_quests/if_else.md new file mode 100644 index 00000000..49f22134 --- /dev/null +++ b/docs/side_quests/if_else.md @@ -0,0 +1,87 @@ +# Part 2: If - Else + +[TODO] + +--- + +## 1. Make the cow quote famous scientists + +This section contains some stretch exercises, to practice what you've learned so far. +Doing these exercises is _not required_ to understand later parts of the training, but provide a fun way to reinforce your learnings by figuring out how to make the cow quote famous scientists. + +```console title="cowsay-output-Grace-Hopper.txt" + _________________________________________________ + / \ +| Humans are allergic to change. They love to | +| say, 'We've always done it this way.' I try to fi | +| ght that. That's why I have a clock on my wall th | +| at runs counter-clockwise. | +| -Grace Hopper | + \ / + ================================================= + \ + \ + ^__^ + (oo)\_______ + (__)\ )\/\ + ||----w | + || || +``` + +### 1.1. Modify the `hello-containers.nf` script to use a getQuote process + +We have a list of computer and biology pioneers in the `containers/data/pioneers.csv` file. +At a high level, to complete this exercise you will need to: + +- Modify the default `params.input_file` to point to the `pioneers.csv` file. +- Create a `getQuote` process that uses the `quote` container to fetch a quote for each input. +- Connect the output of the `getQuote` process to the `cowsay` process to display the quote. + +For the `quote` container image, you can either use the one you built yourself in the previous stretch exercise or use the one you got from Seqera Containers . + +!!! Hint + + A good choice for the `script` block of your getQuote process might be: + ```groovy + script: + def safe_author = author.tokenize(' ').join('-') + """ + quote "$author" > quote-${safe_author}.txt + echo "-${author}" >> quote-${safe_author}.txt + """ + ``` + +You can find a solution to this exercise in `containers/solutions/hello-containers-4.1.nf`. + +### 1.2. Modify your Nextflow pipeline to allow it to execute in `quote` and `sayHello` modes. + +Add some branching logic using to your pipeline to allow it to accept inputs intended for both `quote` and `sayHello`. +Here's an example of how to use an `if` statement in a Nextflow workflow: + +```groovy title="hello-containers.nf" +workflow { + if (params.quote) { + ... + } + else { + ... + } + cowSay(text_ch) +} +``` + +!!! Hint + + You can use `new_ch = processName.out` to assign a name to the output channel of a process. + +You can find a solution to this exercise in `containers/solutions/hello-containers-4.2.nf`. + +### Takeaway + +You know how to use containers in Nextflow to run processes, and how to build some branching logic into your pipelines! + +### What's next? + +Celebrate, take a stretch break and drink some water! + +When you are ready, move on to Part 3 of this training series to learn how to apply what you've learned so far to a more realistic data analysis use case. diff --git a/docs/side_quests/index.md b/docs/side_quests/index.md new file mode 100644 index 00000000..e5178ad3 --- /dev/null +++ b/docs/side_quests/index.md @@ -0,0 +1,27 @@ +--- +title: Side Quests +hide: + - toc +--- + +# Side Quests + +[TODO] This is a collection of standalone training modules that go deeper into specific topics. You can go through them in any order. + +Let's get started! + +[![Open in Gitpod](https://img.shields.io/badge/Gitpod-%20Open%20in%20Gitpod-908a85?logo=gitpod)](https://gitpod.io/#https://github.com/nextflow-io/training) + +## Learning objectives + +The learning objectives are specific to each section. [TODO] + +## Audience & prerequisites + +[TODO] + +**Prerequisites** + +- A GitHub account +- Experience with command line + [TODO] diff --git a/docs/side_quests/orientation.md b/docs/side_quests/orientation.md new file mode 100644 index 00000000..4a204784 --- /dev/null +++ b/docs/side_quests/orientation.md @@ -0,0 +1,46 @@ +# Orientation + +The Gitpod environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself. +However, you do need a (free) account to log in, and you should take a few minutes to familiarize yourself with the interface. + +If you have not yet done so, please follow [this link](../../envsetup/) before going any further. + +## Materials provided + +Throughout this training course, we'll be working in the `tapas/` directory. +This directory contains all the code files, test data and accessory files you will need. + +Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left-hand side of the Gitpod workspace. +Alternatively, you can use the `tree` command. +Throughout the course, we use the output of `tree` to represent directory structure and contents in a readable form, sometimes with minor modifications for clarity. + +Here we generate a table of contents to the second level down: + +```bash +tree . -L 2 +``` + +If you run this inside `tapas`, you should see the following output: [TODO] + +```console title="Directory contents" +. +``` + +!!!note + + Don't worry if this seems like a lot; we'll go through the relevant pieces at each step of the course. + This is just meant to give you an overview. + +**Here's a summary of what you should know to get started:** + +[TODO] + +!!!tip + + If for whatever reason you move out of this directory, you can always run this command to return to it: + + ```bash + cd /workspace/gitpod/tapas + ``` + +Now, to begin the course, click on the arrow in the bottom right corner of this page. diff --git a/hello-nextflow/containers/data/greetings.csv b/hello-nextflow/greetings.csv similarity index 100% rename from hello-nextflow/containers/data/greetings.csv rename to hello-nextflow/greetings.csv diff --git a/mkdocs.yml b/mkdocs.yml index 548855b0..fc3f82dd 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -14,13 +14,24 @@ nav: - hello_nextflow/01_orientation.md - hello_nextflow/02_hello_world.md - hello_nextflow/03_hello_containers.md - - hello_nextflow/04_hello_genomics.md - - hello_nextflow/05_hello_operators.md - - hello_nextflow/06_hello_config.md - - hello_nextflow/07_hello_modules.md - - hello_nextflow/08_hello_nf-test.md - - hello_nextflow/09_hello_nf-core.md - - hello_nextflow/10_hello_seqera.md + - hello_nextflow/04_hello_config.md + - hello_nextflow/05_hello_modules.md + - Run Nextflow: + - run_nextflow/index.md + - run_nextflow/01_orientation.md + - run_nextflow/02_run_basics.md + - run_nextflow/03_run_nf-core.md + - run_nextflow/04_run_seqera.md + - Side Quests: + - side_quests/index.md + - side_quests/orientation.md + - side_quests/containers.md + - side_quests/if_else.md + - Nextflow for Genomics: + - nf4_genomics/index.md + - nf4_genomics/01_orientation.md + - nf4_genomics/02_linear.md + - nf4_genomics/03_collect.md - Fundamentals Training: - basic_training/index.md - basic_training/orientation.md @@ -167,11 +178,9 @@ plugins: restart_increment_after: - envsetup/01_setup.md - hello_nextflow/01_orientation.md - - basic_training/orientation.md - - advanced/orientation.md - - nf_customize/01_orientation.md - - nf_develop/1_01_orientation.md - - troubleshoot/01_orientation.md + - run_nextflow/01_orientation.md + - side_quests/01_orientation.md + - big_nextflow/01_orientation.md exclude: - index.md @@ -186,6 +195,11 @@ plugins: - hands_on/index.md - hands_on/solutions/*md - hello_nextflow/*.md + - basic_training/orientation.md + - advanced/orientation.md + - nf_customize/01_orientation.md + - nf_develop/1_01_orientation.md + - troubleshoot/01_orientation.md - i18n: docs_structure: suffix fallback_to_default: true diff --git a/hello-nextflow/data/bam/reads_father.bam b/nf4-genomics/data/bam/reads_father.bam similarity index 100% rename from hello-nextflow/data/bam/reads_father.bam rename to nf4-genomics/data/bam/reads_father.bam diff --git a/hello-nextflow/data/bam/reads_mother.bam b/nf4-genomics/data/bam/reads_mother.bam similarity index 100% rename from hello-nextflow/data/bam/reads_mother.bam rename to nf4-genomics/data/bam/reads_mother.bam diff --git a/hello-nextflow/data/bam/reads_son.bam b/nf4-genomics/data/bam/reads_son.bam similarity index 100% rename from hello-nextflow/data/bam/reads_son.bam rename to nf4-genomics/data/bam/reads_son.bam diff --git a/hello-nextflow/data/ref/intervals.bed b/nf4-genomics/data/ref/intervals.bed similarity index 100% rename from hello-nextflow/data/ref/intervals.bed rename to nf4-genomics/data/ref/intervals.bed diff --git a/hello-nextflow/data/ref/ref.dict b/nf4-genomics/data/ref/ref.dict similarity index 100% rename from hello-nextflow/data/ref/ref.dict rename to nf4-genomics/data/ref/ref.dict diff --git a/hello-nextflow/data/ref/ref.fasta b/nf4-genomics/data/ref/ref.fasta similarity index 100% rename from hello-nextflow/data/ref/ref.fasta rename to nf4-genomics/data/ref/ref.fasta diff --git a/hello-nextflow/data/ref/ref.fasta.fai b/nf4-genomics/data/ref/ref.fasta.fai similarity index 100% rename from hello-nextflow/data/ref/ref.fasta.fai rename to nf4-genomics/data/ref/ref.fasta.fai diff --git a/hello-nextflow/data/sample_bams.txt b/nf4-genomics/data/sample_bams.txt similarity index 100% rename from hello-nextflow/data/sample_bams.txt rename to nf4-genomics/data/sample_bams.txt diff --git a/hello-nextflow/data/samplesheet.csv b/nf4-genomics/data/samplesheet.csv similarity index 100% rename from hello-nextflow/data/samplesheet.csv rename to nf4-genomics/data/samplesheet.csv diff --git a/hello-nextflow/hello-genomics.nf b/nf4-genomics/hello-genomics.nf similarity index 100% rename from hello-nextflow/hello-genomics.nf rename to nf4-genomics/hello-genomics.nf diff --git a/hello-nextflow/hello-operators.nf b/nf4-genomics/hello-operators.nf similarity index 100% rename from hello-nextflow/hello-operators.nf rename to nf4-genomics/hello-operators.nf diff --git a/hello-nextflow/hello-nf-core/data/sequencer_samplesheet.csv b/run-nextflow/hello-nf-core/data/sequencer_samplesheet.csv similarity index 91% rename from hello-nextflow/hello-nf-core/data/sequencer_samplesheet.csv rename to run-nextflow/hello-nf-core/data/sequencer_samplesheet.csv index 3d02bafe..f2c3b780 100644 --- a/hello-nextflow/hello-nf-core/data/sequencer_samplesheet.csv +++ b/run-nextflow/hello-nf-core/data/sequencer_samplesheet.csv @@ -2,4 +2,4 @@ sample,sequencer,fastq_1,fastq_2 SAMPLE1_PE,sequencer1,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample1_R1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample1_R2.fastq.gz SAMPLE2_PE,sequencer2,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R2.fastq.gz SAMPLE3_SE,sequencer3,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample1_R1.fastq.gz, -SAMPLE3_SE,sequencer3,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R1.fastq.gz, \ No newline at end of file +SAMPLE3_SE,sequencer3,https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/illumina/amplicon/sample2_R1.fastq.gz, diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.editorconfig b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.editorconfig similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.editorconfig rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.editorconfig diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.nf-core.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.nf-core.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.nf-core.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.nf-core.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.pre-commit-config.yaml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.pre-commit-config.yaml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.pre-commit-config.yaml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.pre-commit-config.yaml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.prettierignore b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.prettierignore similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.prettierignore rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.prettierignore diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.prettierrc.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.prettierrc.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.prettierrc.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/.prettierrc.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/LICENSE b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/LICENSE similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/LICENSE rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/LICENSE diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/README.md b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/README.md similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/README.md rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/README.md diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/email_template.html b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/email_template.html similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/email_template.html rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/email_template.html diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/email_template.txt b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/email_template.txt similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/email_template.txt rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/email_template.txt diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/multiqc_config.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/multiqc_config.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/multiqc_config.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/multiqc_config.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/samplesheet.csv b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/samplesheet.csv similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/samplesheet.csv rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/samplesheet.csv diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/schema_input.json b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/schema_input.json similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/schema_input.json rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/schema_input.json diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/sendmail_template.txt b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/sendmail_template.txt similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/sendmail_template.txt rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/assets/sendmail_template.txt diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/base.config b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/base.config similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/base.config rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/base.config diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/modules.config b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/modules.config similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/modules.config rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/modules.config diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test.config b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test.config similarity index 99% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test.config rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test.config index 50ae17ee..474ef319 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test.config +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test.config @@ -27,5 +27,5 @@ params { // TODO nf-core: Give any required params for the test so that command line flags are not needed input = params.pipelines_testdata_base_path + 'viralrecon/samplesheet/samplesheet_test_illumina_amplicon.csv' - + } diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test_full.config b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test_full.config similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test_full.config rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/conf/test_full.config diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/README.md b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/README.md similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/README.md rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/README.md diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/output.md b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/output.md similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/output.md rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/output.md diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/usage.md b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/usage.md similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/usage.md rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/docs/usage.md diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/main.nf b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/main.nf similarity index 99% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/main.nf rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/main.nf index a84e393a..de383a8b 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/main.nf +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/main.nf @@ -77,7 +77,7 @@ workflow { params.plaintext_email, params.outdir, params.monochrome_logs, - + MYORG_MYFIRSTPIPELINE.out.multiqc_report ) } diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules.json b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules.json similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules.json rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules.json diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/local/fastqe.nf b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/local/fastqe.nf similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/local/fastqe.nf rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/local/fastqe.nf diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/environment.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/environment.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/environment.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/environment.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/main.nf b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/main.nf similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/main.nf rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/main.nf diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/meta.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/meta.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/meta.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/meta.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test.snap b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test.snap similarity index 99% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test.snap rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test.snap index 2fcbb5ff..261dc0fa 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test.snap +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/main.nf.test.snap @@ -38,4 +38,4 @@ }, "timestamp": "2024-10-02T17:52:09.185842" } -} \ No newline at end of file +} diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/nextflow.config b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/nextflow.config similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/nextflow.config rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/nextflow.config diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/tags.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/tags.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/tags.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/multiqc/tests/tags.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/environment.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/environment.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/environment.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/environment.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/main.nf b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/main.nf similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/main.nf rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/main.nf diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/meta.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/meta.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/meta.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/meta.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test.snap b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test.snap similarity index 99% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test.snap rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test.snap index da181dcf..90da25d2 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test.snap +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/main.nf.test.snap @@ -75,4 +75,4 @@ }, "timestamp": "2024-05-03T06:11:38.487227" } -} \ No newline at end of file +} diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/tags.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/tags.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/tags.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/modules/nf-core/seqtk/trim/tests/tags.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/nextflow.config b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/nextflow.config similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/nextflow.config rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/nextflow.config diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/nextflow_schema.json b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/nextflow_schema.json similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/nextflow_schema.json rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/nextflow_schema.json diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/local/utils_nfcore_myfirstpipeline_pipeline/main.nf b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/local/utils_nfcore_myfirstpipeline_pipeline/main.nf similarity index 99% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/local/utils_nfcore_myfirstpipeline_pipeline/main.nf rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/local/utils_nfcore_myfirstpipeline_pipeline/main.nf index 107036dd..4e4a4ece 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/local/utils_nfcore_myfirstpipeline_pipeline/main.nf +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/local/utils_nfcore_myfirstpipeline_pipeline/main.nf @@ -105,7 +105,7 @@ workflow PIPELINE_COMPLETION { plaintext_email // boolean: Send plain-text email instead of HTML outdir // path: Path to output directory where results will be published monochrome_logs // boolean: Disable ANSI colour codes in log output - + multiqc_report // string: Path to MultiQC report main: @@ -164,7 +164,7 @@ def toolCitationText() { // Uncomment function in methodsDescriptionText to render in MultiQC report def citation_text = [ "Tools used in the workflow included:", - + "MultiQC (Ewels et al. 2016)", "." ].join(' ').trim() @@ -177,7 +177,7 @@ def toolBibliographyText() { // Can use ternary operators to dynamically construct based conditions, e.g. params["run_xyz"] ? "
  • Author (2023) Pub name, Journal, DOI
    • " : "", // Uncomment function in methodsDescriptionText to render in MultiQC report def reference_text = [ - + "
  • Ewels, P., Magnusson, M., Lundin, S., & Käller, M. (2016). MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics , 32(19), 3047–3048. doi: /10.1093/bioinformatics/btw354
    • " ].join(' ').trim() @@ -220,4 +220,3 @@ def methodsDescriptionText(mqc_methods_yaml) { return description_html.toString() } - diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/main.nf b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/main.nf similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/main.nf rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/main.nf diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/meta.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/meta.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/meta.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/meta.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test.snap b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test.snap similarity index 99% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test.snap rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test.snap index e3f0baf4..846287c4 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test.snap +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.function.nf.test.snap @@ -17,4 +17,4 @@ }, "timestamp": "2024-02-28T12:02:12.425833" } -} \ No newline at end of file +} diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.workflow.nf.test b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.workflow.nf.test similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.workflow.nf.test rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.workflow.nf.test diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/nextflow.config b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/nextflow.config similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/nextflow.config rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/nextflow.config diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/tags.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/tags.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/tags.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nextflow_pipeline/tests/tags.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/main.nf b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/main.nf similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/main.nf rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/main.nf diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/meta.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/meta.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/meta.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/meta.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap similarity index 99% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap index 1037232c..facbdcfd 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap @@ -163,4 +163,4 @@ }, "timestamp": "2024-02-28T12:03:21.714424" } -} \ No newline at end of file +} diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test.snap b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test.snap similarity index 99% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test.snap rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test.snap index 859d1030..84ee1e1d 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test.snap +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.workflow.nf.test.snap @@ -16,4 +16,4 @@ }, "timestamp": "2024-02-28T12:03:25.726491" } -} \ No newline at end of file +} diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/nextflow.config b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/nextflow.config similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/nextflow.config rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/nextflow.config diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/tags.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/tags.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/tags.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfcore_pipeline/tests/tags.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/main.nf b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/main.nf similarity index 99% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/main.nf rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/main.nf index 4994303e..93de2a52 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/main.nf +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/main.nf @@ -43,4 +43,3 @@ workflow UTILS_NFSCHEMA_PLUGIN { emit: dummy_emit = true } - diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/meta.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/meta.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/meta.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/meta.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow.config b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow.config similarity index 98% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow.config rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow.config index 0907ac58..478fb8a0 100644 --- a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow.config +++ b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow.config @@ -5,4 +5,4 @@ plugins { validation { parametersSchema = "${projectDir}/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow_schema.json" monochromeLogs = true -} \ No newline at end of file +} diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow_schema.json b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow_schema.json similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow_schema.json rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/subworkflows/nf-core/utils_nfschema_plugin/tests/nextflow_schema.json diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/tower.yml b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/tower.yml similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/tower.yml rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/tower.yml diff --git a/hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/workflows/myfirstpipeline.nf b/run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/workflows/myfirstpipeline.nf similarity index 100% rename from hello-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/workflows/myfirstpipeline.nf rename to run-nextflow/hello-nf-core/solution/myorg-myfirstpipeline/workflows/myfirstpipeline.nf diff --git a/hello-nextflow/containers/build/Dockerfile b/side-quests/containers/build/Dockerfile similarity index 100% rename from hello-nextflow/containers/build/Dockerfile rename to side-quests/containers/build/Dockerfile diff --git a/hello-nextflow/containers/build/conda.yml b/side-quests/containers/build/conda.yml similarity index 100% rename from hello-nextflow/containers/build/conda.yml rename to side-quests/containers/build/conda.yml diff --git a/hello-nextflow/data/greetings.csv b/side-quests/containers/data/greetings.csv similarity index 100% rename from hello-nextflow/data/greetings.csv rename to side-quests/containers/data/greetings.csv diff --git a/hello-nextflow/containers/data/pioneers.csv b/side-quests/containers/data/pioneers.csv similarity index 100% rename from hello-nextflow/containers/data/pioneers.csv rename to side-quests/containers/data/pioneers.csv diff --git a/hello-nextflow/containers/results/.gitignore b/side-quests/containers/results/.gitignore similarity index 100% rename from hello-nextflow/containers/results/.gitignore rename to side-quests/containers/results/.gitignore diff --git a/hello-nextflow/containers/solutions/hello-containers-3.nf b/side-quests/containers/solutions/hello-containers-3.nf similarity index 100% rename from hello-nextflow/containers/solutions/hello-containers-3.nf rename to side-quests/containers/solutions/hello-containers-3.nf diff --git a/hello-nextflow/containers/solutions/hello-containers-4.1.nf b/side-quests/containers/solutions/hello-containers-4.1.nf similarity index 100% rename from hello-nextflow/containers/solutions/hello-containers-4.1.nf rename to side-quests/containers/solutions/hello-containers-4.1.nf diff --git a/hello-nextflow/containers/solutions/hello-containers-4.2.nf b/side-quests/containers/solutions/hello-containers-4.2.nf similarity index 100% rename from hello-nextflow/containers/solutions/hello-containers-4.2.nf rename to side-quests/containers/solutions/hello-containers-4.2.nf