Major update: template, pipeline rename, documentation, linting, bioc…

…ontainers (#8) * updated nextflow.config with pipeline rename to omicsgenetraitassociation * Template update for nf-core/tools version 2.11.1 * added modules CMA, MMAP, and PASCAL * fixed versions emit values for cma/main.nf and pascal/main.nf * added when directives to cma/main.nf * added MEA modules * pipeline works from CMA to MEA * added config for CMA and MEA configs in modules.config * updated .gitignore to exclude downloaded singularity images and added new config file for testing cma + mea modules * added temporary subworkflow CMA_SUBWORKFLOW which will later be modified to run CMA and MEA * commented out workflow initialization and validation steps for testing * pascal/main.nf edited to extract tarball as nextflow does not support s3 glob support * fixed versions emit name * added test_local.config, modules.config, and updated subworkflows * added pipeline parameters to nextflow.config, updated nextflow_schema.json, and modified samplesheet schema_input.json * deleted --genome required input check from initialise * added module configs to modules.config and edited test.config with minimal test data * added biocontainers for cma/format_cma_input, mea/merge_ora_and_summary, mea/preprocess, mmap/mmap_parse * added meta map propagation or TODOs to do so * updated subworkflows to match updated modules * updated main workflow with updated subworkflows * added TODO notes for modules that have been requested a biocontainer. Removed stringr import from mea/go_analysis/templates/ORA_cmd.R as it is not needed and causes namespace dependency issues. updated main workflow * updated output.md and CITATIONS.md * updated usage.md and added samplesheet.csv to assets * updated README with metro map. updated container for cma R package which has been renamed CMA-->corrmeta and waiting for bioconductor release * fixed nf-core lint test failures * updated files to match editorconfig linting * Template update for nf-core/tools version 2.12.1 * updates to fix prettier linting * updated cma docker image and fixed template updates * nf-validation messes up s3 input parameters * nextflow.config registry (docker,singularity) update to docker.io. CMA module container dropped docker://. added conda environment.yml to format_cma_input. * MEA module added environment.yml to merge_ora_and_summary and preprocess. Updated container tags by dropping docker:// prefix * dropped docker:// from MMAP container tag * updated pascal/main.nf container dropping docker:// * added conda environment.yml and updated to Biocontainers mulled singularity and quay containers for mea/postprocess * fixed quay.io biocontiner * dropped docker:// from go_analysis/main.nf container tag * fix linting * Update docs/output.md Co-authored-by: Matthias Hörtenhuber <[email protected]> * Update docs/output.md Co-authored-by: Matthias Hörtenhuber <[email protected]> * removed .prettierrc --------- Co-authored-by: Matthias Hörtenhuber <[email protected]> Co-authored-by: Matthias Hörtenhuber <[email protected]>
nf-core · Feb 22, 2024 · f64a3c7 · f64a3c7
1 parent ff9979e
commit f64a3c7
Show file tree

Hide file tree

Showing 94 changed files with 1,766 additions and 1,407 deletions.
diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json
@@ -22,7 +22,11 @@
             },
 
             // Add the IDs of extensions you want installed when the container is created.
-            "extensions": ["ms-python.python", "ms-python.vscode-pylance", "nf-core.nf-core-extensionpack"]
+            "extensions": [
+                "ms-python.python",
+                "ms-python.vscode-pylance",
+                "nf-core.nf-core-extensionpack"
+            ]
         }
     }
 }
diff --git a/.editorconfig b/.editorconfig
@@ -22,3 +22,11 @@ indent_size = unset
 
 [/assets/email*]
 indent_size = unset
+
+# ignore Readme
+[README.md]
+indent_style = unset
+
+# ignore python
+[*.{py}]
+indent_style = unset
diff --git a/.github/.dockstore.yml b/.github/.dockstore.yml
@@ -1,6 +1,6 @@
 # Dockstore config version, not pipeline version
 version: 1.2
 workflows:
-  - subclass: nfl
-    primaryDescriptorPath: /nextflow.config
-    publish: True
+    - subclass: nfl
+      primaryDescriptorPath: /nextflow.config
+      publish: True
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
@@ -53,9 +53,9 @@ These tests are run both with the latest available version of `Nextflow` and als
 
 :warning: Only in the unlikely and regretful event of a release happening with a bug.
 
-- On your own fork, make a new branch `patch` based on `upstream/master`.
-- Fix the bug, and bump version (X.Y.Z+1).
-- A PR should be made on `master` from patch to directly this particular bug.
+-   On your own fork, make a new branch `patch` based on `upstream/master`.
+-   Fix the bug, and bump version (X.Y.Z+1).
+-   A PR should be made on `master` from patch to directly this particular bug.
 
 ## Getting help
 
@@ -96,8 +96,8 @@ The process resources can be passed on to the tool dynamically within the proces
 
 Please use the following naming schemes, to make it easy to understand what is going where.
 
-- initial process channel: `ch_output_from_<process>`
-- intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`
+-   initial process channel: `ch_output_from_<process>`
+-   intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`
 
 ### Nextflow version bumping
 
@@ -113,11 +113,11 @@ This repo includes a devcontainer configuration which will create a GitHub Codes
 
 To get started:
 
-- Open the repo in [Codespaces](https://github.com/nf-core/omicsgenetraitassociation/codespaces)
-- Tools installed
-  - nf-core
-  - Nextflow
+-   Open the repo in [Codespaces](https://github.com/nf-core/omicsgenetraitassociation/codespaces)
+-   Tools installed
+    -   nf-core
+    -   Nextflow
 
 Devcontainer specs:
 
-- [DevContainer config](.devcontainer/devcontainer.json)
+-   [DevContainer config](.devcontainer/devcontainer.json)
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml
@@ -2,49 +2,49 @@ name: Bug report
 description: Report something that is broken or incorrect
 labels: bug
 body:
-  - type: markdown
-    attributes:
-      value: |
-        Before you post this issue, please check the documentation:
+    - type: markdown
+      attributes:
+          value: |
+              Before you post this issue, please check the documentation:
 
-        - [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
-        - [nf-core/omicsgenetraitassociation pipeline documentation](https://nf-co.re/omicsgenetraitassociation/usage)
+              - [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
+              - [nf-core/omicsgenetraitassociation pipeline documentation](https://nf-co.re/omicsgenetraitassociation/usage)
 
-  - type: textarea
-    id: description
-    attributes:
-      label: Description of the bug
-      description: A clear and concise description of what the bug is.
-    validations:
-      required: true
+    - type: textarea
+      id: description
+      attributes:
+          label: Description of the bug
+          description: A clear and concise description of what the bug is.
+      validations:
+          required: true
 
-  - type: textarea
-    id: command_used
-    attributes:
-      label: Command used and terminal output
-      description: Steps to reproduce the behaviour. Please paste the command you used to launch the pipeline and the output from your terminal.
-      render: console
-      placeholder: |
-        $ nextflow run ...
+    - type: textarea
+      id: command_used
+      attributes:
+          label: Command used and terminal output
+          description: Steps to reproduce the behaviour. Please paste the command you used to launch the pipeline and the output from your terminal.
+          render: console
+          placeholder: |
+              $ nextflow run ...
 
-        Some output where something broke
+              Some output where something broke
 
-  - type: textarea
-    id: files
-    attributes:
-      label: Relevant files
-      description: |
-        Please drag and drop the relevant files here. Create a `.zip` archive if the extension is not allowed.
-        Your verbose log file `.nextflow.log` is often useful _(this is a hidden file in the directory where you launched the pipeline)_ as well as custom Nextflow configuration files.
+    - type: textarea
+      id: files
+      attributes:
+          label: Relevant files
+          description: |
+              Please drag and drop the relevant files here. Create a `.zip` archive if the extension is not allowed.
+              Your verbose log file `.nextflow.log` is often useful _(this is a hidden file in the directory where you launched the pipeline)_ as well as custom Nextflow configuration files.
 
-  - type: textarea
-    id: system
-    attributes:
-      label: System information
-      description: |
-        * Nextflow version _(eg. 23.04.0)_
-        * Hardware _(eg. HPC, Desktop, Cloud)_
-        * Executor _(eg. slurm, local, awsbatch)_
-        * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_
-        * OS _(eg. CentOS Linux, macOS, Linux Mint)_
-        * Version of nf-core/omicsgenetraitassociation _(eg. 1.1, 1.5, 1.8.2)_
+    - type: textarea
+      id: system
+      attributes:
+          label: System information
+          description: |
+              * Nextflow version _(eg. 23.04.0)_
+              * Hardware _(eg. HPC, Desktop, Cloud)_
+              * Executor _(eg. slurm, local, awsbatch)_
+              * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_
+              * OS _(eg. CentOS Linux, macOS, Linux Mint)_
+              * Version of nf-core/omicsgenetraitassociation _(eg. 1.1, 1.5, 1.8.2)_
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
@@ -1,7 +1,7 @@
 contact_links:
-  - name: Join nf-core
-    url: https://nf-co.re/join
-    about: Please join the nf-core community here
-  - name: "Slack #omicsgenetraitassociation channel"
-    url: https://nfcore.slack.com/channels/omicsgenetraitassociation
-    about: Discussion about the nf-core/omicsgenetraitassociation pipeline
+    - name: Join nf-core
+      url: https://nf-co.re/join
+      about: Please join the nf-core community here
+    - name: "Slack #omicsgenetraitassociation channel"
+      url: https://nfcore.slack.com/channels/omicsgenetraitassociation
+      about: Discussion about the nf-core/omicsgenetraitassociation pipeline
diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml
@@ -2,10 +2,10 @@ name: Feature request
 description: Suggest an idea for the nf-core/omicsgenetraitassociation pipeline
 labels: enhancement
 body:
-  - type: textarea
-    id: description
-    attributes:
-      label: Description of feature
-      description: Please describe your suggestion for a new feature. It might help to describe a problem or use case, plus any alternatives that you have considered.
-    validations:
-      required: true
+    - type: textarea
+      id: description
+      attributes:
+          label: Description of feature
+          description: Please describe your suggestion for a new feature. It might help to describe a problem or use case, plus any alternatives that you have considered.
+      validations:
+          required: true
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -13,14 +13,14 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/omic
 
 ## PR checklist
 
-- [ ] This comment contains a description of changes (with reason).
-- [ ] If you've fixed a bug or added code that should be tested, add tests!
-- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/omicsgenetraitassociation/tree/master/.github/CONTRIBUTING.md)
-- [ ] If necessary, also make a PR on the nf-core/omicsgenetraitassociation _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
-- [ ] Make sure your code lints (`nf-core lint`).
-- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
-- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
-- [ ] Usage Documentation in `docs/usage.md` is updated.
-- [ ] Output Documentation in `docs/output.md` is updated.
-- [ ] `CHANGELOG.md` is updated.
-- [ ] `README.md` is updated (including new tool citations and authors/contributors).
+-   [ ] This comment contains a description of changes (with reason).
+-   [ ] If you've fixed a bug or added code that should be tested, add tests!
+-   [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/omicsgenetraitassociation/tree/master/.github/CONTRIBUTING.md)
+-   [ ] If necessary, also make a PR on the nf-core/omicsgenetraitassociation _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
+-   [ ] Make sure your code lints (`nf-core lint`).
+-   [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
+-   [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
+-   [ ] Usage Documentation in `docs/usage.md` is updated.
+-   [ ] Output Documentation in `docs/output.md` is updated.
+-   [ ] `CHANGELOG.md` is updated.
+-   [ ] `README.md` is updated (including new tool citations and authors/contributors).
diff --git a/.github/workflows/awsfulltest.yml b/.github/workflows/awsfulltest.yml
@@ -4,36 +4,36 @@ name: nf-core AWS full size tests
 # It runs the -profile 'test_full' on AWS batch
 
 on:
-  release:
-    types: [published]
-  workflow_dispatch:
+    release:
+        types: [published]
+    workflow_dispatch:
 jobs:
-  run-tower:
-    name: Run AWS full tests
-    if: github.repository == 'nf-core/omicsgenetraitassociation'
-    runs-on: ubuntu-latest
-    steps:
-      - name: Launch workflow via tower
-        uses: seqeralabs/action-tower-launch@v2
-        # TODO nf-core: You can customise AWS full pipeline tests as required
-        # Add full size test data (but still relatively small datasets for few samples)
-        # on the `test_full.config` test runs with only one set of parameters
-        with:
-          workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
-          access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
-          compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
-          revision: ${{ github.sha }}
-          workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/omicsgenetraitassociation/work-${{ github.sha }}
-          parameters: |
-            {
-              "hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
-              "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/omicsgenetraitassociation/results-${{ github.sha }}"
-            }
-          profiles: test_full
+    run-tower:
+        name: Run AWS full tests
+        if: github.repository == 'nf-core/omicsgenetraitassociation'
+        runs-on: ubuntu-latest
+        steps:
+            - name: Launch workflow via tower
+              uses: seqeralabs/action-tower-launch@v2
+              # TODO nf-core: You can customise AWS full pipeline tests as required
+              # Add full size test data (but still relatively small datasets for few samples)
+              # on the `test_full.config` test runs with only one set of parameters
+              with:
+                  workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
+                  access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
+                  compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
+                  revision: ${{ github.sha }}
+                  workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/omicsgenetraitassociation/work-${{ github.sha }}
+                  parameters: |
+                      {
+                        "hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
+                        "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/omicsgenetraitassociation/results-${{ github.sha }}"
+                      }
+                  profiles: test_full
 
-      - uses: actions/upload-artifact@v3
-        with:
-          name: Tower debug log file
-          path: |
-            tower_action_*.log
-            tower_action_*.json
+            - uses: actions/upload-artifact@v4
+              with:
+                  name: Tower debug log file
+                  path: |
+                      tower_action_*.log
+                      tower_action_*.json
diff --git a/.github/workflows/awstest.yml b/.github/workflows/awstest.yml
@@ -3,31 +3,31 @@ name: nf-core AWS test
 # It runs the -profile 'test' on AWS batch
 
 on:
-  workflow_dispatch:
+    workflow_dispatch:
 jobs:
-  run-tower:
-    name: Run AWS tests
-    if: github.repository == 'nf-core/omicsgenetraitassociation'
-    runs-on: ubuntu-latest
-    steps:
-      # Launch workflow using Tower CLI tool action
-      - name: Launch workflow via tower
-        uses: seqeralabs/action-tower-launch@v2
-        with:
-          workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
-          access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
-          compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
-          revision: ${{ github.sha }}
-          workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/omicsgenetraitassociation/work-${{ github.sha }}
-          parameters: |
-            {
-              "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/omicsgenetraitassociation/results-test-${{ github.sha }}"
-            }
-          profiles: test
+    run-tower:
+        name: Run AWS tests
+        if: github.repository == 'nf-core/omicsgenetraitassociation'
+        runs-on: ubuntu-latest
+        steps:
+            # Launch workflow using Tower CLI tool action
+            - name: Launch workflow via tower
+              uses: seqeralabs/action-tower-launch@v2
+              with:
+                  workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
+                  access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
+                  compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
+                  revision: ${{ github.sha }}
+                  workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/omicsgenetraitassociation/work-${{ github.sha }}
+                  parameters: |
+                      {
+                        "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/omicsgenetraitassociation/results-test-${{ github.sha }}"
+                      }
+                  profiles: test
 
-      - uses: actions/upload-artifact@v3
-        with:
-          name: Tower debug log file
-          path: |
-            tower_action_*.log
-            tower_action_*.json
+            - uses: actions/upload-artifact@v4
+              with:
+                  name: Tower debug log file
+                  path: |
+                      tower_action_*.log
+                      tower_action_*.json