Merge pull request #333 from sanger-tol/dp24_module_update_samtools

Dp24 module update samtools
sanger-tol · Nov 21, 2024 · 7a0ca25 · 7a0ca25
2 parents 701b19b + 85217d9
commit 7a0ca25
Show file tree

Hide file tree

Showing 179 changed files with 7,822 additions and 1,612 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,6 +3,75 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.2.0] - Ancient Destiny - [2024-11-15]
+
+Our 3rd release for sanger-tol/treeval.
+
+### Enhancements & Fixes
+
+- Togglable subworkflows
+- Adds a JBrowse Only workflow (this will lead to an update to the FULL workflow which can now call JBROWSE_ONLY and RAPID).
+- Updates to containers (local modules) to remove Anaconda dependencies following policy changes.
+- Updates to modules to remove Anaconda dependencies following policy changes
+- CONDA warnings for modules which cannot use CONDA.
+- Removable of a liberal use of spaces.
+- reformat_intersect was previously not outputing version data.
+- Adding arch specification to Pretext GitHub actions runner. Hopefully this will stop the spurious errors we see on there.
+- Addition of steps into schema.
+
+### Parameters
+
+| Old Parameter | New Parameter |
+| ------------- | ------------- |
+| -             | --steps       |
+
+### Software dependencies
+
+Note, since the pipeline is using Nextflow DSL2, each process will be run with its own Biocontainer. This means that on occasion it is entirely possible for the pipeline to be using different versions of the same tool. However, the overall software dependency changes compared to the last release have been listed below for reference.
+
+| Module                                 | Old Version      | New Versions |
+| -------------------------------------- | ---------------- | ------------ |
+| bamtobed_sort ( bedtools + samtools )  | 2.31.0 + 1.17    |              |
+| bedtools                               | 2.31.1           |              |
+| busco                                  | 5.5.0            |              |
+| bwa-mem2                               | 2.2.1            |              |
+| cat                                    | 2.3.4            |              |
+| chunk_fasta ( pyfasta )                | 0.5.2-1          |              |
+| cooler                                 | 0.9.2            |              |
+| cram_filter_align_bwamem2_fixmate_sort | -                |              |
+| ^ ( samtools + bwamem2 ) ^             | 1.17 + 2.2.1     |              |
+| coreutils                              | 9.1              |              |
+| fastk                                  | 1.0.1            |              |
+| gcc                                    | 10.4.0           |              |
+| find_telomere_windows ( java-jdk )     | 8.0.112          |              |
+| generate_cram_csv ( samtools )         | 1.17             |              |
+| gnu-sort                               | 8.25             |              |
+| juicer_tools_pre ( java-jdk )          | 8.0.112          |              |
+| perl                                   | 5.26.2           |              |
+| merquryfk                              | 1.0.1            |              |
+| minimap2 + samtools                    | 2.24 + 1.14      |              |
+| miniprot                               | 0.11--he4a0461_2 |              |
+| mummer                                 | 3.23             |              |
+| paftools ( minimap2 + samtools )       | 2.24 + 1.14      |              |
+| pretextmap + samtools                  | 0.0.2 + 1.17     |              |
+| python                                 | 3.9              | -            |
+| - pandas                               | 1.5.2            | -            |
+| samtools                               | 1.18             | 1.21         |
+| selfcomp_splitfasta ( perl-bioperl )   | 1.7.8-1          |              |
+| seqtk                                  | 1.4              |              |
+| tabix                                  | 1.11             |              |
+| ucsc                                   | 377              |              |
+| windowmasker (blast)                   | 2.14.0           |              |
+
+## [1.1.1] - Ancient Aurora (H1) - [2024-04-26]
+
+### Enhancements & Fixes
+
+- Generate CRAM CSV fix to allow for multi-readgroup cram files
+- Removing KMER_READCOV
+- tmp directory was being used
+- Output file adjustment (names and location)
+
 ## [1.1.0] - Ancient Aurora - [2024-04-26]
 
 The second release for sanger-tol, created with the [nf-core](https://nf-co.re/) template.
@@ -42,18 +111,16 @@ This builds on the initial release by adding subworkflows which generate kmer ba
 
 #### Hot Fix 1
 
-- Adding support for multi-library cram input.
-
-#### Hot Fix 2
-
-- Adding support to select subworkflows to use in pipeline run.
+- Generate CRAM CSV fix to allow for multi-readgroup cram files
+- Removing KMER_READCOV
+- tmp directory was being used
+- Output file adjustment (names and location)
 
 ### Parameters
 
 | Old Parameter | New Parameter |
 | ------------- | ------------- |
 | -             | --juicer      |
-| -             | --steps       |
 
 ### Software dependencies
 

diff --git a/README.md b/README.md
@@ -7,7 +7,7 @@
 
 ## Introduction
 
-**sanger-tol/treeval [1.1.0 - Ancient Aurora]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass.
+**sanger-tol/treeval [1.2.0 - Ancient Destiny-]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass.
 
 The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from [nf-core/modules](https://github.com/nf-core/modules) in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!
 
@@ -80,8 +80,6 @@ If you would like to contribute to this pipeline, please see the [contributing g
 
 ## Citations
 
-<!--TODO: Citation-->
-
 If you use sanger-tol/treeval for your analysis, please cite it using the following doi: [10.5281/zenodo.10047653](https://doi.org/10.5281/zenodo.10047653).
 
 ### Tools

diff --git a/conf/base.config b/conf/base.config
@@ -225,14 +225,14 @@ process {
         memory = { check_max( 100.GB    * task.attempt, 'memory'  ) }
     }
 
-    withName: BUSCO {
+    withName: BUSCO_BUSCO {
         cpus   = { check_max( 16        * task.attempt, 'cpus'    ) }
         memory = { check_max( 50.GB     * task.attempt, 'memory'  ) }
         time   = { check_max( 20.h      * task.attempt, 'time'    ) }
     }
 
     // Large Genomes > 4Gb
-    //withName: BUSCO {
+    //withName: BUSCO_BUSCO {
     //    cpus   = { check_max( 30      * task.attempt, 'cpus'    ) }
     //    memory = { check_max( 100.GB  * task.attempt, 'memory'  ) }
     //    time   = { check_max( 300.h   * task.attempt, 'time'    ) }

diff --git a/conf/modules.config b/conf/modules.config
@@ -315,7 +315,7 @@ process {
         ext.prefix      = { "${meta.id}_ancestral" }
     }
 
-    withName: "BUSCO" {
+    withName: "BUSCO_BUSCO" {
         ext.args        = "--offline"
     }
 

diff --git a/modules.json b/modules.json
@@ -7,7 +7,7 @@
                 "nf-core": {
                     "bedtools/bamtobed": {
                         "branch": "master",
-                        "git_sha": "1d1cb7bfef6cf67fbc7faafa6992ad8bdc3045b3",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "bedtools/genomecov": {
@@ -17,12 +17,12 @@
                     },
                     "bedtools/intersect": {
                         "branch": "master",
-                        "git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "bedtools/makewindows": {
                         "branch": "master",
-                        "git_sha": "3b248b84694d1939ac4bb33df84bf6233a34d668",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "bedtools/map": {
@@ -32,27 +32,28 @@
                     },
                     "bedtools/merge": {
                         "branch": "master",
-                        "git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "bedtools/sort": {
                         "branch": "master",
-                        "git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
-                    "busco": {
+                    "busco/busco": {
                         "branch": "master",
-                        "git_sha": "e3126f437c336c826f242842fe51769cfce0ec2d",
-                        "installed_by": ["modules"]
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
+                        "installed_by": ["modules"],
+                        "patch": "modules/nf-core/busco/busco/busco-busco.diff"
                     },
                     "bwamem2/index": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "cat/cat": {
                         "branch": "master",
-                        "git_sha": "81f27e75847087865299cc46605deb3b09b4e0a2",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "cooler/cload": {
@@ -67,12 +68,12 @@
                     },
                     "custom/dumpsoftwareversions": {
                         "branch": "master",
-                        "git_sha": "8ec825f465b9c17f9d83000022995b4f7de6fe93",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "custom/getchromsizes": {
                         "branch": "master",
-                        "git_sha": "1b0ffa4e5aed5b7e3cd4311af31bd3b2c8345051",
+                        "git_sha": "49f4e50534fe4b64101e62ea41d5dc43b1324358",
                         "installed_by": ["modules"],
                         "patch": "modules/nf-core/custom/getchromsizes/custom-getchromsizes.diff"
                     },
@@ -83,7 +84,7 @@
                     },
                     "gnu/sort": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "merquryfk/merquryfk": {
@@ -99,17 +100,17 @@
                     },
                     "minimap2/index": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "miniprot/align": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "miniprot/index": {
                         "branch": "master",
-                        "git_sha": "8d737766e8f3c1417212b4b56acb959f3c356d26",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "mummer": {
@@ -120,79 +121,79 @@
                     },
                     "paftools/sam2paf": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "pretextmap": {
                         "branch": "master",
-                        "git_sha": "decfb802f2e573efb7b44ff06b11ecf16853054d",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"],
                         "patch": "modules/nf-core/pretextmap/pretextmap.diff"
                     },
                     "pretextsnapshot": {
                         "branch": "master",
-                        "git_sha": "911696ea0b62df80e900ef244d7867d177971f73",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"],
                         "patch": "modules/nf-core/pretextsnapshot/pretextsnapshot.diff"
                     },
                     "samtools/faidx": {
                         "branch": "master",
-                        "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
+                        "git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208",
                         "installed_by": ["modules"]
                     },
                     "samtools/index": {
                         "branch": "master",
-                        "git_sha": "a64788f5ad388f1d2ac5bd5f1f3f8fc81476148c",
+                        "git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208",
                         "installed_by": ["modules"]
                     },
                     "samtools/markdup": {
                         "branch": "master",
-                        "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
+                        "git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208",
                         "installed_by": ["modules"]
                     },
                     "samtools/merge": {
                         "branch": "master",
-                        "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
+                        "git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208",
                         "installed_by": ["modules"]
                     },
                     "samtools/sort": {
                         "branch": "master",
-                        "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
+                        "git_sha": "b7800db9b069ed505db3f9d91b8c72faea9be17b",
                         "installed_by": ["modules"]
                     },
                     "samtools/view": {
                         "branch": "master",
-                        "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
+                        "git_sha": "669eb24fd82a9d3cb18ad0e73673ecb26827f683",
                         "installed_by": ["modules"]
                     },
                     "seqtk/cutn": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "tabix/bgziptabix": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "f448e846bdadd80fc8be31fbbc78d9f5b5131a45",
                         "installed_by": ["modules"]
                     },
                     "ucsc/bedgraphtobigwig": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "49f4e50534fe4b64101e62ea41d5dc43b1324358",
                         "installed_by": ["modules"]
                     },
                     "ucsc/bedtobigbed": {
                         "branch": "master",
-                        "git_sha": "9e51255c4f8ec69fb6ccf68593392835f14fecb8",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "windowmasker/mkcounts": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     },
                     "windowmasker/ustat": {
                         "branch": "master",
-                        "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
+                        "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
                         "installed_by": ["modules"]
                     }
                 }

diff --git a/modules/nf-core/bedtools/bamtobed/environment.yml b/modules/nf-core/bedtools/bamtobed/environment.yml