Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dp24 module update samtools #333

Merged
merged 14 commits into from
Nov 21, 2024
Merged

Dp24 module update samtools #333

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
cb5d998
Updates for modules, README and config
DLBPointon Nov 15, 2024
79bd32f
Updating SAMTOOLS
DLBPointon Nov 20, 2024
217bd32
linting
DLBPointon Nov 20, 2024
30a576c
Updating all easy to update modules
DLBPointon Nov 20, 2024
e93fd9a
Updating all easy to update modules
DLBPointon Nov 20, 2024
dc1e076
Updating all easy to update modules
DLBPointon Nov 20, 2024
3c660bb
Updating all easy to update modules
DLBPointon Nov 20, 2024
32152a5
Fixes to modules
DLBPointon Nov 20, 2024
1beeeb5
linting
DLBPointon Nov 20, 2024
4ad0c65
Adding steps to skip BUSCO
DLBPointon Nov 21, 2024
160af01
Pinning BUSCO to V5.5
DLBPointon Nov 21, 2024
55ed6d6
Removing steps var
DLBPointon Nov 21, 2024
0ada598
Linting
DLBPointon Nov 21, 2024
85217d9
Merge pull request #335 from sanger-tol/dp24_module_update_2
DLBPointon Nov 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 73 additions & 6 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,75 @@
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.2.0] - Ancient Destiny - [2024-11-15]

Our 3rd release for sanger-tol/treeval.

### Enhancements & Fixes

- Togglable subworkflows
- Adds a JBrowse Only workflow (this will lead to an update to the FULL workflow which can now call JBROWSE_ONLY and RAPID).
- Updates to containers (local modules) to remove Anaconda dependencies following policy changes.
- Updates to modules to remove Anaconda dependencies following policy changes
- CONDA warnings for modules which cannot use CONDA.
- Removable of a liberal use of spaces.
- reformat_intersect was previously not outputing version data.
- Adding arch specification to Pretext GitHub actions runner. Hopefully this will stop the spurious errors we see on there.
- Addition of steps into schema.

### Parameters

| Old Parameter | New Parameter |
| ------------- | ------------- |
| - | --steps |

### Software dependencies

Note, since the pipeline is using Nextflow DSL2, each process will be run with its own Biocontainer. This means that on occasion it is entirely possible for the pipeline to be using different versions of the same tool. However, the overall software dependency changes compared to the last release have been listed below for reference.

| Module | Old Version | New Versions |
| -------------------------------------- | ---------------- | ------------ |
| bamtobed_sort ( bedtools + samtools ) | 2.31.0 + 1.17 | |
| bedtools | 2.31.1 | |
| busco | 5.5.0 | |
| bwa-mem2 | 2.2.1 | |
| cat | 2.3.4 | |
| chunk_fasta ( pyfasta ) | 0.5.2-1 | |
| cooler | 0.9.2 | |
| cram_filter_align_bwamem2_fixmate_sort | - | |
| ^ ( samtools + bwamem2 ) ^ | 1.17 + 2.2.1 | |
| coreutils | 9.1 | |
| fastk | 1.0.1 | |
| gcc | 10.4.0 | |
| find_telomere_windows ( java-jdk ) | 8.0.112 | |
| generate_cram_csv ( samtools ) | 1.17 | |
| gnu-sort | 8.25 | |
| juicer_tools_pre ( java-jdk ) | 8.0.112 | |
| perl | 5.26.2 | |
| merquryfk | 1.0.1 | |
| minimap2 + samtools | 2.24 + 1.14 | |
| miniprot | 0.11--he4a0461_2 | |
| mummer | 3.23 | |
| paftools ( minimap2 + samtools ) | 2.24 + 1.14 | |
| pretextmap + samtools | 0.0.2 + 1.17 | |
| python | 3.9 | - |
| - pandas | 1.5.2 | - |
| samtools | 1.18 | 1.21 |
| selfcomp_splitfasta ( perl-bioperl ) | 1.7.8-1 | |
| seqtk | 1.4 | |
| tabix | 1.11 | |
| ucsc | 377 | |
| windowmasker (blast) | 2.14.0 | |

## [1.1.1] - Ancient Aurora (H1) - [2024-04-26]

### Enhancements & Fixes

- Generate CRAM CSV fix to allow for multi-readgroup cram files
- Removing KMER_READCOV
- tmp directory was being used
- Output file adjustment (names and location)

## [1.1.0] - Ancient Aurora - [2024-04-26]

The second release for sanger-tol, created with the [nf-core](https://nf-co.re/) template.
Expand Down Expand Up @@ -42,18 +111,16 @@ This builds on the initial release by adding subworkflows which generate kmer ba

#### Hot Fix 1

- Adding support for multi-library cram input.

#### Hot Fix 2

- Adding support to select subworkflows to use in pipeline run.
- Generate CRAM CSV fix to allow for multi-readgroup cram files
- Removing KMER_READCOV
- tmp directory was being used
- Output file adjustment (names and location)

### Parameters

| Old Parameter | New Parameter |
| ------------- | ------------- |
| - | --juicer |
| - | --steps |

### Software dependencies

Expand Down
4 changes: 1 addition & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

## Introduction

**sanger-tol/treeval [1.1.0 - Ancient Aurora]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass.
**sanger-tol/treeval [1.2.0 - Ancient Destiny-]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass.

The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from [nf-core/modules](https://github.com/nf-core/modules) in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!

Expand Down Expand Up @@ -80,8 +80,6 @@ If you would like to contribute to this pipeline, please see the [contributing g

## Citations

<!--TODO: Citation-->

If you use sanger-tol/treeval for your analysis, please cite it using the following doi: [10.5281/zenodo.10047653](https://doi.org/10.5281/zenodo.10047653).

### Tools
Expand Down
4 changes: 2 additions & 2 deletions conf/base.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,14 +225,14 @@ process {
memory = { check_max( 100.GB * task.attempt, 'memory' ) }
}

withName: BUSCO {
withName: BUSCO_BUSCO {
cpus = { check_max( 16 * task.attempt, 'cpus' ) }
memory = { check_max( 50.GB * task.attempt, 'memory' ) }
time = { check_max( 20.h * task.attempt, 'time' ) }
}

// Large Genomes > 4Gb
//withName: BUSCO {
//withName: BUSCO_BUSCO {
// cpus = { check_max( 30 * task.attempt, 'cpus' ) }
// memory = { check_max( 100.GB * task.attempt, 'memory' ) }
// time = { check_max( 300.h * task.attempt, 'time' ) }
Expand Down
2 changes: 1 addition & 1 deletion conf/modules.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,7 @@ process {
ext.prefix = { "${meta.id}_ancestral" }
}

withName: "BUSCO" {
withName: "BUSCO_BUSCO" {
ext.args = "--offline"
}

Expand Down
63 changes: 32 additions & 31 deletions modules.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"nf-core": {
"bedtools/bamtobed": {
"branch": "master",
"git_sha": "1d1cb7bfef6cf67fbc7faafa6992ad8bdc3045b3",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"bedtools/genomecov": {
Expand All @@ -17,12 +17,12 @@
},
"bedtools/intersect": {
"branch": "master",
"git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"bedtools/makewindows": {
"branch": "master",
"git_sha": "3b248b84694d1939ac4bb33df84bf6233a34d668",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"bedtools/map": {
Expand All @@ -32,27 +32,28 @@
},
"bedtools/merge": {
"branch": "master",
"git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"bedtools/sort": {
"branch": "master",
"git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"busco": {
"busco/busco": {
"branch": "master",
"git_sha": "e3126f437c336c826f242842fe51769cfce0ec2d",
"installed_by": ["modules"]
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"],
"patch": "modules/nf-core/busco/busco/busco-busco.diff"
},
"bwamem2/index": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"cat/cat": {
"branch": "master",
"git_sha": "81f27e75847087865299cc46605deb3b09b4e0a2",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"cooler/cload": {
Expand All @@ -67,12 +68,12 @@
},
"custom/dumpsoftwareversions": {
"branch": "master",
"git_sha": "8ec825f465b9c17f9d83000022995b4f7de6fe93",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"custom/getchromsizes": {
"branch": "master",
"git_sha": "1b0ffa4e5aed5b7e3cd4311af31bd3b2c8345051",
"git_sha": "49f4e50534fe4b64101e62ea41d5dc43b1324358",
"installed_by": ["modules"],
"patch": "modules/nf-core/custom/getchromsizes/custom-getchromsizes.diff"
},
Expand All @@ -83,7 +84,7 @@
},
"gnu/sort": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"merquryfk/merquryfk": {
Expand All @@ -99,17 +100,17 @@
},
"minimap2/index": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"miniprot/align": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"miniprot/index": {
"branch": "master",
"git_sha": "8d737766e8f3c1417212b4b56acb959f3c356d26",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"mummer": {
Expand All @@ -120,79 +121,79 @@
},
"paftools/sam2paf": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"pretextmap": {
"branch": "master",
"git_sha": "decfb802f2e573efb7b44ff06b11ecf16853054d",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"],
"patch": "modules/nf-core/pretextmap/pretextmap.diff"
},
"pretextsnapshot": {
"branch": "master",
"git_sha": "911696ea0b62df80e900ef244d7867d177971f73",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"],
"patch": "modules/nf-core/pretextsnapshot/pretextsnapshot.diff"
},
"samtools/faidx": {
"branch": "master",
"git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
"git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208",
"installed_by": ["modules"]
},
"samtools/index": {
"branch": "master",
"git_sha": "a64788f5ad388f1d2ac5bd5f1f3f8fc81476148c",
"git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208",
"installed_by": ["modules"]
},
"samtools/markdup": {
"branch": "master",
"git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
"git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208",
"installed_by": ["modules"]
},
"samtools/merge": {
"branch": "master",
"git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
"git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208",
"installed_by": ["modules"]
},
"samtools/sort": {
"branch": "master",
"git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
"git_sha": "b7800db9b069ed505db3f9d91b8c72faea9be17b",
"installed_by": ["modules"]
},
"samtools/view": {
"branch": "master",
"git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c",
"git_sha": "669eb24fd82a9d3cb18ad0e73673ecb26827f683",
"installed_by": ["modules"]
},
"seqtk/cutn": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"tabix/bgziptabix": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "f448e846bdadd80fc8be31fbbc78d9f5b5131a45",
"installed_by": ["modules"]
},
"ucsc/bedgraphtobigwig": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "49f4e50534fe4b64101e62ea41d5dc43b1324358",
"installed_by": ["modules"]
},
"ucsc/bedtobigbed": {
"branch": "master",
"git_sha": "9e51255c4f8ec69fb6ccf68593392835f14fecb8",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"windowmasker/mkcounts": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"windowmasker/ustat": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
}
}
Expand Down
2 changes: 0 additions & 2 deletions modules/nf-core/bedtools/bamtobed/environment.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading