Skip to content

Commit

Permalink
Merge pull request #339 from sanger-tol/dp24_module_update_3
Browse files Browse the repository at this point in the history
Dp24 module update 3
  • Loading branch information
weaglesBio authored Nov 27, 2024
2 parents a4b7ffd + 8cf822c commit 20045d6
Show file tree
Hide file tree
Showing 40 changed files with 1,630 additions and 374 deletions.
75 changes: 42 additions & 33 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,16 @@ Our 3rd release for sanger-tol/treeval.
- Adds a JBrowse Only workflow (this will lead to an update to the FULL workflow which can now call JBROWSE_ONLY and RAPID).
- Updates to containers (local modules) to remove Anaconda dependencies following policy changes.
- Updates to modules to remove Anaconda dependencies following policy changes
- The majority of these updates only remove the `default` channel from the environment.yml
- CONDA warnings for modules which cannot use CONDA.
- Removable of a liberal use of spaces.
- reformat_intersect was previously not outputing version data.
- Adding arch specification to Pretext GitHub actions runner. Hopefully this will stop the spurious errors we see on there.
- Addition of steps into schema.
- Adds \*ktab as an output.
- Updated singularity containers
- Added `--metaeuk` to BUSCO_BUSCO, default was causing pipeline errors on Actions -- Needs more investigation.
- Replaced Pyfasta split (depreciated 6 years ago) with Seqkit split which is frequently updated and very fast.

### Parameters

Expand All @@ -29,39 +34,43 @@ Our 3rd release for sanger-tol/treeval.

Note, since the pipeline is using Nextflow DSL2, each process will be run with its own Biocontainer. This means that on occasion it is entirely possible for the pipeline to be using different versions of the same tool. However, the overall software dependency changes compared to the last release have been listed below for reference.

| Module | Old Version | New Versions |
| -------------------------------------- | ---------------- | ------------ |
| bamtobed_sort ( bedtools + samtools ) | 2.31.0 + 1.17 | |
| bedtools | 2.31.1 | |
| busco | 5.5.0 | |
| bwa-mem2 | 2.2.1 | |
| cat | 2.3.4 | |
| chunk_fasta ( pyfasta ) | 0.5.2-1 | |
| cooler | 0.9.2 | |
| cram_filter_align_bwamem2_fixmate_sort | - | |
| ^ ( samtools + bwamem2 ) ^ | 1.17 + 2.2.1 | |
| coreutils | 9.1 | |
| fastk | 1.0.1 | |
| gcc | 10.4.0 | |
| find_telomere_windows ( java-jdk ) | 8.0.112 | |
| generate_cram_csv ( samtools ) | 1.17 | |
| gnu-sort | 8.25 | |
| juicer_tools_pre ( java-jdk ) | 8.0.112 | |
| perl | 5.26.2 | |
| merquryfk | 1.0.1 | |
| minimap2 + samtools | 2.24 + 1.14 | |
| miniprot | 0.11--he4a0461_2 | |
| mummer | 3.23 | |
| paftools ( minimap2 + samtools ) | 2.24 + 1.14 | |
| pretextmap + samtools | 0.0.2 + 1.17 | |
| python | 3.9 | - |
| - pandas | 1.5.2 | - |
| samtools | 1.18 | 1.21 |
| selfcomp_splitfasta ( perl-bioperl ) | 1.7.8-1 | |
| seqtk | 1.4 | |
| tabix | 1.11 | |
| ucsc | 377 | |
| windowmasker (blast) | 2.14.0 | |
| Module | Old Version | New Versions |
| -------------------------------------- | ---------------- | ----------------- |
| bamtobed_sort ( bedtools + samtools ) | 2.31.0 + 1.17 | |
| bedtools | 2.31.1 | - |
| busco\* | 5.5.0 | - |
| bwa-mem2 | 2.2.1 | |
| cat | 2.3.4 | |
| chunk_fasta ( pyfasta ) | 0.5.2-1 | REMOVED |
| cooler | 0.9.2 | |
| cram_filter_align_bwamem2_fixmate_sort | - | |
| ^ ( samtools + bwamem2 ) ^ | 1.17 + 2.2.1 | |
| coreutils | 9.1 | |
| fastk | 1.0.1 | |
| gcc | 10.4.0 | |
| find_telomere_windows ( java-jdk ) | 8.0.112 | |
| generate_cram_csv ( samtools ) | 1.17 | |
| gnu-sort | 8.25 | 9.3 |
| juicer_tools_pre ( java-jdk ) | 8.0.112 | |
| perl | 5.26.2 | |
| merquryfk | 1.0.1 | |
| minimap2 + samtools | 2.24 + 1.14 | |
| minimap2_index | 2.24 | 2.28 |
| miniprot | 0.11--he4a0461_2 | |
| mummer | 3.23 | |
| paftools ( minimap2 + samtools ) | 2.24 + 1.14 | |
| pretextmap + samtools | 0.0.2 + 1.17 | 0.0.3 + 1.17 |
| python | 3.9 | - |
| - pandas | 1.5.2 | - |
| samtools | 1.18 | 1.21 |
| selfcomp_splitfasta ( perl-bioperl ) | 1.7.8-1 | |
| seqtk | 1.4 | |
| seqkit | ADDED | 2.9.0--h9ee0642_0 |
| tabix | 1.11 | |
| ucsc | 377 | 447 |
| windowmasker (blast) | 2.14.0 | 2.15.0 |

- busco is currently pinned to v5.5.0 - Upgrading v5.7.1 would cause github actions to crash. Further investigation needed.

## [1.1.1] - Ancient Aurora (H1) - [2024-04-26]

Expand Down
4 changes: 2 additions & 2 deletions CITATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,9 +112,9 @@
## Software packaging/containerisation tools

- [Anaconda](https://anaconda.com)
- [Conda](https://conda.org/)

> Anaconda Software Distribution. 2016. Computer software. Vers. 2-2.4.0. Anaconda, Web.
> conda contributors. conda: A system-level, binary package and environment manager running on all major operating systems and platforms. Computer software. https://github.com/conda/conda
- [Bioconda](https://pubmed.ncbi.nlm.nih.gov/29967506/)

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[![Cite with Zenodo](https://zenodo.org/badge/509096312.svg)](https://zenodo.org/doi/10.5281/zenodo.10047653)
[![Nextflow](https://img.shields.io/badge/nextflow%20DSL2-%E2%89%A522.10.1-23aa62.svg)](https://www.nextflow.io/)
[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=anaconda)](https://docs.conda.io/en/latest/)
[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=conda)](https://docs.conda.io/en/latest/)
[![run with docker](https://img.shields.io/badge/run%20with-docker-0db7ed?labelColor=000000&logo=docker)](https://www.docker.com/)
[![run with singularity](https://img.shields.io/badge/run%20with-singularity-1d355c.svg?labelColor=000000)](https://sylabs.io/docs/)
[![Launch on Nextflow Tower](https://img.shields.io/badge/Launch%20%F0%9F%9A%80-Nextflow%20Tower-%234256e7)](https://tower.nf/launch?pipeline=https://github.com/sanger-tol/treeval)
Expand Down
2 changes: 1 addition & 1 deletion conf/modules.config
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,7 @@ process {
}

withName: "BUSCO_BUSCO" {
ext.args = "--offline"
ext.args = "--offline --metaeuk"
}


Expand Down
15 changes: 7 additions & 8 deletions modules.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
},
"bedtools/genomecov": {
"branch": "master",
"git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83",
"git_sha": "bfa8975eefb8df3e480a44ac9e594f23f52b2963",
"installed_by": ["modules"]
},
"bedtools/intersect": {
Expand All @@ -27,7 +27,7 @@
},
"bedtools/map": {
"branch": "master",
"git_sha": "3b248b84694d1939ac4bb33df84bf6233a34d668",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"bedtools/merge": {
Expand Down Expand Up @@ -58,12 +58,12 @@
},
"cooler/cload": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"cooler/zoomify": {
"branch": "master",
"git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"]
},
"custom/dumpsoftwareversions": {
Expand Down Expand Up @@ -94,7 +94,7 @@
},
"minimap2/align": {
"branch": "master",
"git_sha": "603ecbd9f45300c9788f197d2a15a005685b4220",
"git_sha": "666652151335353eef2fcd58880bcef5bc2928e1",
"installed_by": ["modules"],
"patch": "modules/nf-core/minimap2/align/minimap2-align.diff"
},
Expand All @@ -115,9 +115,8 @@
},
"mummer": {
"branch": "master",
"git_sha": "9e51255c4f8ec69fb6ccf68593392835f14fecb8",
"installed_by": ["modules"],
"patch": "modules/nf-core/mummer/mummer.diff"
"git_sha": "a15872dde4c4affaac2dfee6c3c65fadab4719bd",
"installed_by": ["modules"]
},
"paftools/sam2paf": {
"branch": "master",
Expand Down
25 changes: 10 additions & 15 deletions modules/local/chunkfasta.nf → modules/local/seqkit/split/main.nf
Original file line number Diff line number Diff line change
@@ -1,48 +1,43 @@
process CHUNKFASTA {
process SEQKIT_SPLIT {
tag "${meta.id}"
label 'process_low'

conda "conda-forge::pyfasta=0.5.2-1"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/pyfasta:0.5.2--py_1' :
'biocontainers/pyfasta:0.5.2--py_1' }"
'https://depot.galaxyproject.org/singularity/seqkit:2.9.0--h9ee0642_0' :
'biocontainers/seqkit:2.9.0--h9ee0642_0' }"

input:
tuple val(meta), path('input.fasta')
val(number_of_chunks)

output:
tuple val(meta), path('*.fasta'), emit: fasta
path "versions.yml" , emit: versions
tuple val(meta), path('*.fa'), emit: fasta
path "versions.yml", emit: versions

script:
def VERSION = '0.5.2' // Tool does not report version
// This should be abstracted outside of the container to
// stop it spinning up in the first place,
// however dsl2 can't do comparisons with channels which makes it harder
"""
if [ $number_of_chunks -le 1 ]; then
mv input.fasta ${meta.id}_whole.fasta
mv input.fasta ${meta.id}_whole.fa
else
pyfasta split -n $number_of_chunks input.fasta
seqkit split input.fasta -p $number_of_chunks -O ./
fi
cat <<-END_VERSIONS > versions.yml
"${task.process}":
python: \$(python --version | sed 's/Python //g')
pyfasta: $VERSION
seqkit: \$(seqkit version | sed -e "s/seqkit v//g")
END_VERSIONS
"""

stub:
def VERSION = '0.5.2' // Tool does not report version
"""
touch ${meta.id}.fa
touch ${prefix}.fa
cat <<-END_VERSIONS > versions.yml
"${task.process}":
python: \$(python --version | sed 's/Python //g')
pyfasta: $VERSION
seqkit: \$(seqkit version | sed -e "s/seqkit v//g")
END_VERSIONS
"""
}
2 changes: 0 additions & 2 deletions modules/nf-core/bedtools/genomecov/environment.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 10 additions & 3 deletions modules/nf-core/bedtools/genomecov/main.nf

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

72 changes: 42 additions & 30 deletions modules/nf-core/bedtools/genomecov/meta.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 20045d6

Please sign in to comment.