diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md old mode 100644 new mode 100755 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2e226b4f..7b613b70 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -36,7 +36,7 @@ jobs: uses: tj-actions/branch-names@v8 - name: Install Nextflow - uses: nf-core/setup-nextflow@v1 + uses: nf-core/setup-nextflow@v2 with: version: "${{ matrix.NXF_VER }}" diff --git a/CHANGELOG.md b/CHANGELOG.md index 0170b9f2..dd0869b0 100755 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,84 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.2.0] - Ancient Destiny - [2024-11-15] + +Our 3rd release for sanger-tol/treeval. + +### Enhancements & Fixes + +- Togglable subworkflows +- Adds a JBrowse Only workflow (this will lead to an update to the FULL workflow which can now call JBROWSE_ONLY and RAPID). +- Updates to containers (local modules) to remove Anaconda dependencies following policy changes. +- Updates to modules to remove Anaconda dependencies following policy changes + - The majority of these updates only remove the `default` channel from the environment.yml +- CONDA warnings for modules which cannot use CONDA. +- Removable of a liberal use of spaces. +- reformat_intersect was previously not outputing version data. +- Adding arch specification to Pretext GitHub actions runner. Hopefully this will stop the spurious errors we see on there. +- Addition of steps into schema. +- Adds \*ktab as an output. +- Updated singularity containers +- Added `--metaeuk` to BUSCO_BUSCO, default was causing pipeline errors on Actions -- Needs more investigation. +- Replaced Pyfasta split (depreciated 6 years ago) with Seqkit split which is frequently updated and very fast. + +### Parameters + +| Old Parameter | New Parameter | +| ------------- | ------------- | +| - | --steps | + +### Software dependencies + +Note, since the pipeline is using Nextflow DSL2, each process will be run with its own Biocontainer. This means that on occasion it is entirely possible for the pipeline to be using different versions of the same tool. However, the overall software dependency changes compared to the last release have been listed below for reference. + +| Module | Old Version | New Versions | +| -------------------------------------- | ---------------- | ----------------- | +| bamtobed_sort ( bedtools + samtools ) | 2.31.0 + 1.17 | | +| bedtools | 2.31.1 | - | +| busco\* | 5.5.0 | - | +| bwa-mem2 | 2.2.1 | | +| cat | 2.3.4 | | +| chunk_fasta ( pyfasta ) | 0.5.2-1 | REMOVED | +| cooler | 0.9.2 | | +| cram_filter_align_bwamem2_fixmate_sort | - | | +| ^ ( samtools + bwamem2 ) ^ | 1.17 + 2.2.1 | | +| coreutils | 9.1 | | +| fastk | 1.0.1 | | +| gcc | 10.4.0 | | +| find_telomere_windows ( java-jdk ) | 8.0.112 | | +| generate_cram_csv ( samtools ) | 1.17 | | +| gnu-sort | 8.25 | 9.3 | +| juicer_tools_pre ( java-jdk ) | 8.0.112 | | +| perl | 5.26.2 | | +| merquryfk | 1.0.1 | | +| minimap2 + samtools | 2.24 + 1.14 | | +| minimap2_index | 2.24 | 2.28 | +| miniprot | 0.11--he4a0461_2 | | +| mummer | 3.23 | | +| paftools ( minimap2 + samtools ) | 2.24 + 1.14 | | +| pretextmap + samtools | 0.0.2 + 1.17 | 0.0.3 + 1.17 | +| python | 3.9 | - | +| - pandas | 1.5.2 | - | +| samtools | 1.18 | 1.21 | +| selfcomp_splitfasta ( perl-bioperl ) | 1.7.8-1 | | +| seqtk | 1.4 | | +| seqkit | ADDED | 2.9.0--h9ee0642_0 | +| tabix | 1.11 | | +| ucsc | 377 | 447 | +| windowmasker (blast) | 2.14.0 | 2.15.0 | + +- busco is currently pinned to v5.5.0 - Upgrading v5.7.1 would cause github actions to crash. Further investigation needed. + +## [1.1.1] - Ancient Aurora (H1) - [2024-04-26] + +### Enhancements & Fixes + +- Generate CRAM CSV fix to allow for multi-readgroup cram files +- Removing KMER_READCOV +- tmp directory was being used +- Output file adjustment (names and location) + ## [1.1.0] - Ancient Aurora - [2024-04-26] The second release for sanger-tol, created with the [nf-core](https://nf-co.re/) template. diff --git a/CITATIONS.md b/CITATIONS.md index d179e501..78d4983e 100755 --- a/CITATIONS.md +++ b/CITATIONS.md @@ -108,9 +108,9 @@ ## Software packaging/containerisation tools -- [Anaconda](https://anaconda.com) +- [Conda](https://conda.org/) - > Anaconda Software Distribution. 2016. Computer software. Vers. 2-2.4.0. Anaconda, Web. + > conda contributors. conda: A system-level, binary package and environment manager running on all major operating systems and platforms. Computer software. https://github.com/conda/conda - [Bioconda](https://pubmed.ncbi.nlm.nih.gov/29967506/) diff --git a/README.md b/README.md index ffaa43bc..a1a95647 100755 --- a/README.md +++ b/README.md @@ -1,13 +1,13 @@ [![Cite with Zenodo](https://zenodo.org/badge/509096312.svg)](https://zenodo.org/doi/10.5281/zenodo.10047653) [![Nextflow](https://img.shields.io/badge/nextflow%20DSL2-%E2%89%A522.10.1-23aa62.svg)](https://www.nextflow.io/) -[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=anaconda)](https://docs.conda.io/en/latest/) +[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=conda)](https://docs.conda.io/en/latest/) [![run with docker](https://img.shields.io/badge/run%20with-docker-0db7ed?labelColor=000000&logo=docker)](https://www.docker.com/) [![run with singularity](https://img.shields.io/badge/run%20with-singularity-1d355c.svg?labelColor=000000)](https://sylabs.io/docs/) [![Launch on Nextflow Tower](https://img.shields.io/badge/Launch%20%F0%9F%9A%80-Nextflow%20Tower-%234256e7)](https://tower.nf/launch?pipeline=https://github.com/sanger-tol/treeval) ## Introduction -**sanger-tol/treeval [1.1.0 - Ancient Aurora]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass. +**sanger-tol/treeval [1.2.0 - Ancient Destiny-]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass. The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from [nf-core/modules](https://github.com/nf-core/modules) in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community! @@ -80,8 +80,6 @@ If you would like to contribute to this pipeline, please see the [contributing g ## Citations - - If you use sanger-tol/treeval for your analysis, please cite it using the following doi: [10.5281/zenodo.10047653](https://doi.org/10.5281/zenodo.10047653). ### Tools diff --git a/assets/github_testing/TreeValTinyFullTest.yaml b/assets/github_testing/TreeValTinyFullTest.yaml index c7ade25e..8e215cc8 100755 --- a/assets/github_testing/TreeValTinyFullTest.yaml +++ b/assets/github_testing/TreeValTinyFullTest.yaml @@ -19,10 +19,8 @@ kmer_profile: kmer_length: 31 dir: /home/runner/work/treeval/treeval/TreeValTinyData/ alignment: - data_dir: /home/runner/work/treeval/treeval/TreeValTinyData/gene_alignment_data/ - common_name: "" # For future implementation (adding bee, wasp, ant etc) - geneset_id: "LaetiporusSulphureus.gfLaeSulp1" - #Path should end up looking like "{data_dir}{classT}/{common_name}/csv_data/{geneset}-data.csv" + genesets: + - /home/runner/work/treeval/treeval/TreeValTinyData/gene_alignment_data/fungi/csv_data/LaetiporusSulphureus.gfLaeSulp1-data.csv self_comp: motif_len: 0 mummer_chunk: 10 @@ -31,8 +29,7 @@ intron: telomere: teloseq: TTAGGG synteny: - synteny_path: /home/runner/work/treeval/treeval/treeval/TreeValTinyData/synteny - synteny_genomes: "LaetiporusSulphureus" + - /home/runner/work/treeval/treeval/TreeValTinyData/synteny/fungi/LaetiporusSulphureus.fasta busco: lineages_path: /home/runner/work/treeval/treeval/TreeValTinyData/busco/subset/ lineage: fungi_odb10 diff --git a/assets/local_testing/nxOscDF5033-BGA.yaml b/assets/local_testing/nxOscDF5033-BGA.yaml index 3338674d..b0c71b3e 100755 --- a/assets/local_testing/nxOscDF5033-BGA.yaml +++ b/assets/local_testing/nxOscDF5033-BGA.yaml @@ -11,8 +11,8 @@ assem_reads: hic: /workspace/treeval-curation/Oscheius_DF5033/hic-arima2/ supplementary: path # Not currently in use alignment: - data_dir: /workspace/treeval-curation/gene_alignment_data/ - geneset: "OscheiusTipulae.ASM1342590v1,CaenorhabditisElegans.WBcel235,Gae_host.Gae" + genesets: + - /lustre/scratch123/tol/resources/treeval/gene_alignment_data/nematode/csv_data/OscheiusTipulae.ASM1342590v1-data.csv self_comp: motif_len: 0 mummer_chunk: 10 @@ -21,8 +21,7 @@ intron: telomere: teloseq: TTAGGG synteny: - synteny_path: /nfs/treeoflife-01/teams/tola/users/dp24/treeval/TreeValTinyData/synteny/ - synteny_genomes: "LaetiporusSulphureus" + - /nfs/treeoflife-01/teams/tola/users/dp24/treeval/TreeValTinyData/synteny/fungi/LaetiporusSulphureus.fasta busco: lineages_path: /workspace/treeval-curation/busco/v5 lineage: nematoda_odb10 diff --git a/assets/local_testing/nxOscDF5033.yaml b/assets/local_testing/nxOscDF5033.yaml index 92e0a899..5c0493b5 100755 --- a/assets/local_testing/nxOscDF5033.yaml +++ b/assets/local_testing/nxOscDF5033.yaml @@ -19,10 +19,10 @@ kmer_profile: kmer_length: 31 dir: /lustre/scratch123/tol/resources/treeval/treeval-testdata/TreeValSmallData/Oscheius_DF5033/genomic_data/nxOscSpes1/pacbio/ alignment: - data_dir: /lustre/scratch123/tol/resources/treeval/gene_alignment_data/ - common_name: "" # For future implementation (adding bee, wasp, ant etc) - geneset_id: "OscheiusTipulae.ASM1342590v1,CaenorhabditisElegans.WBcel235,Gae_host.Gae" - #Path should end up looking like "{data_dir}{classT}/{common_name}/csv_data/{geneset}-data.csv" + genesets: + - /lustre/scratch123/tol/resources/treeval/gene_alignment_data/nematode/csv_data/OscheiusTipulae.ASM1342590v1-data.csv + - /lustre/scratch123/tol/resources/treeval/gene_alignment_data/nematode/csv_data/CaenorhabditisElegans.WBcel235-data.csv + - /lustre/scratch123/tol/resources/treeval/gene_alignment_data/nematode/csv_data/Gae_host.Gae-data.csv self_comp: motif_len: 0 mummer_chunk: 10 @@ -31,8 +31,7 @@ intron: telomere: teloseq: TTAGGG synteny: - synteny_path: /nfs/treeoflife-01/teams/tola/users/dp24/treeval/TreeValTinyData/synteny/ - synteny_genomes: "" + - /lustre/scratch123/tol/resources/treeval/synteny/bird/bCucCan1.fasta busco: lineages_path: /lustre/scratch123/tol/resources/busco/v5 lineage: nematoda_odb10 diff --git a/assets/local_testing/nxOscSUBSET.yaml b/assets/local_testing/nxOscSUBSET.yaml index 604f09dc..6c3fbd49 100755 --- a/assets/local_testing/nxOscSUBSET.yaml +++ b/assets/local_testing/nxOscSUBSET.yaml @@ -19,10 +19,8 @@ kmer_profile: kmer_length: 31 dir: /lustre/scratch123/tol/resources/treeval/treeval-testdata/TreeValSmallData/Oscheius_DF5033/genomic_data/nxOscSpes1/pacbio/ alignment: - data_dir: /lustre/scratch123/tol/resources/treeval/gene_alignment_data/ - common_name: "" # For future implementation (adding bee, wasp, ant etc) - geneset_id: "Gae_host.Gae" - #Path should end up looking like "{data_dir}{classT}/{common_name}/csv_data/{geneset}-data.csv" + genesets: + - /lustre/scratch123/tol/resources/treeval/gene_alignment_data/nematode/csv_data/OscheiusTipulae.ASM1342590v1-data.csv self_comp: motif_len: 0 mummer_chunk: 10 @@ -31,8 +29,8 @@ intron: telomere: teloseq: TTAGGG synteny: - synteny_path: /nfs/treeoflife-01/teams/tola/users/dp24/treeval/TreeValTinyData/synteny/ - synteny_genomes: "" + - /lustre/scratch123/tol/resources/treeval/synteny/bird/bCucCan1.fasta + - /lustre/scratch123/tol/resources/treeval/synteny/bird/bGalGal1.fasta busco: lineages_path: /lustre/scratch123/tol/resources/busco/v5 lineage: nematoda_odb10 diff --git a/bin/FKprof b/bin/FKprof deleted file mode 100755 index 891e81f7..00000000 Binary files a/bin/FKprof and /dev/null differ diff --git a/bin/awk_filter_reads.sh b/bin/awk_filter_reads.sh index d50aa9ec..54857762 100755 --- a/bin/awk_filter_reads.sh +++ b/bin/awk_filter_reads.sh @@ -1 +1,7 @@ -awk 'BEGIN{OFS="\t"}{if($1 ~ /^\@/) {print($0)} else {$2=and($2,compl(2048)); print(substr($0,2))}}' +version='1.0.0' +if [ $1 == '-v' ]; +then + echo "$version" +else + awk 'BEGIN{OFS="\t"}{if($1 ~ /^\@/) {print($0)} else {$2=and($2,compl(2048)); print(substr($0,2))}}' +fi diff --git a/bin/bed_to_contacts.sh b/bin/bed_to_contacts.sh index b2e884ca..a49be321 100755 --- a/bin/bed_to_contacts.sh +++ b/bin/bed_to_contacts.sh @@ -1,2 +1,9 @@ #!/bin/bash -paste -d '\t' - - < $1 | awk 'BEGIN {FS="\t"; OFS="\t"} {if ($1 > $7) {print substr($4,1,length($4)-2),$12,$7,$8,"16",$6,$1,$2,"8",$11,$5} else {print substr($4,1,length($4)-2),$6,$1,$2,"8",$12,$7,$8,"16",$5,$11} }' | tr '\-+' '01' | sort -k3,3d -k7,7d | awk 'NF==11' + +version='1.0.0' +if [ $1 == '-v' ]; +then + echo "$version" +else + paste -d '\t' - - < $1 | awk 'BEGIN {FS="\t"; OFS="\t"} {if ($1 > $7) {print substr($4,1,length($4)-2),$12,$7,$8,"16",$6,$1,$2,"8",$11,$5} else {print substr($4,1,length($4)-2),$6,$1,$2,"8",$12,$7,$8,"16",$5,$11} }' | tr '\-+' '01' | sort -k3,3d -k7,7d | awk 'NF==11' +fi diff --git a/bin/find_telomere b/bin/find_telomere deleted file mode 100755 index b7d84a15..00000000 Binary files a/bin/find_telomere and /dev/null differ diff --git a/bin/generate_cram_csv.sh b/bin/generate_cram_csv.sh index 069c44e1..962f6956 100755 --- a/bin/generate_cram_csv.sh +++ b/bin/generate_cram_csv.sh @@ -71,6 +71,11 @@ if [ -z "$1" ]; then exit 1 fi +if [ $1 == "-v" ]; then + echo "1.0" + exit 1 +fi + cram_path=$1 chunkn=0 outcsv=$2 diff --git a/bin/get_avgcov.sh b/bin/get_avgcov.sh index bea3da09..9f25cb35 100755 --- a/bin/get_avgcov.sh +++ b/bin/get_avgcov.sh @@ -8,7 +8,7 @@ # Author = yy5 # ------------------- version='1.0.0' -if [ $1 == '-v']; +if [ $1 == '-v' ]; then echo "$version" else diff --git a/bin/get_busco_gene.sh b/bin/get_busco_gene.sh index 84ed187a..4d9b92d2 100755 --- a/bin/get_busco_gene.sh +++ b/bin/get_busco_gene.sh @@ -11,5 +11,10 @@ # Update for BUSCO 5.5.0 - by we3 # Reorder start and end so smallest always second column. Also, trim range from scaffold name in first column. # ------------------- - -cat $1| grep -v '#'|awk '$2!="Missing"'| awk '{if($4>$5){print $3"\t"$5"\t"$4"\t"$1"\t"$7"\t"$6"\t"$9}else{print $3"\t"$4"\t"$5"\t"$1"\t"$7"\t"$6"\t"$9}}'| awk -F'\t' -v OFS='\t' '{if($7==""){$7="no_orthodb_link"}; sub(/:.*/,"",$1);print $1,$2,$3,$4,$5,$6,$7}' +version='1.0.0' +if [ $1 == '-v' ]; +then + echo "$version" +else + cat $1| grep -v '#'|awk '$2!="Missing"'| awk '{if($4>$5){print $3"\t"$5"\t"$4"\t"$1"\t"$7"\t"$6"\t"$9}else{print $3"\t"$4"\t"$5"\t"$1"\t"$7"\t"$6"\t"$9}}'| awk -F'\t' -v OFS='\t' '{if($7==""){$7="no_orthodb_link"}; sub(/:.*/,"",$1);print $1,$2,$3,$4,$5,$6,$7}' +fi diff --git a/bin/grep_pg.sh b/bin/grep_pg.sh index 680b5ec2..f8e6a7fb 100755 --- a/bin/grep_pg.sh +++ b/bin/grep_pg.sh @@ -7,4 +7,10 @@ # ------------------- # Author = yy5 -grep -v "^\@PG" | awk '{if($1 ~ /^\@/) {print($0)} else {if(and($2,64)>0) {print(1$0)} else {print(2$0)}}}' +version='1.0.0' +if [ $1 == '-v' ]; +then + echo "$version" +else + grep -v "^\@PG" | awk '{if($1 ~ /^\@/) {print($0)} else {if(and($2,64)>0) {print(1$0)} else {print(2$0)}}}' +fi diff --git a/bin/paf_to_bed.sh b/bin/paf_to_bed.sh index 7de63334..28e72eb9 100755 --- a/bin/paf_to_bed.sh +++ b/bin/paf_to_bed.sh @@ -10,7 +10,7 @@ version='1.0.0' -if [ $1 == '-v']; +if [ $1 == '-v' ]; then echo "$version" else diff --git a/conf/base.config b/conf/base.config index 93360fc4..a057a3a5 100755 --- a/conf/base.config +++ b/conf/base.config @@ -88,6 +88,7 @@ process { withName:SAMTOOLS_MERGE { cpus = { check_max( 16 * 1, 'cpus' ) } memory = { check_max( 50.GB * task.attempt, 'memory') } + time = { check_max( 30.h * task.attempt, 'time') } } // RESOURCES: MEMORY INTENSIVE STEPS, SOFTWARE TO BE UPDATED TO COMBAT THIS @@ -163,7 +164,7 @@ process { } withName: CRAM_FILTER_ALIGN_BWAMEM2_FIXMATE_SORT { cpus = { check_max( 16 * 1 , 'cpus' ) } - memory = { check_max( 1.GB * ( reference.size() < 2e9 ? 50 : Math.ceil( ( reference.size() / 1e+9 ) * 20 ) * Math.ceil( task.attempt * 1 ) ) , 'memory') } + memory = { check_max( 1.GB * ( reference.size() < 2e9 ? 80 : Math.ceil( ( reference.size() / 1e+9 ) * 30 ) * Math.ceil( task.attempt * 1 ) ) , 'memory') } } withName: CRAM_FILTER_MINIMAP2_FILTER5END_FIXMATE_SORT { @@ -179,11 +180,13 @@ process { withName: PRETEXTMAP_STANDRD{ cpus = { check_max( 8 * 1, 'cpus' ) } memory = { check_max( 3.GB * task.attempt, 'memory' ) } + time = { check_max( 1.h * ( ( fasta.size() < 4e9 ? 24 : 48 ) * Math.ceil( task.attempt * 1 ) ), 'time' ) } } withName: PRETEXTMAP_HIGHRES { cpus = { check_max( 6 * task.attempt, 'cpus' ) } memory = { check_max( 20.GB * Math.ceil( task.attempt * 2.6 ), 'memory' ) } + time = { check_max( 1.h * ( ( fasta.size() < 4e9 ? 24 : 48 ) * Math.ceil( task.attempt * 1 ) ), 'time' ) } } withName: PRETEXT_GRAPH { @@ -207,7 +210,8 @@ process { // add a cpus 16 if bam.size() >= 50GB withName: BAMTOBED_SORT { cpus = { check_max( 12 * 1, 'cpus' ) } - memory = { check_max( 3.GB * Math.ceil( bam.size() / 1e+9 ) * task.attempt, 'memory' ) } + memory = { check_max( 2.GB * ( ( bam.size() < 150e9 ? Math.ceil( bam.size() / 1e+9 ) : Math.ceil( bam.size() / 4e+9 ) ) * Math.ceil( task.attempt * 1 ) ), 'memory' ) } + time = { check_max( 30.h * task.attempt, 'time' ) } } withName: SAMTOOLS_MARKDUP { @@ -217,7 +221,7 @@ process { withName: COOLER_CLOAD { cpus = { check_max( 16 * 1, 'cpus' ) } - memory = { check_max( 6.GB * task.attempt, 'memory' ) } + memory = { check_max( 20.GB * task.attempt, 'memory' ) } } withName: MERQURYFK_MERQURYFK { @@ -225,14 +229,14 @@ process { memory = { check_max( 100.GB * task.attempt, 'memory' ) } } - withName: BUSCO { + withName: BUSCO_BUSCO { cpus = { check_max( 16 * task.attempt, 'cpus' ) } memory = { check_max( 50.GB * task.attempt, 'memory' ) } time = { check_max( 20.h * task.attempt, 'time' ) } } // Large Genomes > 4Gb - //withName: BUSCO { + //withName: BUSCO_BUSCO { // cpus = { check_max( 30 * task.attempt, 'cpus' ) } // memory = { check_max( 100.GB * task.attempt, 'memory' ) } // time = { check_max( 300.h * task.attempt, 'time' ) } @@ -244,12 +248,6 @@ process { memory = { check_max( 100.GB * task.attempt, 'memory' ) } } - withName: FKUTILS_FKPROF { - cpus = { check_max( 25 * task.attempt, 'cpus' ) } - memory = { check_max( 1.GB * ( reference.size() < 2e9 ? 50 : Math.ceil( ( reference.size() / 1e+9 ) * 20 ) * Math.ceil( task.attempt * 1 ) ), 'memory' ) } - time = { check_max( 36.h * task.attempt, 'time' ) } - } - // // GENERAL MODULE LIMITS // Based on reports from SummaryStats @@ -278,6 +276,7 @@ process { withName: GET_PAIRED_CONTACT_BED { cpus = { check_max( ${ file.size() > 1e11 ? 12 : 6 } , 'cpus' ) } memory = { check_max( 1.GB * Math.ceil( file.size() / 2e+9 ) * task.attempt , 'memory' ) } + time = { check_max( 30.h * task.attempt, 'time' ) } } // @@ -308,4 +307,10 @@ process { withName: BEDTOOLS_INTERSECT { memory = { check_max( 6.GB * (task.attempt * task.attempt), 'memory' ) } } + + withName: GENERATE_CRAM_CSV { + cpus = { check_max( 6 , 'cpus' ) } + memory = { check_max( 30.GB * task.attempt , 'memory' ) } + time = { check_max( 10.h * task.attempt , 'time' ) } + } } diff --git a/conf/modules.config b/conf/modules.config index dafd5469..fb2ee8ec 100755 --- a/conf/modules.config +++ b/conf/modules.config @@ -57,6 +57,14 @@ process { ] } + withName: FASTK_FASTK { + publishDir = [ + path: { "${params.outdir}/hic_files" }, + mode: params.publish_dir_mode, + pattern: "*.ktab*" + ] + } + // // MODULE CONFIGS // @@ -307,8 +315,8 @@ process { ext.prefix = { "${meta.id}_ancestral" } } - withName: "BUSCO" { - ext.args = "--offline" + withName: "BUSCO_BUSCO" { + ext.args = "--offline --metaeuk" } @@ -336,6 +344,15 @@ process { ext.prefix = { "${meta.id}_hr" } } + + withName: YAHS { + publishDir = [ + path: { "${params.outdir}/hic_files/alignment_bin" }, + mode: params.publish_dir_mode + ] + } + + withName: "SNAPSHOT_SRES" { ext.args = { "--sequences '=full' --resolution 1440" } ext.prefix = { "${meta.id}_normal" } diff --git a/docs/output.md b/docs/output.md index 95c52f66..db8e6dba 100755 --- a/docs/output.md +++ b/docs/output.md @@ -213,6 +213,7 @@ This subworkflow performs a k-mer count using [FASTK_FASTK](https://nf-co.re/mod - `hic_files/` - `*.ref.spectra-cn.ln.png`: .png file of copy number k-mer spectra. + - `*.ktab*`: K-mer table files. diff --git a/docs/usage.md b/docs/usage.md index 50d3db59..fa94c02f 100755 --- a/docs/usage.md +++ b/docs/usage.md @@ -8,7 +8,7 @@ The TreeVal pipeline has a few requirements before being able to run: -- The `gene_alignment_data` (this refers to the alignment.data_dir and alignment.geneset data noted in the yaml, which we will explain later) and `synteny_data` follow a particular directory structure. +- The `gene_alignment_data` requires a specific .csv format. - HiC CRAM files must be pre-indexed in the same location as the CRAM file, e.g., `samtools index {cram file}`. A check and automated indexing of the cram file will be added in the future. @@ -16,7 +16,7 @@ The TreeVal pipeline has a few requirements before being able to run: ## Prior to running TreeVal -:warning: Please ensure you read the following sections on Directory Structure (`gene_alignment_data`, `synteny`, scripts), HiC data prep and Pacbio data prep. Without these you may not be able to successfully run the TreeVal pipeline. If nothing is clear then please leave an issue report. +:warning: Please ensure you read the following sections on Directory Structure (`gene_alignment_data` and scripts), HiC data prep and Pacbio data prep. Without these you may not be able to successfully run the TreeVal pipeline. If nothing is clear then please leave an issue report. We now also support ( and encourage ) using the nf-co2footprint plugin (on Nextflow versions >= 23.07) which generates statistics on how much energy your pipeline uses as well as the amount of Co2 it helps produce. As it is pre-release, you will need to compile this plugin your self and store it in your `$NXF_HOME/plugins` directory, which you can find with `echo $NXF_HOME`. We have included the relevant config file `co2footprint.config` in this repo. The plugin can be used be including `-plugins nf-co2footprint@{VERSION} -c co2footprint.config` in your nextflow command. Please head to the website to find out more [NF-CO2FOOTPRINT](https://nextflow-io.github.io/nf-co2footprint/contributing/setup/). @@ -44,110 +44,16 @@ You should now be able to run the pipeline as you see fit. -### Directory Structure +### Gene Alignment and Synteny Data
Details -The working example found below in the Gene alignment and synteny sections (below), will cover setting up the `synteny` and `gene_alignment_data` directories as well as downloading some example data. +#### Step 1 -- Preparing Synteny data -These two sub-workflows, for now, need the use of the variables `defined_class`, `synteny_genome_path`, `data_dir` and `geneset`. These variables are found inside the yaml ( this is the file that will tell TreeVal what and where everything is ). Currently, we don't use `common_name`, e.g., `bee`, `wasp`, `moth`, etc. However, we hope to make use of it in the future as our `gene_alignment_data` "database" grows and requires a higher degree of organisation. +For synteny you should provide the full genomic fasta file, of any high quality genome you want to be compared against. -First, you should set up a directory in our recommended structure: - -``` -treeval-resources -│ -├─ gene_alignment_data/ -│ ├─ { defined_class } -│ ├─ csv_data -│ │ └─ { Organism.Accession }-data.csv # Generated by our scripts -│ ├─ { Organism } # Here and below is generated by our scripts -│ ├─ { Organism.Accession } -│ ├─ cdna -│ │ └─ { Chunked fasta files } -│ ├─ rna -│ │ └─ { Chunked fasta files } -│ ├─ cds -│ │ └─ { Chunked fasta files } -│ └─ peps -│ └─ { Chunked fasta files } -│ -├─ gene_alignment_prep/ -│ ├─ scripts/ # We supply these in this repo -│ ├─ raw_fasta/ # Storing your fasta downloaded from NCBI or Ensembl -│ └─ treeval-datasets.tsv # Organism, common_name, clade, family, group, link_to_data, notes -│ -├─ synteny/ -│ └─ {defined_class} -│ -├─ treeval_yaml/ # Storage folder for you yaml files, it's useful to keep them -│ -└─ treeval_stats/ # Storage for you treeval output stats file whether for upload to our repo -``` - -The above naming will be further explained - -`defined_class` can be your own system of classification, as long as it is consistent. At Sanger we use the below, we advise you do too. Again, the value that is entered into the yaml (the file we will use to tell TreeVal where everything is), is used to find `gene_alignment_data` as well as syntenic genomes. - -![defined_class](../docs/images/Sanger-classT.png) - -
- -### Synteny - -
- Details - -For synteny you should store the full genomic fasta file, of any high quality genome you want to be compared against, in teh above created directory. - -For bird we recommend the Golden Eagle ( _Aquila chrysaetos_ ) and the Zebrafinch (_Taeniopygia guttata_), which can be downloaded from NCBI. Rename, these files to something more human readable, and drop them into the `synteny/bird/` folder. Any TreeVal run you now perform where the `defined_class` is bird will run a syntenic alignment against all genomes in that folder. It would be best to keep this to around three unless needed. Again, this is something we could expand on with the `common_name` field if requested in the future. - -
- -### Gene Alignment and Synteny Data and Directories - -
- Details - -Seeing as this can be quite a complicated to set up, here's a walk through. - -#### Step 1 - Set up the directories - -Lets set up the directory structure as if we want to run it on a bird genome. - -``` -mkdir -p gene_alignment_prep/scripts/ - -cp treeval/bin/treeval-dataprep/* gene_alignment_prep/scripts/ - -mkdir -p gene_alignment_prep/raw_fasta/ - -mkdir -p gene_alignment_data/bird/csv_data/ - -mkdir -p synteny/bird/ -``` - -The naming of the bird folder here is important, keep this in mind. - -So now we have this structure: - -``` -~/treeval-resources - │ - ├─ synteny/ - │ └─ bird/ - │ - ├─ gene_alignment_data/ - │ └─ bird/ - │ └─ csv_data/ - │ - └─ gene_alignment_prep/ - ├─ scripts/ - └─ raw_fasta/ -``` - -#### Step 2 - Download some data +For bird we recommend the Golden Eagle ( _Aquila chrysaetos_ ) and the Zebrafinch (_Taeniopygia guttata_), which can be downloaded from NCBI. Now, let's download our syntenic alignment data. I think the Zebrafinch (_Taeniopygia guttata_) would be good against the Chicken (_Gallus gallus_). @@ -163,6 +69,8 @@ This leaves us with a file called `bTaeGut1_4.fasta` the genomic assembly of `bT Now lets move into the `raw_data` folder and download some data, this may take some time. +#### Step 1 -- Preparing Gene alignment data + ``` cd ../../gene_alignment_prep/raw_data/ @@ -181,6 +89,8 @@ Now that's all downloaded we need to prep it. At this point it is all still gzip The below code will look through the current folder for files ending with `.fasta.gz` and decompresses it, it will then run our python script, `GA_data_prep.py`. +NOTE: This will soon be superseeded by a rust tools `treeval_utils`, this will replace all of the accessory pythons scripts included in this document. Whilst also adding a program to verify your YAML. + Command: ```bash @@ -276,40 +186,18 @@ This is all useful for the pipeline which generates job ids based on the org col #### Step 4 -- Understand where we are at -So we have now generated the directory structure for `gene_alignment_data`. So now let's use what we know to fill out the yaml. +Now let's use what we know to fill out the yaml. The yaml is a file that we need in order to tell the pipeline where everything is, an example can be found [here](https://raw.githubusercontent.com/sanger-tol/treeval/dev/assets/local_testing/nxOscDF5033.yaml). -Here we can see a number of fields that need to be filled out, the easiest being `synteny_genome_path` and `data_dir`. These refer to the directories we made earlier so we can replace them as such: - ```yaml alignment: - data_dir: /FULL/PATH/TO/treeval-resources/gene_alignment_data/ - -synteny_genome_path: /FULL/PATH/TO/treeval-resources/synteny + genesets: + - /FULL/PATH/TO/-data.csv +synteny: + - /FULL/PATH/TO/.fasta ``` -I said earlier that the the fact we called a folder `bird` was important, this is because it now becomes our `defined_class`: - -```yaml -defined_class: bird -``` - -During pipeline execution, this is appended onto the end of `data_dir` and `synteny_genome_path` in order to find the correct files to use. So now all of the files inside `/FULL/PATH/TO/treeval-resources/synteny/bird/ ` will be used for syntenic alignments. Likewise with our `alignment.data_dir`, TreeVal will turn this into `/FULL/PATH/TO/treeval-resources/gene_alignment_data/bird/` and then appends `csv_data/`. - -In Step 3, we generated some files which will be living in our `/FULL/PATH/TO/treeval-resources/gene_alignment_data/bird/csv_data/` folder and look like `GallusGallus.GRCg7b-data.csv`. These (minus the `-data.csv`) will be what we enter into the `geneset` field in the yaml. The `common_name` is a field we don't currently use. - -```yaml -alignment: - data_dir: /FULL/PATH/TO/treeval-resources/gene_alignment_data/ - common_name: "" # For future implementation (adding bee, wasp, ant etc) - geneset: "GallusGallus.GRCg7b" -``` - -However, what is cool about this field (geneset) is that you can add as many as you want. So say you have the `alignment.data_dir` for the Finch saved as `TaeniopygiaGuttata.bTaeGut1_4`. The geneset field becomes: `geneset: "GallusGallus.GRCg7b,TaeniopygiaGuttata.bTaeGut1_4"` - -Hopefully this explains things a bit better and you understand how this sticks together! -
### HiC data Preparation @@ -348,7 +236,7 @@ If your data isn't already in these formats, then let us know and we'll see how This command iterates through your bam files and converts them to fastq via samtools. -``` +```bash cd { TO FOLDER OF BAM FILES } mkdir fastq for i in *bam @@ -380,6 +268,7 @@ done This simply gzips (compresses) the fasta files. ```bash +cd fasta for i in .fasta; do echo $i gzip $i @@ -445,9 +334,7 @@ The following is an example YAML file we have used during production: [nxOscDF50 - `hic_cram`: path (ending with `/`) to folder containing cram files. - `hic_aligner`: choice between `bwamam2` and `minimap2` - `alignment` - - `data_dir`: Gene alignment data path (ending with `/`). - - `common_name`: For future implementation (adding bee, wasp, ant etc) - - `geneset_id`: a csv list of geneset data to be used + - `genesets`: List of Gene alignment data .csv file paths. - `kmer_profile`: - `kmer_length`: length of kmer to be used in plotting - `dir`: directory containing old plot to be regenerated if applicable @@ -455,7 +342,7 @@ The following is an example YAML file we have used during production: [nxOscDF50 - `motif_len`: Length of motif to be used in self complementary sequence finding - `mummer_chunk`: Size of chunks used by MUMMER module. - `synteny` - - `synteny_genome_path`: Path to syntenic genomes grouped by clade. + - List of paths to syntenic genomes grouped by clade. - `outdir`: Will be required in future development. - `intron:` - `size`: base pair size of introns default is 50k diff --git a/lib/Utils.groovy b/lib/Utils.groovy index 8d030f4e..13cb02df 100755 --- a/lib/Utils.groovy +++ b/lib/Utils.groovy @@ -22,7 +22,7 @@ class Utils { // Check that all channels are present // This channel list is ordered by required channel priority. - def required_channels_in_order = ['conda-forge', 'bioconda', 'defaults'] + def required_channels_in_order = ['conda-forge', 'bioconda'] def channels_missing = ((required_channels_in_order as Set) - (channels as Set)) as Boolean // Check that they are in the right order diff --git a/modules.json b/modules.json old mode 100755 new mode 100644 index e227d47c..5d6662aa --- a/modules.json +++ b/modules.json @@ -7,192 +7,199 @@ "nf-core": { "bedtools/bamtobed": { "branch": "master", - "git_sha": "1d1cb7bfef6cf67fbc7faafa6992ad8bdc3045b3", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "bedtools/genomecov": { "branch": "master", - "git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83", + "git_sha": "bfa8975eefb8df3e480a44ac9e594f23f52b2963", "installed_by": ["modules"] }, "bedtools/intersect": { "branch": "master", - "git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "bedtools/makewindows": { "branch": "master", - "git_sha": "3b248b84694d1939ac4bb33df84bf6233a34d668", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "bedtools/map": { "branch": "master", - "git_sha": "3b248b84694d1939ac4bb33df84bf6233a34d668", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "bedtools/merge": { "branch": "master", - "git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "bedtools/sort": { "branch": "master", - "git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, - "busco": { + "busco/busco": { "branch": "master", - "git_sha": "e3126f437c336c826f242842fe51769cfce0ec2d", - "installed_by": ["modules"] + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", + "installed_by": ["modules"], + "patch": "modules/nf-core/busco/busco/busco-busco.diff" }, "bwamem2/index": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "cat/cat": { "branch": "master", - "git_sha": "81f27e75847087865299cc46605deb3b09b4e0a2", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "cooler/cload": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "cooler/zoomify": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "custom/dumpsoftwareversions": { "branch": "master", - "git_sha": "8ec825f465b9c17f9d83000022995b4f7de6fe93", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "custom/getchromsizes": { "branch": "master", - "git_sha": "1b0ffa4e5aed5b7e3cd4311af31bd3b2c8345051", + "git_sha": "49f4e50534fe4b64101e62ea41d5dc43b1324358", "installed_by": ["modules"], "patch": "modules/nf-core/custom/getchromsizes/custom-getchromsizes.diff" }, "fastk/fastk": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", - "installed_by": ["modules"] + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", + "installed_by": ["modules"], + "patch": "modules/nf-core/fastk/fastk/fastk-fastk.diff" }, "gnu/sort": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "merquryfk/merquryfk": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", - "installed_by": ["modules"] + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", + "installed_by": ["modules"], + "patch": "modules/nf-core/merquryfk/merquryfk/merquryfk-merquryfk.diff" }, "minimap2/align": { "branch": "master", - "git_sha": "603ecbd9f45300c9788f197d2a15a005685b4220", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"], "patch": "modules/nf-core/minimap2/align/minimap2-align.diff" }, "minimap2/index": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "miniprot/align": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "miniprot/index": { "branch": "master", - "git_sha": "8d737766e8f3c1417212b4b56acb959f3c356d26", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "mummer": { "branch": "master", - "git_sha": "9e51255c4f8ec69fb6ccf68593392835f14fecb8", - "installed_by": ["modules"], - "patch": "modules/nf-core/mummer/mummer.diff" + "git_sha": "a15872dde4c4affaac2dfee6c3c65fadab4719bd", + "installed_by": ["modules"] }, "paftools/sam2paf": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "pretextmap": { "branch": "master", - "git_sha": "decfb802f2e573efb7b44ff06b11ecf16853054d", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"], "patch": "modules/nf-core/pretextmap/pretextmap.diff" }, "pretextsnapshot": { "branch": "master", - "git_sha": "911696ea0b62df80e900ef244d7867d177971f73", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"], "patch": "modules/nf-core/pretextsnapshot/pretextsnapshot.diff" }, "samtools/faidx": { "branch": "master", - "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c", + "git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208", "installed_by": ["modules"] }, "samtools/index": { "branch": "master", - "git_sha": "a64788f5ad388f1d2ac5bd5f1f3f8fc81476148c", + "git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208", "installed_by": ["modules"] }, "samtools/markdup": { "branch": "master", - "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c", + "git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208", "installed_by": ["modules"] }, "samtools/merge": { "branch": "master", - "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c", + "git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208", "installed_by": ["modules"] }, "samtools/sort": { "branch": "master", - "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c", + "git_sha": "b7800db9b069ed505db3f9d91b8c72faea9be17b", "installed_by": ["modules"] }, "samtools/view": { "branch": "master", - "git_sha": "ce0b1aed7d504883061e748f492a31bf44c5777c", + "git_sha": "669eb24fd82a9d3cb18ad0e73673ecb26827f683", "installed_by": ["modules"] }, "seqtk/cutn": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "tabix/bgziptabix": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "f448e846bdadd80fc8be31fbbc78d9f5b5131a45", "installed_by": ["modules"] }, "ucsc/bedgraphtobigwig": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "49f4e50534fe4b64101e62ea41d5dc43b1324358", "installed_by": ["modules"] }, "ucsc/bedtobigbed": { "branch": "master", - "git_sha": "9e51255c4f8ec69fb6ccf68593392835f14fecb8", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "windowmasker/mkcounts": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", "installed_by": ["modules"] }, "windowmasker/ustat": { "branch": "master", - "git_sha": "3f5420aa22e00bd030a2556dfdffc9e164ec0ec5", + "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", + "installed_by": ["modules"] + }, + "yahs": { + "branch": "master", + "git_sha": "d3a3d04073af68437f07e0fb9b88aa6172c3f678", "installed_by": ["modules"] } } diff --git a/modules/local/find_telomere_regions.nf b/modules/local/find_telomere_regions.nf index c9f0e6a3..092926dc 100755 --- a/modules/local/find_telomere_regions.nf +++ b/modules/local/find_telomere_regions.nf @@ -2,7 +2,7 @@ process FIND_TELOMERE_REGIONS { tag "${meta.id}" label 'process_low' - container 'docker.io/library/gcc:10.4.0' + container 'quay.io/sanger-tol/telomere:0.0.1-c1' input: tuple val( meta ), path( file ) diff --git a/modules/local/generate_cram_csv.nf b/modules/local/generate_cram_csv.nf index 6a06bb87..2bccab9d 100755 --- a/modules/local/generate_cram_csv.nf +++ b/modules/local/generate_cram_csv.nf @@ -2,7 +2,14 @@ process GENERATE_CRAM_CSV { tag "${meta.id}" label 'process_tiny' - container 'quay.io/sanger-tol/cramfilter_bwamem2_minimap2_samtools_perl:0.001-c1' + container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? + 'https://depot.galaxyproject.org/singularity/mulled-v2-1a6fe65bd6674daba65066aa796ed8f5e8b4687b:688e175eb0db54de17822ba7810cc9e20fa06dd5-0' : + 'biocontainers/mulled-v2-1a6fe65bd6674daba65066aa796ed8f5e8b4687b:688e175eb0db54de17822ba7810cc9e20fa06dd5-0' }" + + // Exit if running this module with -profile conda / -profile mamba + if (workflow.profile.tokenize(',').intersect(['conda', 'mamba']).size() >= 1) { + error "GENERATE_CRAM_CSV module does not support Conda. Please use Docker / Singularity instead." + } input: tuple val(meta), path(crampath) @@ -19,6 +26,7 @@ process GENERATE_CRAM_CSV { cat <<-END_VERSIONS > versions.yml "${task.process}": + generate_cram_csv: \$(generate_cram_csv.sh -v) samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//' ) END_VERSIONS """ @@ -29,6 +37,7 @@ process GENERATE_CRAM_CSV { cat <<-END_VERSIONS > versions.yml "${task.process}": + generate_cram_csv: \$(generate_cram_csv.sh -v) samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//' ) END_VERSIONS """ diff --git a/modules/local/graphoverallcoverage.nf b/modules/local/graphoverallcoverage.nf index b8cc8777..572e793d 100755 --- a/modules/local/graphoverallcoverage.nf +++ b/modules/local/graphoverallcoverage.nf @@ -2,7 +2,14 @@ process GRAPHOVERALLCOVERAGE { tag "$meta.id" label "process_single" - container 'quay.io/sanger-tol/cramfilter_bwamem2_minimap2_samtools_perl:0.001-c1' + container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? + 'https://depot.galaxyproject.org/singularity/mulled-v2-1a6fe65bd6674daba65066aa796ed8f5e8b4687b:688e175eb0db54de17822ba7810cc9e20fa06dd5-0' : + 'biocontainers/mulled-v2-1a6fe65bd6674daba65066aa796ed8f5e8b4687b:688e175eb0db54de17822ba7810cc9e20fa06dd5-0' }" + + // Exit if running this module with -profile conda / -profile mamba + if (workflow.profile.tokenize(',').intersect(['conda', 'mamba']).size() >= 1) { + error "GRAPHOVERALLCOVERAGE module does not support Conda. Please use Docker / Singularity instead." + } input: tuple val(meta), path(bed) diff --git a/modules/local/pretext_graph.nf b/modules/local/pretext_graph.nf index 3f600441..f2bdc029 100644 --- a/modules/local/pretext_graph.nf +++ b/modules/local/pretext_graph.nf @@ -2,7 +2,7 @@ process PRETEXT_GRAPH { tag "$meta.id" label 'process_single' - container "quay.io/sanger-tol/pretext:0.0.2-yy5-c3" + container "quay.io/sanger-tol/pretext:0.0.3-yy5-c1" input: tuple val(meta), path(pretext_file) diff --git a/modules/local/rename_ids.nf b/modules/local/rename_ids.nf index f69f518d..ba5db7ca 100755 --- a/modules/local/rename_ids.nf +++ b/modules/local/rename_ids.nf @@ -24,7 +24,8 @@ process RENAME_IDS { cat <<-END_VERSIONS > versions.yml "${task.process}": - coreutils: $VERSION + rename_ids: \$(rename_ids.sh -v) + coreutils: $VERSION END_VERSIONS /$ @@ -36,8 +37,8 @@ process RENAME_IDS { cat <<-END_VERSIONS > versions.yml "${task.process}": - rename_ids: \$(rename_ids.sh -v) - coreutils: $VERSION + rename_ids: \$(rename_ids.sh -v) + coreutils: $VERSION END_VERSIONS """ diff --git a/modules/local/chunkfasta.nf b/modules/local/seqkit/split/main.nf similarity index 50% rename from modules/local/chunkfasta.nf rename to modules/local/seqkit/split/main.nf index afb5050e..d9266738 100755 --- a/modules/local/chunkfasta.nf +++ b/modules/local/seqkit/split/main.nf @@ -1,48 +1,43 @@ -process CHUNKFASTA { +process SEQKIT_SPLIT { tag "${meta.id}" label 'process_low' - conda "conda-forge::pyfasta=0.5.2-1" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/pyfasta:0.5.2--py_1' : - 'biocontainers/pyfasta:0.5.2--py_1' }" + 'https://depot.galaxyproject.org/singularity/seqkit:2.9.0--h9ee0642_0' : + 'biocontainers/seqkit:2.9.0--h9ee0642_0' }" input: tuple val(meta), path('input.fasta') val(number_of_chunks) output: - tuple val(meta), path('*.fasta'), emit: fasta - path "versions.yml" , emit: versions + tuple val(meta), path('*.fa'), emit: fasta + path "versions.yml", emit: versions script: - def VERSION = '0.5.2' // Tool does not report version // This should be abstracted outside of the container to // stop it spinning up in the first place, // however dsl2 can't do comparisons with channels which makes it harder """ if [ $number_of_chunks -le 1 ]; then - mv input.fasta ${meta.id}_whole.fasta + mv input.fasta ${meta.id}_whole.fa else - pyfasta split -n $number_of_chunks input.fasta + seqkit split input.fasta -p $number_of_chunks -O ./ fi cat <<-END_VERSIONS > versions.yml "${task.process}": - python: \$(python --version | sed 's/Python //g') - pyfasta: $VERSION + seqkit: \$(seqkit version | sed -e "s/seqkit v//g") END_VERSIONS """ stub: - def VERSION = '0.5.2' // Tool does not report version """ - touch ${meta.id}.fa + touch ${prefix}.fa cat <<-END_VERSIONS > versions.yml "${task.process}": - python: \$(python --version | sed 's/Python //g') - pyfasta: $VERSION + seqkit: \$(seqkit version | sed -e "s/seqkit v//g") END_VERSIONS """ } diff --git a/modules/nf-core/bedtools/bamtobed/environment.yml b/modules/nf-core/bedtools/bamtobed/environment.yml index 4b50139e..5683bc05 100644 --- a/modules/nf-core/bedtools/bamtobed/environment.yml +++ b/modules/nf-core/bedtools/bamtobed/environment.yml @@ -1,7 +1,5 @@ -name: bedtools_bamtobed channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::bedtools=2.31.1 diff --git a/modules/nf-core/bedtools/bamtobed/meta.yml b/modules/nf-core/bedtools/bamtobed/meta.yml index b6d14929..20171885 100644 --- a/modules/nf-core/bedtools/bamtobed/meta.yml +++ b/modules/nf-core/bedtools/bamtobed/meta.yml @@ -12,30 +12,33 @@ tools: A set of tools for genomic analysis tasks, specifically enabling genome arithmetic (merge, count, complement) on various file types. documentation: https://bedtools.readthedocs.io/en/latest/content/tools/complement.html licence: ["MIT"] + identifier: biotools:bedtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - bam: - type: file - description: Input BAM file - pattern: "*.{bam}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - bam: + type: file + description: Input BAM file + pattern: "*.{bam}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - bed: - type: file - description: Bed file containing genomic intervals. - pattern: "*.{bed}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bed": + type: file + description: Bed file containing genomic intervals. + pattern: "*.{bed}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@yuukiiwa" - "@drpatelh" diff --git a/modules/nf-core/bedtools/bamtobed/tests/main.nf.test b/modules/nf-core/bedtools/bamtobed/tests/main.nf.test index 9c65ef70..297f1813 100644 --- a/modules/nf-core/bedtools/bamtobed/tests/main.nf.test +++ b/modules/nf-core/bedtools/bamtobed/tests/main.nf.test @@ -14,7 +14,7 @@ nextflow_process { """ input[0] = [ [ id:'test' ], // meta map - file(params.test_data['sarscov2']['illumina']['test_single_end_bam'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.single_end.bam', checkIfExists: true) ] """ } @@ -38,7 +38,7 @@ nextflow_process { """ input[0] = [ [ id:'test' ], // meta map - file(params.test_data['sarscov2']['illumina']['test_single_end_bam'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.single_end.bam', checkIfExists: true) ] """ } diff --git a/modules/nf-core/bedtools/genomecov/environment.yml b/modules/nf-core/bedtools/genomecov/environment.yml index 8fbe20c3..5683bc05 100644 --- a/modules/nf-core/bedtools/genomecov/environment.yml +++ b/modules/nf-core/bedtools/genomecov/environment.yml @@ -1,7 +1,5 @@ -name: bedtools_genomecov channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::bedtools=2.31.1 diff --git a/modules/nf-core/bedtools/genomecov/main.nf b/modules/nf-core/bedtools/genomecov/main.nf index 7a4d9c45..35e2ab14 100644 --- a/modules/nf-core/bedtools/genomecov/main.nf +++ b/modules/nf-core/bedtools/genomecov/main.nf @@ -4,13 +4,14 @@ process BEDTOOLS_GENOMECOV { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/bedtools:2.31.1--hf5e1c6e_0' : - 'biocontainers/bedtools:2.31.1--hf5e1c6e_0' }" + 'https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/63/6397750e9730a3fbcc5b4c43f14bd141c64c723fd7dad80e47921a68a7c3cd21/data': + 'community.wave.seqera.io/library/bedtools_coreutils:a623c13f66d5262b' }" input: tuple val(meta), path(intervals), val(scale) path sizes val extension + val sort output: tuple val(meta), path("*.${extension}"), emit: genomecov @@ -20,12 +21,16 @@ process BEDTOOLS_GENOMECOV { task.ext.when == null || task.ext.when script: - def args = task.ext.args ?: '' + def args = task.ext.args ?: '' def args_list = args.tokenize() args += (scale > 0 && scale != 1) ? " -scale $scale" : "" if (!args_list.contains('-bg') && (scale > 0 && scale != 1)) { args += " -bg" } + // Sorts output file by chromosome and position using additional options for performance and consistency + // See https://www.biostars.org/p/66927/ for further details + def buffer = task.memory ? "--buffer-size=${task.memory.toGiga().intdiv(2)}G" : '' + def sort_cmd = sort ? "| LC_ALL=C sort --parallel=$task.cpus $buffer -k1,1 -k2,2n" : '' def prefix = task.ext.prefix ?: "${meta.id}" if (intervals.name =~ /\.bam/) { @@ -34,6 +39,7 @@ process BEDTOOLS_GENOMECOV { genomecov \\ -ibam $intervals \\ $args \\ + $sort_cmd \\ > ${prefix}.${extension} cat <<-END_VERSIONS > versions.yml @@ -48,6 +54,7 @@ process BEDTOOLS_GENOMECOV { -i $intervals \\ -g $sizes \\ $args \\ + $sort_cmd \\ > ${prefix}.${extension} cat <<-END_VERSIONS > versions.yml diff --git a/modules/nf-core/bedtools/genomecov/meta.yml b/modules/nf-core/bedtools/genomecov/meta.yml index 2b2385e3..41b1f8f4 100644 --- a/modules/nf-core/bedtools/genomecov/meta.yml +++ b/modules/nf-core/bedtools/genomecov/meta.yml @@ -1,5 +1,6 @@ name: bedtools_genomecov -description: Computes histograms (default), per-base reports (-d) and BEDGRAPH (-bg) summaries of feature coverage (e.g., aligned sequences) for a given genome. +description: Computes histograms (default), per-base reports (-d) and BEDGRAPH (-bg) + summaries of feature coverage (e.g., aligned sequences) for a given genome. keywords: - bed - bam @@ -12,39 +13,50 @@ tools: A set of tools for genomic analysis tasks, specifically enabling genome arithmetic (merge, count, complement) on various file types. documentation: https://bedtools.readthedocs.io/en/latest/content/tools/genomecov.html licence: ["MIT"] + identifier: biotools:bedtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - intervals: - type: file - description: BAM/BED/GFF/VCF - pattern: "*.{bam|bed|gff|vcf}" - - scale: - type: integer - description: Number containing the scale factor for the output. Set to 1 to disable. Setting to a value other than 1 will also get the -bg bedgraph output format as this is required for this command switch - - sizes: - type: file - description: Tab-delimited table of chromosome names in the first column and chromosome sizes in the second column - - extension: - type: string - description: Extension of the output file (e. g., ".bg", ".bedgraph", ".txt", ".tab", etc.) It is set arbitrarily by the user and corresponds to the file format which depends on arguments. + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - intervals: + type: file + description: BAM/BED/GFF/VCF + pattern: "*.{bam|bed|gff|vcf}" + - scale: + type: integer + description: Number containing the scale factor for the output. Set to 1 to + disable. Setting to a value other than 1 will also get the -bg bedgraph output + format as this is required for this command switch + - - sizes: + type: file + description: Tab-delimited table of chromosome names in the first column and + chromosome sizes in the second column + - - extension: + type: string + description: Extension of the output file (e. g., ".bg", ".bedgraph", ".txt", + ".tab", etc.) It is set arbitrarily by the user and corresponds to the file + format which depends on arguments. + - - sort: + type: boolean + description: Sort the output output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - genomecov: - type: file - description: Computed genome coverage file - pattern: "*.${extension}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.${extension}": + type: file + description: Computed genome coverage file + pattern: "*.${extension}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@edmundmiller" - "@sruthipsuresh" diff --git a/modules/nf-core/bedtools/genomecov/tests/main.nf.test b/modules/nf-core/bedtools/genomecov/tests/main.nf.test index 21e69aed..16a03492 100644 --- a/modules/nf-core/bedtools/genomecov/tests/main.nf.test +++ b/modules/nf-core/bedtools/genomecov/tests/main.nf.test @@ -9,19 +9,20 @@ nextflow_process { tag "bedtools" tag "bedtools/genomecov" - test("sarscov2 - no scale") { + test("sarscov2 - no scale") { when { process { - """ + """ input[0] = [ [ id:'test' ], // meta map - file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true), + file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/bam/test.paired_end.bam", checkIfExists: true), 1 ] // sizes input[1] = [] // extension input[2] = "txt" + input[3] = true """ } } @@ -29,25 +30,25 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot(process.out).match("no_scale") } + { assert snapshot(process.out).match() } ) } - } test("sarscov2 - dummy sizes") { when { process { - """ + """ input[0] = [ [ id:'test'], - file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true), + file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/bam/test.paired_end.bam", checkIfExists: true), 0.5 ] // sizes input[1] = file('dummy_chromosome_sizes') // extension input[2] = 'txt' + input[3] = false """ } } @@ -55,25 +56,25 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot(process.out).match("dummy_sizes") } + { assert snapshot(process.out).match() } ) } - } test("sarscov2 - scale") { when { process { - """ + """ input[0] = [ [ id:'test'], - file(params.test_data['sarscov2']['genome']['baits_bed'], checkIfExists: true), + file(params.modules_testdata_base_path + "genomics/sarscov2/genome/bed/baits.bed", checkIfExists: true), 0.5 ] // sizes - input[1] = file(params.test_data['sarscov2']['genome']['genome_sizes'], checkIfExists: true) + input[1] = file(params.modules_testdata_base_path + "genomics/sarscov2/genome/genome.sizes", checkIfExists: true) // extension input[2] = 'txt' + input[3] = false """ } } @@ -81,27 +82,55 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot(process.out).match("scale") } + { assert snapshot(process.out).match() } ) } - } - test("stub") { + test("sarscov2 - no scale - stub") { options "-stub" when { process { - """ + """ input[0] = [ [ id:'test' ], // meta map - file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true), + file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/bam/test.paired_end.bam", checkIfExists: true), 1 ] // sizes input[1] = [] // extension + input[2] = "txt" + input[3] = true + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("sarscov2 - dummy sizes - stub") { + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test'], + file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/bam/test.paired_end.bam", checkIfExists: true), + 0.5 + ] + // sizes + input[1] = file('dummy_chromosome_sizes') + // extension input[2] = 'txt' + input[3] = false """ } } @@ -109,10 +138,37 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot(file(process.out.genomecov[0][1]).name).match("stub") } + { assert snapshot(process.out).match() } ) } + } + + test("sarscov2 - scale - stub") { + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test'], + file(params.modules_testdata_base_path + "genomics/sarscov2/genome/bed/baits.bed", checkIfExists: true), + 0.5 + ] + // sizes + input[1] = file(params.modules_testdata_base_path + "genomics/sarscov2/genome/genome.sizes", checkIfExists: true) + // extension + input[2] = 'txt' + input[3] = false + """ + } + } + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } } } diff --git a/modules/nf-core/bedtools/genomecov/tests/main.nf.test.snap b/modules/nf-core/bedtools/genomecov/tests/main.nf.test.snap index 8f9191e4..da6dbe87 100644 --- a/modules/nf-core/bedtools/genomecov/tests/main.nf.test.snap +++ b/modules/nf-core/bedtools/genomecov/tests/main.nf.test.snap @@ -1,5 +1,5 @@ { - "dummy_sizes": { + "sarscov2 - dummy sizes": { "content": [ { "0": [ @@ -26,9 +26,13 @@ ] } ], - "timestamp": "2023-12-05T17:35:58.35232" + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-05T11:59:33.898146" }, - "no_scale": { + "sarscov2 - no scale - stub": { "content": [ { "0": [ @@ -36,7 +40,7 @@ { "id": "test" }, - "test.coverage.txt:md5,66083198daca6c001d328ba9616e9b53" + "test.coverage.txt:md5,d41d8cd98f00b204e9800998ecf8427e" ] ], "1": [ @@ -47,7 +51,7 @@ { "id": "test" }, - "test.coverage.txt:md5,66083198daca6c001d328ba9616e9b53" + "test.coverage.txt:md5,d41d8cd98f00b204e9800998ecf8427e" ] ], "versions": [ @@ -55,15 +59,46 @@ ] } ], - "timestamp": "2023-12-05T17:35:51.142496" + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-05T11:59:52.483371" }, - "stub": { + "sarscov2 - scale": { "content": [ - "test.coverage.txt" + { + "0": [ + [ + { + "id": "test" + }, + "test.coverage.txt:md5,de3c59c0ea123bcdbbad27bc0a0a601e" + ] + ], + "1": [ + "versions.yml:md5,5fd44452613992a6f71f2c73d2e117f2" + ], + "genomecov": [ + [ + { + "id": "test" + }, + "test.coverage.txt:md5,de3c59c0ea123bcdbbad27bc0a0a601e" + ] + ], + "versions": [ + "versions.yml:md5,5fd44452613992a6f71f2c73d2e117f2" + ] + } ], - "timestamp": "2023-12-05T17:36:13.084709" + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-05T11:59:43.69501" }, - "scale": { + "sarscov2 - scale - stub": { "content": [ { "0": [ @@ -71,7 +106,7 @@ { "id": "test" }, - "test.coverage.txt:md5,de3c59c0ea123bcdbbad27bc0a0a601e" + "test.coverage.txt:md5,d41d8cd98f00b204e9800998ecf8427e" ] ], "1": [ @@ -82,7 +117,73 @@ { "id": "test" }, - "test.coverage.txt:md5,de3c59c0ea123bcdbbad27bc0a0a601e" + "test.coverage.txt:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,5fd44452613992a6f71f2c73d2e117f2" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-05T12:00:09.930036" + }, + "sarscov2 - no scale": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.coverage.txt:md5,66083198daca6c001d328ba9616e9b53" + ] + ], + "1": [ + "versions.yml:md5,5fd44452613992a6f71f2c73d2e117f2" + ], + "genomecov": [ + [ + { + "id": "test" + }, + "test.coverage.txt:md5,66083198daca6c001d328ba9616e9b53" + ] + ], + "versions": [ + "versions.yml:md5,5fd44452613992a6f71f2c73d2e117f2" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-05T11:59:25.448817" + }, + "sarscov2 - dummy sizes - stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.coverage.txt:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,5fd44452613992a6f71f2c73d2e117f2" + ], + "genomecov": [ + [ + { + "id": "test" + }, + "test.coverage.txt:md5,d41d8cd98f00b204e9800998ecf8427e" ] ], "versions": [ @@ -90,6 +191,10 @@ ] } ], - "timestamp": "2023-12-05T17:36:05.962006" + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-05T12:00:01.086433" } } \ No newline at end of file diff --git a/modules/nf-core/bedtools/genomecov/tests/tags.yml b/modules/nf-core/bedtools/genomecov/tests/tags.yml deleted file mode 100644 index 55fce478..00000000 --- a/modules/nf-core/bedtools/genomecov/tests/tags.yml +++ /dev/null @@ -1,2 +0,0 @@ -bedtools/genomecov: - - "modules/nf-core/bedtools/genomecov/**" diff --git a/modules/nf-core/bedtools/intersect/environment.yml b/modules/nf-core/bedtools/intersect/environment.yml index 2a343050..5683bc05 100644 --- a/modules/nf-core/bedtools/intersect/environment.yml +++ b/modules/nf-core/bedtools/intersect/environment.yml @@ -1,7 +1,5 @@ -name: bedtools_intersect channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::bedtools=2.31.1 diff --git a/modules/nf-core/bedtools/intersect/meta.yml b/modules/nf-core/bedtools/intersect/meta.yml index 0939cb54..45ecf377 100644 --- a/modules/nf-core/bedtools/intersect/meta.yml +++ b/modules/nf-core/bedtools/intersect/meta.yml @@ -10,43 +10,47 @@ tools: A set of tools for genomic analysis tasks, specifically enabling genome arithmetic (merge, count, complement) on various file types. documentation: https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html licence: ["MIT"] + identifier: biotools:bedtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - intervals1: - type: file - description: BAM/BED/GFF/VCF - pattern: "*.{bam|bed|gff|vcf}" - - intervals2: - type: file - description: BAM/BED/GFF/VCF - pattern: "*.{bam|bed|gff|vcf}" - - meta2: - type: map - description: | - Groovy Map containing reference chromosome sizes - e.g. [ id:'test' ] - - chrom_sizes: - type: file - description: Chromosome sizes file - pattern: "*{.sizes,.txt}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - intervals1: + type: file + description: BAM/BED/GFF/VCF + pattern: "*.{bam|bed|gff|vcf}" + - intervals2: + type: file + description: BAM/BED/GFF/VCF + pattern: "*.{bam|bed|gff|vcf}" + - - meta2: + type: map + description: | + Groovy Map containing reference chromosome sizes + e.g. [ id:'test' ] + - chrom_sizes: + type: file + description: Chromosome sizes file + pattern: "*{.sizes,.txt}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - intersect: - type: file - description: File containing the description of overlaps found between the two features - pattern: "*.${extension}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.${extension}": + type: file + description: File containing the description of overlaps found between the two + features + pattern: "*.${extension}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@edmundmiller" - "@sruthipsuresh" diff --git a/modules/nf-core/bedtools/intersect/tests/main.nf.test b/modules/nf-core/bedtools/intersect/tests/main.nf.test new file mode 100644 index 00000000..cd770946 --- /dev/null +++ b/modules/nf-core/bedtools/intersect/tests/main.nf.test @@ -0,0 +1,90 @@ +nextflow_process { + + name "Test Process BEDTOOLS_INTERSECT" + script "../main.nf" + process "BEDTOOLS_INTERSECT" + config "./nextflow.config" + + tag "modules" + tag "modules_nfcore" + tag "bedtools" + tag "bedtools/intersect" + + test("sarscov2 - bed - bed") { + + when { + process { + """ + input[0] = [ + [ id:'test' ], + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test2.bed', checkIfExists: true) + ] + + input[1] = [[:], []] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("sarscov2 - bam - bam") { + + when { + process { + """ + input[0] = [ + [ id:'test' ], + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.bam', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/baits.bed', checkIfExists: true) + ] + + input[1] = [[:], []] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("sarscov2 - bed - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test' ], + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test2.bed', checkIfExists: true) + ] + + input[1] = [[:], []] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + +} diff --git a/modules/nf-core/bedtools/intersect/tests/main.nf.test.snap b/modules/nf-core/bedtools/intersect/tests/main.nf.test.snap new file mode 100644 index 00000000..b748dd49 --- /dev/null +++ b/modules/nf-core/bedtools/intersect/tests/main.nf.test.snap @@ -0,0 +1,101 @@ +{ + "sarscov2 - bam - bam": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test_out.bam:md5,738324efe2b1e442ceb6539a630c3fe6" + ] + ], + "1": [ + "versions.yml:md5,42ba439339672f4a9193f0f0fe7a7f64" + ], + "intersect": [ + [ + { + "id": "test" + }, + "test_out.bam:md5,738324efe2b1e442ceb6539a630c3fe6" + ] + ], + "versions": [ + "versions.yml:md5,42ba439339672f4a9193f0f0fe7a7f64" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-17T20:55:57.454847668" + }, + "sarscov2 - bed - bed": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test_out.bed:md5,afcbf01c2f2013aad71dbe8e34f2c15c" + ] + ], + "1": [ + "versions.yml:md5,42ba439339672f4a9193f0f0fe7a7f64" + ], + "intersect": [ + [ + { + "id": "test" + }, + "test_out.bed:md5,afcbf01c2f2013aad71dbe8e34f2c15c" + ] + ], + "versions": [ + "versions.yml:md5,42ba439339672f4a9193f0f0fe7a7f64" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-17T20:55:49.072132931" + }, + "sarscov2 - bed - stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test_out.bed:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,42ba439339672f4a9193f0f0fe7a7f64" + ], + "intersect": [ + [ + { + "id": "test" + }, + "test_out.bed:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,42ba439339672f4a9193f0f0fe7a7f64" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-17T20:56:06.259192552" + } +} \ No newline at end of file diff --git a/modules/nf-core/bedtools/intersect/tests/nextflow.config b/modules/nf-core/bedtools/intersect/tests/nextflow.config new file mode 100644 index 00000000..f1f9e693 --- /dev/null +++ b/modules/nf-core/bedtools/intersect/tests/nextflow.config @@ -0,0 +1,5 @@ +process { + withName: BEDTOOLS_INTERSECT { + ext.prefix = { "${meta.id}_out" } + } +} diff --git a/modules/nf-core/bedtools/intersect/tests/tags.yml b/modules/nf-core/bedtools/intersect/tests/tags.yml new file mode 100644 index 00000000..6219cc40 --- /dev/null +++ b/modules/nf-core/bedtools/intersect/tests/tags.yml @@ -0,0 +1,2 @@ +bedtools/intersect: + - "modules/nf-core/bedtools/intersect/**" diff --git a/modules/nf-core/bedtools/makewindows/environment.yml b/modules/nf-core/bedtools/makewindows/environment.yml index 0de3c15d..5683bc05 100644 --- a/modules/nf-core/bedtools/makewindows/environment.yml +++ b/modules/nf-core/bedtools/makewindows/environment.yml @@ -1,7 +1,5 @@ -name: bedtools_makewindows channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::bedtools=2.31.1 diff --git a/modules/nf-core/bedtools/makewindows/meta.yml b/modules/nf-core/bedtools/makewindows/meta.yml index f89d7175..df047f66 100644 --- a/modules/nf-core/bedtools/makewindows/meta.yml +++ b/modules/nf-core/bedtools/makewindows/meta.yml @@ -7,35 +7,39 @@ keywords: - chunking tools: - bedtools: - description: A set of tools for genomic analysis tasks, specifically enabling genome arithmetic (merge, count, complement) on various file types. + description: A set of tools for genomic analysis tasks, specifically enabling + genome arithmetic (merge, count, complement) on various file types. homepage: https://bedtools.readthedocs.io documentation: https://bedtools.readthedocs.io/en/latest/content/tools/makewindows.html doi: "10.1093/bioinformatics/btq033" licence: ["MIT"] + identifier: biotools:bedtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - regions: - type: file - description: BED file OR Genome details file () - pattern: "*.{bed,tab,fai}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - regions: + type: file + description: BED file OR Genome details file () + pattern: "*.{bed,tab,fai}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - bed: - type: file - description: BED file containing the windows - pattern: "*.bed" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bed": + type: file + description: BED file containing the windows + pattern: "*.bed" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@kevbrick" - "@nvnieuwk" diff --git a/modules/nf-core/bedtools/makewindows/tests/main.nf.test b/modules/nf-core/bedtools/makewindows/tests/main.nf.test new file mode 100644 index 00000000..b27e59b6 --- /dev/null +++ b/modules/nf-core/bedtools/makewindows/tests/main.nf.test @@ -0,0 +1,58 @@ + +nextflow_process { + + name "Test Process BEDTOOLS_MAKEWINDOWS" + script "../main.nf" + process "BEDTOOLS_MAKEWINDOWS" + config "./nextflow.config" + + tag "modules" + tag "modules_nfcore" + tag "bedtools" + tag "bedtools/makewindows" + + test("test-bedtools-makewindows-bed") { + + when { + process { + """ + input[0] = [ + [ id:'test2'], + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true) + ] + + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("test-bedtools-makewindows-fai") { + + when { + process { + """ + input[0] = [ + [ id:'test2'], + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta.fai', checkIfExists: true) + ] + + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + +} diff --git a/modules/nf-core/bedtools/makewindows/tests/main.nf.test.snap b/modules/nf-core/bedtools/makewindows/tests/main.nf.test.snap new file mode 100644 index 00000000..22cfbc17 --- /dev/null +++ b/modules/nf-core/bedtools/makewindows/tests/main.nf.test.snap @@ -0,0 +1,68 @@ +{ + "test-bedtools-makewindows-fai": { + "content": [ + { + "0": [ + [ + { + "id": "test2" + }, + "test2.bed:md5,622d1f62786fe4239b76c53168f21c54" + ] + ], + "1": [ + "versions.yml:md5,f797078cc8b8bac7e6906685d4867be5" + ], + "bed": [ + [ + { + "id": "test2" + }, + "test2.bed:md5,622d1f62786fe4239b76c53168f21c54" + ] + ], + "versions": [ + "versions.yml:md5,f797078cc8b8bac7e6906685d4867be5" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-08-26T14:03:31.430455" + }, + "test-bedtools-makewindows-bed": { + "content": [ + { + "0": [ + [ + { + "id": "test2" + }, + "test2.bed:md5,0cf6ed2b6f470cd44a247da74ca4fe4e" + ] + ], + "1": [ + "versions.yml:md5,f797078cc8b8bac7e6906685d4867be5" + ], + "bed": [ + [ + { + "id": "test2" + }, + "test2.bed:md5,0cf6ed2b6f470cd44a247da74ca4fe4e" + ] + ], + "versions": [ + "versions.yml:md5,f797078cc8b8bac7e6906685d4867be5" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-08-26T14:03:27.118372" + } +} \ No newline at end of file diff --git a/modules/nf-core/bedtools/makewindows/tests/nextflow.config b/modules/nf-core/bedtools/makewindows/tests/nextflow.config new file mode 100644 index 00000000..fa16733f --- /dev/null +++ b/modules/nf-core/bedtools/makewindows/tests/nextflow.config @@ -0,0 +1,5 @@ +process { + withName: BEDTOOLS_MAKEWINDOWS { + ext.args = '-w 50 ' + } +} diff --git a/modules/nf-core/bedtools/map/environment.yml b/modules/nf-core/bedtools/map/environment.yml index f61ee028..5683bc05 100644 --- a/modules/nf-core/bedtools/map/environment.yml +++ b/modules/nf-core/bedtools/map/environment.yml @@ -1,7 +1,5 @@ -name: bedtools_map channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::bedtools=2.31.1 diff --git a/modules/nf-core/bedtools/map/meta.yml b/modules/nf-core/bedtools/map/meta.yml index 0267f6ff..4e56bb94 100644 --- a/modules/nf-core/bedtools/map/meta.yml +++ b/modules/nf-core/bedtools/map/meta.yml @@ -12,43 +12,47 @@ tools: A set of tools for genomic analysis tasks, specifically enabling genome arithmetic (merge, count, complement) on various file types. documentation: https://bedtools.readthedocs.io/en/latest/content/tools/map.html licence: ["MIT"] + identifier: biotools:bedtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - intervals1: - type: file - description: BAM/BED/GFF/VCF - pattern: "*.{bed|gff|vcf}" - - intervals2: - type: file - description: BAM/BED/GFF/VCF - pattern: "*.{bed|gff|vcf}" - - meta2: - type: map - description: | - Groovy Map containing reference chromosome sizes - e.g. [ id:'test' ] - - chrom_sizes: - type: file - description: Chromosome sizes file - pattern: "*{.sizes,.txt}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - intervals1: + type: file + description: BAM/BED/GFF/VCF + pattern: "*.{bed|gff|vcf}" + - intervals2: + type: file + description: BAM/BED/GFF/VCF + pattern: "*.{bed|gff|vcf}" + - - meta2: + type: map + description: | + Groovy Map containing reference chromosome sizes + e.g. [ id:'test' ] + - chrom_sizes: + type: file + description: Chromosome sizes file + pattern: "*{.sizes,.txt}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - mapped: - type: file - description: File containing the description of overlaps found between the features in A and the features in B, with statistics - pattern: "*.${extension}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.${extension}": + type: file + description: File containing the description of overlaps found between the features + in A and the features in B, with statistics + pattern: "*.${extension}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@ekushele" maintainers: diff --git a/modules/nf-core/bedtools/map/tests/main.nf.test b/modules/nf-core/bedtools/map/tests/main.nf.test index 4adc0a21..e46cf743 100644 --- a/modules/nf-core/bedtools/map/tests/main.nf.test +++ b/modules/nf-core/bedtools/map/tests/main.nf.test @@ -18,8 +18,8 @@ nextflow_process { """ input[0] = [ [ id:'test' ], - file(params.test_data['sarscov2']['genome']['test_bed'], checkIfExists: true), - file(params.test_data['sarscov2']['genome']['test2_bed'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test2.bed', checkIfExists: true) ] input[1] = [[],[]] """ @@ -42,8 +42,8 @@ nextflow_process { """ input[0] = [ [ id:'test' ], - file(params.test_data['sarscov2']['genome']['test_bed'], checkIfExists: true), - file(params.test_data['sarscov2']['illumina']['test_vcf'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/vcf/test.vcf', checkIfExists: true) ] input[1] = [[],[]] """ @@ -67,8 +67,8 @@ nextflow_process { """ input[0] = [ [ id:'test' ], - file(params.test_data['sarscov2']['genome']['test_bed'], checkIfExists: true), - file(params.test_data['sarscov2']['genome']['test2_bed'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test2.bed', checkIfExists: true) ] input[1] = [[],[]] """ diff --git a/modules/nf-core/bedtools/merge/environment.yml b/modules/nf-core/bedtools/merge/environment.yml index 99707878..5683bc05 100644 --- a/modules/nf-core/bedtools/merge/environment.yml +++ b/modules/nf-core/bedtools/merge/environment.yml @@ -1,7 +1,5 @@ -name: bedtools_merge channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::bedtools=2.31.1 diff --git a/modules/nf-core/bedtools/merge/meta.yml b/modules/nf-core/bedtools/merge/meta.yml index d7463e3d..6da54205 100644 --- a/modules/nf-core/bedtools/merge/meta.yml +++ b/modules/nf-core/bedtools/merge/meta.yml @@ -1,5 +1,6 @@ name: bedtools_merge -description: combines overlapping or “book-ended” features in an interval file into a single feature which spans all of the combined features. +description: combines overlapping or “book-ended” features in an interval file into + a single feature which spans all of the combined features. keywords: - bed - merge @@ -11,30 +12,33 @@ tools: A set of tools for genomic analysis tasks, specifically enabling genome arithmetic (merge, count, complement) on various file types. documentation: https://bedtools.readthedocs.io/en/latest/content/tools/merge.html licence: ["MIT"] + identifier: biotools:bedtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - bed: - type: file - description: Input BED file - pattern: "*.{bed}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - bed: + type: file + description: Input BED file + pattern: "*.{bed}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - bed: - type: file - description: Overlapped bed file with combined features - pattern: "*.{bed}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bed": + type: file + description: Overlapped bed file with combined features + pattern: "*.{bed}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@edmundmiller" - "@sruthipsuresh" diff --git a/modules/nf-core/bedtools/merge/tests/main.nf.test b/modules/nf-core/bedtools/merge/tests/main.nf.test new file mode 100644 index 00000000..2959d6b9 --- /dev/null +++ b/modules/nf-core/bedtools/merge/tests/main.nf.test @@ -0,0 +1,34 @@ +nextflow_process { + + name "Test Process BEDTOOLS_MERGE" + script "../main.nf" + config "./nextflow.config" + process "BEDTOOLS_MERGE" + + tag "modules" + tag "modules_nfcore" + tag "bedtools" + tag "bedtools/merge" + + test("test_bedtools_merge") { + + when { + process { + """ + input[0] = [ [ id:'test'], + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true) + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + +} \ No newline at end of file diff --git a/modules/nf-core/bedtools/merge/tests/main.nf.test.snap b/modules/nf-core/bedtools/merge/tests/main.nf.test.snap new file mode 100644 index 00000000..ee6c4e63 --- /dev/null +++ b/modules/nf-core/bedtools/merge/tests/main.nf.test.snap @@ -0,0 +1,35 @@ +{ + "test_bedtools_merge": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test_out.bed:md5,0cf6ed2b6f470cd44a247da74ca4fe4e" + ] + ], + "1": [ + "versions.yml:md5,2d134badb4cd1e4e903696c7967f28d6" + ], + "bed": [ + [ + { + "id": "test" + }, + "test_out.bed:md5,0cf6ed2b6f470cd44a247da74ca4fe4e" + ] + ], + "versions": [ + "versions.yml:md5,2d134badb4cd1e4e903696c7967f28d6" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-03-18T17:07:09.721153" + } +} \ No newline at end of file diff --git a/modules/nf-core/bedtools/merge/tests/nextflow.config b/modules/nf-core/bedtools/merge/tests/nextflow.config new file mode 100644 index 00000000..16444e98 --- /dev/null +++ b/modules/nf-core/bedtools/merge/tests/nextflow.config @@ -0,0 +1,7 @@ +process { + + withName: BEDTOOLS_MERGE { + ext.prefix = { "${meta.id}_out" } + } + +} diff --git a/modules/nf-core/bedtools/merge/tests/tags.yml b/modules/nf-core/bedtools/merge/tests/tags.yml new file mode 100644 index 00000000..60c8cad1 --- /dev/null +++ b/modules/nf-core/bedtools/merge/tests/tags.yml @@ -0,0 +1,2 @@ +bedtools/merge: + - "modules/nf-core/bedtools/merge/**" diff --git a/modules/nf-core/bedtools/sort/environment.yml b/modules/nf-core/bedtools/sort/environment.yml index 87b2e425..5683bc05 100644 --- a/modules/nf-core/bedtools/sort/environment.yml +++ b/modules/nf-core/bedtools/sort/environment.yml @@ -1,7 +1,5 @@ -name: bedtools_sort channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::bedtools=2.31.1 diff --git a/modules/nf-core/bedtools/sort/meta.yml b/modules/nf-core/bedtools/sort/meta.yml index 7c915f5f..313698f1 100644 --- a/modules/nf-core/bedtools/sort/meta.yml +++ b/modules/nf-core/bedtools/sort/meta.yml @@ -11,35 +11,38 @@ tools: A set of tools for genomic analysis tasks, specifically enabling genome arithmetic (merge, count, complement) on various file types. documentation: https://bedtools.readthedocs.io/en/latest/content/tools/sort.html licence: ["MIT"] + identifier: biotools:bedtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - intervals: - type: file - description: BED/BEDGRAPH - pattern: "*.{bed|bedGraph}" - - genome_file: - type: file - description: | - Optional reference genome 2 column file that defines the expected chromosome order. - pattern: "*.{fai,txt,chromsizes}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - intervals: + type: file + description: BED/BEDGRAPH + pattern: "*.{bed|bedGraph}" + - - genome_file: + type: file + description: | + Optional reference genome 2 column file that defines the expected chromosome order. + pattern: "*.{fai,txt,chromsizes}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - sorted: - type: file - description: Sorted output file - pattern: "*.${extension}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.${extension}": + type: file + description: Sorted output file + pattern: "*.${extension}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@edmundmiller" - "@sruthipsuresh" diff --git a/modules/nf-core/bedtools/sort/tests/main.nf.test b/modules/nf-core/bedtools/sort/tests/main.nf.test new file mode 100644 index 00000000..b1f36dd9 --- /dev/null +++ b/modules/nf-core/bedtools/sort/tests/main.nf.test @@ -0,0 +1,58 @@ +nextflow_process { + + name "Test Process BEDTOOLS_SORT" + script "../main.nf" + config "./nextflow.config" + process "BEDTOOLS_SORT" + + tag "modules" + tag "modules_nfcore" + tag "bedtools" + tag "bedtools/sort" + + test("test_bedtools_sort") { + + when { + process { + """ + input[0] = [ [ id:'test'], + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true) + ] + input[1] = [] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + + test("test_bedtools_sort_with_genome") { + + when { + process { + """ + input[0] = [ [ id:'test'], + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true) + ] + input[1] = file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta.fai', checkIfExists: true) + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + +} \ No newline at end of file diff --git a/modules/nf-core/bedtools/sort/tests/main.nf.test.snap b/modules/nf-core/bedtools/sort/tests/main.nf.test.snap new file mode 100644 index 00000000..f10e8b98 --- /dev/null +++ b/modules/nf-core/bedtools/sort/tests/main.nf.test.snap @@ -0,0 +1,68 @@ +{ + "test_bedtools_sort_with_genome": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test_out.testtext:md5,fe4053cf4de3aebbdfc3be2efb125a74" + ] + ], + "1": [ + "versions.yml:md5,cdbae2c7ebc41e534aaf0835779061f8" + ], + "sorted": [ + [ + { + "id": "test" + }, + "test_out.testtext:md5,fe4053cf4de3aebbdfc3be2efb125a74" + ] + ], + "versions": [ + "versions.yml:md5,cdbae2c7ebc41e534aaf0835779061f8" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-03-19T10:13:11.830452" + }, + "test_bedtools_sort": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test_out.testtext:md5,fe4053cf4de3aebbdfc3be2efb125a74" + ] + ], + "1": [ + "versions.yml:md5,cdbae2c7ebc41e534aaf0835779061f8" + ], + "sorted": [ + [ + { + "id": "test" + }, + "test_out.testtext:md5,fe4053cf4de3aebbdfc3be2efb125a74" + ] + ], + "versions": [ + "versions.yml:md5,cdbae2c7ebc41e534aaf0835779061f8" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-03-19T10:16:40.535947" + } +} \ No newline at end of file diff --git a/modules/nf-core/bedtools/sort/tests/nextflow.config b/modules/nf-core/bedtools/sort/tests/nextflow.config new file mode 100644 index 00000000..f203c99c --- /dev/null +++ b/modules/nf-core/bedtools/sort/tests/nextflow.config @@ -0,0 +1,8 @@ +process { + + withName: BEDTOOLS_SORT { + ext.prefix = { "${meta.id}_out" } + ext.suffix = "testtext" + } + +} \ No newline at end of file diff --git a/modules/nf-core/bedtools/sort/tests/tags.yml b/modules/nf-core/bedtools/sort/tests/tags.yml new file mode 100644 index 00000000..47c85eea --- /dev/null +++ b/modules/nf-core/bedtools/sort/tests/tags.yml @@ -0,0 +1,2 @@ +bedtools/sort: + - "modules/nf-core/bedtools/sort/**" diff --git a/modules/nf-core/busco/environment.yml b/modules/nf-core/busco/busco/environment.yml similarity index 50% rename from modules/nf-core/busco/environment.yml rename to modules/nf-core/busco/busco/environment.yml index f872d057..5b918b45 100644 --- a/modules/nf-core/busco/environment.yml +++ b/modules/nf-core/busco/busco/environment.yml @@ -1,7 +1,5 @@ -name: busco channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::busco=5.5.0 + - bioconda::busco=5.7.1 diff --git a/modules/nf-core/busco/main.nf b/modules/nf-core/busco/busco/main.nf similarity index 80% rename from modules/nf-core/busco/main.nf rename to modules/nf-core/busco/busco/main.nf index e7100405..f7c1a662 100644 --- a/modules/nf-core/busco/main.nf +++ b/modules/nf-core/busco/busco/main.nf @@ -1,14 +1,14 @@ -process BUSCO { +process BUSCO_BUSCO { tag "$meta.id" label 'process_medium' conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/busco:5.5.0--pyhdfd78af_0': - 'biocontainers/busco:5.5.0--pyhdfd78af_0' }" + 'https://depot.galaxyproject.org/singularity/busco:5.7.1--pyhdfd78af_0': + 'biocontainers/busco:5.7.1--pyhdfd78af_0' }" input: - tuple val(meta), path('tmp_input/*') + tuple val(meta), path(fasta, stageAs:'tmp_input/*') val mode // Required: One of genome, proteins, or transcriptome val lineage // Required: lineage to check against, "auto" enables --auto-lineage instead path busco_lineages_path // Recommended: path to busco lineages - downloads if not set @@ -16,13 +16,13 @@ process BUSCO { output: tuple val(meta), path("*-busco.batch_summary.txt") , emit: batch_summary - tuple val(meta), path("short_summary.*.txt") , emit: short_summaries_txt, optional: true - tuple val(meta), path("short_summary.*.json") , emit: short_summaries_json, optional: true - tuple val(meta), path("*-busco/*/run_*/full_table.tsv") , emit: full_table, optional: true - tuple val(meta), path("*-busco/*/run_*/missing_busco_list.tsv") , emit: missing_busco_list, optional: true - tuple val(meta), path("*-busco/*/run_*/single_copy_proteins.faa") , emit: single_copy_proteins, optional: true + tuple val(meta), path("short_summary.*.txt") , emit: short_summaries_txt , optional: true + tuple val(meta), path("short_summary.*.json") , emit: short_summaries_json , optional: true + tuple val(meta), path("*-busco/*/run_*/full_table.tsv") , emit: full_table , optional: true + tuple val(meta), path("*-busco/*/run_*/missing_busco_list.tsv") , emit: missing_busco_list , optional: true + tuple val(meta), path("*-busco/*/run_*/single_copy_proteins.faa") , emit: single_copy_proteins , optional: true tuple val(meta), path("*-busco/*/run_*/busco_sequences") , emit: seq_dir - tuple val(meta), path("*-busco/*/translated_proteins") , emit: translated_dir, optional: true + tuple val(meta), path("*-busco/*/translated_proteins") , emit: translated_dir , optional: true tuple val(meta), path("*-busco") , emit: busco_dir path "versions.yml" , emit: versions @@ -91,4 +91,17 @@ process BUSCO { busco: \$( busco --version 2>&1 | sed 's/^BUSCO //' ) END_VERSIONS """ + + stub: + def prefix = task.ext.prefix ?: "${meta.id}-${lineage}" + def fasta_name = files(fasta).first().name - '.gz' + """ + touch ${prefix}-busco.batch_summary.txt + mkdir -p ${prefix}-busco/$fasta_name/run_${lineage}/busco_sequences + + cat <<-END_VERSIONS > versions.yml + "${task.process}": + busco: \$( busco --version 2>&1 | sed 's/^BUSCO //' ) + END_VERSIONS + """ } diff --git a/modules/nf-core/busco/busco/meta.yml b/modules/nf-core/busco/busco/meta.yml new file mode 100644 index 00000000..7cb6d69c --- /dev/null +++ b/modules/nf-core/busco/busco/meta.yml @@ -0,0 +1,152 @@ +name: busco_busco +description: Benchmarking Universal Single Copy Orthologs +keywords: + - quality control + - genome + - transcriptome + - proteome +tools: + - busco: + description: BUSCO provides measures for quantitative assessment of genome assembly, + gene set, and transcriptome completeness based on evolutionarily informed expectations + of gene content from near-universal single-copy orthologs selected from OrthoDB. + homepage: https://busco.ezlab.org/ + documentation: https://busco.ezlab.org/busco_userguide.html + tool_dev_url: https://gitlab.com/ezlab/busco + doi: "10.1007/978-1-4939-9173-0_14" + licence: ["MIT"] + identifier: biotools:busco +input: + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - fasta: + type: file + description: Nucleic or amino acid sequence file in FASTA format. + pattern: "*.{fasta,fna,fa,fasta.gz,fna.gz,fa.gz}" + - - mode: + type: string + description: The mode to run Busco in. One of genome, proteins, or transcriptome + pattern: "{genome,proteins,transcriptome}" + - - lineage: + type: string + description: The BUSCO lineage to use, or "auto" to automatically select lineage + - - busco_lineages_path: + type: directory + description: Path to local BUSCO lineages directory. + - - config_file: + type: file + description: Path to BUSCO config file. +output: + - batch_summary: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*-busco.batch_summary.txt": + type: file + description: Summary of all sequence files analyzed + pattern: "*-busco.batch_summary.txt" + - short_summaries_txt: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - short_summary.*.txt: + type: file + description: Short Busco summary in plain text format + pattern: "short_summary.*.txt" + - short_summaries_json: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - short_summary.*.json: + type: file + description: Short Busco summary in JSON format + pattern: "short_summary.*.json" + - full_table: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*-busco/*/run_*/full_table.tsv": + type: file + description: Full BUSCO results table + pattern: "full_table.tsv" + - missing_busco_list: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*-busco/*/run_*/missing_busco_list.tsv": + type: file + description: List of missing BUSCOs + pattern: "missing_busco_list.tsv" + - single_copy_proteins: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*-busco/*/run_*/single_copy_proteins.faa": + type: file + description: Fasta file of single copy proteins (transcriptome mode) + pattern: "single_copy_proteins.faa" + - seq_dir: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*-busco/*/run_*/busco_sequences": + type: directory + description: BUSCO sequence directory + pattern: "busco_sequences" + - translated_dir: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*-busco/*/translated_proteins": + type: directory + description: Six frame translations of each transcript made by the transcriptome + mode + pattern: "translated_dir" + - busco_dir: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*-busco": + type: directory + description: BUSCO lineage specific output + pattern: "*-busco" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" +authors: + - "@priyanka-surana" + - "@charles-plessy" + - "@mahesh-panchal" + - "@muffato" + - "@jvhagey" + - "@gallvp" +maintainers: + - "@priyanka-surana" + - "@charles-plessy" + - "@mahesh-panchal" + - "@muffato" + - "@jvhagey" + - "@gallvp" diff --git a/modules/nf-core/busco/busco/tests/main.nf.test b/modules/nf-core/busco/busco/tests/main.nf.test new file mode 100644 index 00000000..bb7b49a9 --- /dev/null +++ b/modules/nf-core/busco/busco/tests/main.nf.test @@ -0,0 +1,415 @@ +nextflow_process { + + name "Test Process BUSCO_BUSCO" + script "../main.nf" + process "BUSCO_BUSCO" + + tag "modules" + tag "modules_nfcore" + tag "busco" + tag "busco/busco" + + test("test_busco_genome_single_fasta") { + + config './nextflow.config' + + when { + process { + """ + input[0] = [ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/prokaryotes/bacteroides_fragilis/genome/genome.fna.gz', checkIfExists: true) + ] + input[1] = 'genome' + input[2] = 'bacteria_odb10' // Launch with 'auto' to use --auto-lineage, and specified lineages // 'auto' removed from test due to memory issues + input[3] = [] // Download busco lineage + input[4] = [] // No config + """ + } + } + + then { + assert process.success + + with(path(process.out.short_summaries_txt[0][1]).text) { + assert contains('BUSCO version') + assert contains('The lineage dataset is') + assert contains('BUSCO was run in mode') + assert contains('Complete BUSCOs') + assert contains('Missing BUSCOs') + assert contains('Dependencies and versions') + } + + with(path(process.out.short_summaries_json[0][1]).text) { + assert contains('one_line_summary') + assert contains('mode') + assert contains('dataset') + } + + assert snapshot( + process.out.batch_summary[0][1], + process.out.full_table[0][1], + process.out.missing_busco_list[0][1], + process.out.versions[0] + ).match() + + with(file(process.out.seq_dir[0][1]).listFiles().collect { it.name }) { + assert contains('single_copy_busco_sequences.tar.gz') + assert contains('multi_copy_busco_sequences.tar.gz') + assert contains('fragmented_busco_sequences.tar.gz') + } + + with(path("${process.out.busco_dir[0][1]}/logs/busco.log").text) { + assert contains('DEBUG:busco.run_BUSCO') + assert contains('Results from dataset') + assert contains('how to cite BUSCO') + } + + assert process.out.single_copy_proteins == [] + assert process.out.translated_dir == [] + } + } + + test("test_busco_genome_multi_fasta") { + + config './nextflow.config' + + when { + process { + """ + input[0] = [ + [ id:'test' ], // meta map + [ + file(params.modules_testdata_base_path + 'genomics/prokaryotes/bacteroides_fragilis/genome/genome.fna.gz', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/prokaryotes/candidatus_portiera_aleyrodidarum/genome/genome.fasta', checkIfExists: true) + ] + ] + input[1] = 'genome' + input[2] = 'bacteria_odb10' + input[3] = [] + input[4] = [] + """ + } + } + + then { + assert process.success + + with(path(process.out.short_summaries_txt[0][1][0]).text) { + assert contains('BUSCO version') + assert contains('The lineage dataset is') + assert contains('BUSCO was run in mode') + assert contains('Complete BUSCOs') + assert contains('Missing BUSCOs') + assert contains('Dependencies and versions') + } + + with(path(process.out.short_summaries_txt[0][1][1]).text) { + assert contains('BUSCO version') + assert contains('The lineage dataset is') + assert contains('BUSCO was run in mode') + assert contains('Complete BUSCOs') + assert contains('Missing BUSCOs') + assert contains('Dependencies and versions') + } + + with(path(process.out.short_summaries_json[0][1][0]).text) { + assert contains('one_line_summary') + assert contains('mode') + assert contains('dataset') + } + + with(path(process.out.short_summaries_json[0][1][1]).text) { + assert contains('one_line_summary') + assert contains('mode') + assert contains('dataset') + } + + assert snapshot( + process.out.batch_summary[0][1], + process.out.full_table[0][1], + process.out.missing_busco_list[0][1], + process.out.versions[0] + ).match() + + with(file(process.out.seq_dir[0][1][0]).listFiles().collect { it.name }) { + assert contains('single_copy_busco_sequences.tar.gz') + assert contains('multi_copy_busco_sequences.tar.gz') + assert contains('fragmented_busco_sequences.tar.gz') + } + + with(file(process.out.seq_dir[0][1][1]).listFiles().collect { it.name }) { + assert contains('single_copy_busco_sequences.tar.gz') + assert contains('multi_copy_busco_sequences.tar.gz') + assert contains('fragmented_busco_sequences.tar.gz') + } + + with(path("${process.out.busco_dir[0][1]}/logs/busco.log").text) { + assert contains('DEBUG:busco.run_BUSCO') + assert contains('Results from dataset') + assert contains('how to cite BUSCO') + } + + assert process.out.single_copy_proteins == [] + assert process.out.translated_dir == [] + } + + } + + test("test_busco_eukaryote_metaeuk") { + + config './nextflow.metaeuk.config' + + when { + process { + """ + input[0] = [ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ] + input[1] = 'genome' + input[2] = 'eukaryota_odb10' + input[3] = [] + input[4] = [] + """ + } + } + + then { + assert process.success + + with(path(process.out.short_summaries_txt[0][1]).text) { + assert contains('BUSCO version') + assert contains('The lineage dataset is') + assert contains('BUSCO was run in mode') + assert contains('Complete BUSCOs') + assert contains('Missing BUSCOs') + assert contains('Dependencies and versions') + } + + with(path(process.out.short_summaries_json[0][1]).text) { + assert contains('one_line_summary') + assert contains('mode') + assert contains('dataset') + } + + assert snapshot( + process.out.batch_summary[0][1], + process.out.full_table[0][1], + process.out.missing_busco_list[0][1], + process.out.versions[0] + ).match() + + with(file(process.out.seq_dir[0][1]).listFiles().collect { it.name }) { + assert contains('single_copy_busco_sequences.tar.gz') + assert contains('multi_copy_busco_sequences.tar.gz') + assert contains('fragmented_busco_sequences.tar.gz') + } + + with(path("${process.out.busco_dir[0][1]}/logs/busco.log").text) { + assert contains('DEBUG:busco.run_BUSCO') + assert contains('Results from dataset') + assert contains('how to cite BUSCO') + + } + + assert process.out.single_copy_proteins == [] + assert process.out.translated_dir == [] + } + + } + + test("test_busco_eukaryote_augustus") { + + config './nextflow.augustus.config' + + when { + process { + """ + input[0] = [ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ] + input[1] = 'genome' + input[2] = 'eukaryota_odb10' + input[3] = [] + input[4] = [] + """ + } + } + + then { + assert process.success + + assert snapshot( + process.out.batch_summary[0][1], + process.out.versions[0] + ).match() + + with(path("${process.out.busco_dir[0][1]}/logs/busco.log").text) { + assert contains('DEBUG:busco.run_BUSCO') + assert contains('Augustus did not recognize any genes') + + } + + assert process.out.short_summaries_json == [] + assert process.out.short_summaries_txt == [] + assert process.out.missing_busco_list == [] + assert process.out.full_table == [] + assert process.out.single_copy_proteins == [] + assert process.out.translated_dir == [] + } + + } + + test("test_busco_protein") { + + config './nextflow.config' + + when { + process { + """ + input[0] = [ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/prokaryotes/candidatus_portiera_aleyrodidarum/genome/proteome.fasta', checkIfExists: true) + ] + input[1] = 'proteins' + input[2] = 'bacteria_odb10' + input[3] = [] + input[4] = [] + """ + } + } + + then { + assert process.success + + with(path(process.out.short_summaries_txt[0][1]).text) { + assert contains('BUSCO version') + assert contains('The lineage dataset is') + assert contains('BUSCO was run in mode') + assert contains('Complete BUSCOs') + assert contains('Missing BUSCOs') + assert contains('Dependencies and versions') + } + + with(path(process.out.short_summaries_json[0][1]).text) { + assert contains('one_line_summary') + assert contains('mode') + assert contains('dataset') + } + + assert snapshot( + process.out.batch_summary[0][1], + process.out.full_table[0][1], + process.out.missing_busco_list[0][1], + process.out.versions[0] + ).match() + + with(file(process.out.seq_dir[0][1]).listFiles().collect { it.name }) { + assert contains('single_copy_busco_sequences.tar.gz') + assert contains('multi_copy_busco_sequences.tar.gz') + assert contains('fragmented_busco_sequences.tar.gz') + } + + with(path("${process.out.busco_dir[0][1]}/logs/busco.log").text) { + assert contains('DEBUG:busco.run_BUSCO') + assert contains('Results from dataset') + assert contains('how to cite BUSCO') + } + + assert process.out.single_copy_proteins == [] + assert process.out.translated_dir == [] + } + + } + + test("test_busco_transcriptome") { + + config './nextflow.config' + + when { + process { + """ + input[0] = [ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/prokaryotes/bacteroides_fragilis/illumina/fasta/test1.contigs.fa.gz', checkIfExists: true) + ] + input[1] = 'transcriptome' + input[2] = 'bacteria_odb10' + input[3] = [] + input[4] = [] + """ + } + } + + then { + assert process.success + + with(path(process.out.short_summaries_txt[0][1]).text) { + assert contains('BUSCO version') + assert contains('The lineage dataset is') + assert contains('BUSCO was run in mode') + assert contains('Complete BUSCOs') + assert contains('Missing BUSCOs') + assert contains('Dependencies and versions') + } + + with(path(process.out.short_summaries_json[0][1]).text) { + assert contains('one_line_summary') + assert contains('mode') + assert contains('dataset') + } + + assert snapshot( + process.out.batch_summary[0][1], + process.out.full_table[0][1], + process.out.missing_busco_list[0][1], + process.out.translated_dir[0][1], + process.out.single_copy_proteins[0][1], + process.out.versions[0] + ).match() + + with(file(process.out.seq_dir[0][1]).listFiles().collect { it.name }) { + assert contains('single_copy_busco_sequences.tar.gz') + assert contains('multi_copy_busco_sequences.tar.gz') + assert contains('fragmented_busco_sequences.tar.gz') + } + + with(path("${process.out.busco_dir[0][1]}/logs/busco.log").text) { + assert contains('DEBUG:busco.run_BUSCO') + assert contains('Results from dataset') + assert contains('how to cite BUSCO') + } + } + + } + + test("minimal-stub") { + + options '-stub' + + when { + process { + """ + input[0] = [ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/prokaryotes/bacteroides_fragilis/genome/genome.fna.gz', checkIfExists: true) + ] + input[1] = 'genome' + input[2] = 'bacteria_odb10' + input[3] = [] + input[4] = [] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + +} diff --git a/modules/nf-core/busco/busco/tests/main.nf.test.snap b/modules/nf-core/busco/busco/tests/main.nf.test.snap new file mode 100644 index 00000000..1b6411bc --- /dev/null +++ b/modules/nf-core/busco/busco/tests/main.nf.test.snap @@ -0,0 +1,230 @@ +{ + "minimal-stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test-bacteria_odb10-busco.batch_summary.txt:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + + ], + "4": [ + + ], + "5": [ + + ], + "6": [ + [ + { + "id": "test" + }, + [ + + ] + ] + ], + "7": [ + + ], + "8": [ + [ + { + "id": "test" + }, + [ + [ + [ + [ + + ] + ] + ] + ] + ] + ], + "9": [ + "versions.yml:md5,3fc94714b95c2dc15399a4229d9dd1d9" + ], + "batch_summary": [ + [ + { + "id": "test" + }, + "test-bacteria_odb10-busco.batch_summary.txt:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "busco_dir": [ + [ + { + "id": "test" + }, + [ + [ + [ + [ + + ] + ] + ] + ] + ] + ], + "full_table": [ + + ], + "missing_busco_list": [ + + ], + "seq_dir": [ + [ + { + "id": "test" + }, + [ + + ] + ] + ], + "short_summaries_json": [ + + ], + "short_summaries_txt": [ + + ], + "single_copy_proteins": [ + + ], + "translated_dir": [ + + ], + "versions": [ + "versions.yml:md5,3fc94714b95c2dc15399a4229d9dd1d9" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-03T13:28:04.451297" + }, + "test_busco_eukaryote_augustus": { + "content": [ + "test-eukaryota_odb10-busco.batch_summary.txt:md5,3ea3bdc423a461dae514d816bdc61c89", + "versions.yml:md5,3fc94714b95c2dc15399a4229d9dd1d9" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-03T13:26:36.974986" + }, + "test_busco_genome_single_fasta": { + "content": [ + "test-bacteria_odb10-busco.batch_summary.txt:md5,21b3fb771cf36be917cc451540d999be", + "full_table.tsv:md5,638fe7590f442c57361554dae330eca1", + "missing_busco_list.tsv:md5,1530af4fe7673a6d001349537bcd410a", + "versions.yml:md5,3fc94714b95c2dc15399a4229d9dd1d9" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-03T13:22:45.07816" + }, + "test_busco_genome_multi_fasta": { + "content": [ + "test-bacteria_odb10-busco.batch_summary.txt:md5,fcd3c208913e8abda3d6742c43fec5fa", + [ + "full_table.tsv:md5,c657edcc7d0de0175869717551df6e83", + "full_table.tsv:md5,638fe7590f442c57361554dae330eca1" + ], + [ + "missing_busco_list.tsv:md5,aceb66e347a353cb7fca8e2a725f9112", + "missing_busco_list.tsv:md5,1530af4fe7673a6d001349537bcd410a" + ], + "versions.yml:md5,3fc94714b95c2dc15399a4229d9dd1d9" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-03T13:23:50.255602" + }, + "test_busco_eukaryote_metaeuk": { + "content": [ + "test-eukaryota_odb10-busco.batch_summary.txt:md5,ff6d8277e452a83ce9456bbee666feb6", + "full_table.tsv:md5,92b1b1d5cb5ea0e2093d16f00187e8c7", + "missing_busco_list.tsv:md5,0352e563de290bf804c708323c35a9e3", + "versions.yml:md5,3fc94714b95c2dc15399a4229d9dd1d9" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-03T13:25:38.159041" + }, + "test_busco_transcriptome": { + "content": [ + "test-bacteria_odb10-busco.batch_summary.txt:md5,8734b3f379c4c0928e5dd4ea1873dc64", + "full_table.tsv:md5,1b2ce808fdafa744c56b5f781551272d", + "missing_busco_list.tsv:md5,a6931b6470262b997b8b99ea0f1d14a4", + [ + "1024388at2.faa:md5,797d603d262a6595a112e25b73e878b0", + "1054741at2.faa:md5,cd4b928cba6b19b4437746ba507e7195", + "1093223at2.faa:md5,df9549708e5ffcfaee6a74dd70a0e5dc", + "1151822at2.faa:md5,12726afc1cdc40c13392e1596e93df3a", + "143460at2.faa:md5,d887431fd988a5556a523440f02d9594", + "1491686at2.faa:md5,d03362d19979b27306c192f1c74a84e5", + "1504821at2.faa:md5,4f5f6e5c57bac0092c1d85ded73d7e67", + "1574817at2.faa:md5,1153e55998c2929eacad2aed7d08d248", + "1592033at2.faa:md5,bb7a59e5f3a57ba12d10dabf4c77ab57", + "1623045at2.faa:md5,8fe38155feb1802beb97ef7714837bf5", + "1661836at2.faa:md5,6c6d592c2fbb0d7a4e5e1f47a15644f0", + "1674344at2.faa:md5,bb41b44e53565a54cadf0b780532fe08", + "1698718at2.faa:md5,f233860000028eb00329aa85236c71e5", + "1990650at2.faa:md5,34a2d29c5f8b6253159ddb7a43fa1829", + "223233at2.faa:md5,dec6705c7846c989296e73942f953cbc", + "402899at2.faa:md5,acc0f271f9a586d2ce1ee41669b22999", + "505485at2.faa:md5,aa0391f8fa5d9bd19b30d844d5a99845", + "665824at2.faa:md5,47f8ad43b6a6078206feb48c2e552793", + "776861at2.faa:md5,f8b90c13f7c6be828dea3bb920195e3d", + "874197at2.faa:md5,8d22a35a768debe6f376fc695d233a69", + "932854at2.faa:md5,2eff2de1ab83b22f3234a529a44e22bb", + "95696at2.faa:md5,247bfd1aef432f7b5456307768e9149c" + ], + "single_copy_proteins.faa:md5,73e2c5d6a9b0f01f2deea3cc5f21b764", + "versions.yml:md5,3fc94714b95c2dc15399a4229d9dd1d9" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-03T13:27:53.992893" + }, + "test_busco_protein": { + "content": [ + "test-bacteria_odb10-busco.batch_summary.txt:md5,f5a782378f9f94a748aa907381fdef91", + "full_table.tsv:md5,812ab6a0496fccab774643cf40c4f2a8", + "missing_busco_list.tsv:md5,aceb66e347a353cb7fca8e2a725f9112", + "versions.yml:md5,3fc94714b95c2dc15399a4229d9dd1d9" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-03T13:27:12.724862" + } +} \ No newline at end of file diff --git a/modules/nf-core/busco/busco/tests/nextflow.augustus.config b/modules/nf-core/busco/busco/tests/nextflow.augustus.config new file mode 100644 index 00000000..84daa69d --- /dev/null +++ b/modules/nf-core/busco/busco/tests/nextflow.augustus.config @@ -0,0 +1,5 @@ +process { + withName: 'BUSCO_BUSCO' { + ext.args = '--tar --augustus' + } +} diff --git a/modules/nf-core/busco/busco/tests/nextflow.config b/modules/nf-core/busco/busco/tests/nextflow.config new file mode 100644 index 00000000..1ec3fec0 --- /dev/null +++ b/modules/nf-core/busco/busco/tests/nextflow.config @@ -0,0 +1,5 @@ +process { + withName: 'BUSCO_BUSCO' { + ext.args = '--tar' + } +} diff --git a/modules/nf-core/busco/busco/tests/nextflow.metaeuk.config b/modules/nf-core/busco/busco/tests/nextflow.metaeuk.config new file mode 100644 index 00000000..c1418445 --- /dev/null +++ b/modules/nf-core/busco/busco/tests/nextflow.metaeuk.config @@ -0,0 +1,5 @@ +process { + withName: 'BUSCO_BUSCO' { + ext.args = '--tar --metaeuk' + } +} diff --git a/modules/nf-core/busco/busco/tests/old_test.yml b/modules/nf-core/busco/busco/tests/old_test.yml new file mode 100644 index 00000000..75177f5d --- /dev/null +++ b/modules/nf-core/busco/busco/tests/old_test.yml @@ -0,0 +1,624 @@ +- name: busco test_busco_genome_single_fasta + command: nextflow run ./tests/modules/nf-core/busco -entry test_busco_genome_single_fasta -c ./tests/config/nextflow.config + tags: + - busco + files: + - path: output/busco/short_summary.specific.bacteria_odb10.genome.fna.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/short_summary.specific.bacteria_odb10.genome.fna.txt + contains: + - "BUSCO version" + - "The lineage dataset is" + - "BUSCO was run in mode" + - "Complete BUSCOs" + - "Missing BUSCOs" + - "Dependencies and versions" + - path: output/busco/test-bacteria_odb10-busco.batch_summary.txt + md5sum: bc2440f8a68d7fbf931ff911c1c3fdfa + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/bbtools_err.log + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/bbtools_out.log + md5sum: 9caf1a1434414c78562eb0bbb9c0e53f + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/hmmsearch_err.log + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/hmmsearch_out.log + contains: + - "# hmmsearch :: search profile(s) against a sequence database" + - "# target sequence database:" + - "Internal pipeline statistics summary:" + - "[ok]" + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/prodigal_err.log + md5sum: 538510cfc7483498210f01e53fe035ad + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/prodigal_out.log + md5sum: 61050b0706addc9498b2088a2d6efa9a + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/.checkpoint + contains: + - "Tool: prodigal" + - "Completed" + - "jobs" + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/predicted.faa + md5sum: 836e9a80d33d8b89168f07ddc13ee991 + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/predicted.fna + md5sum: 20eeb75f86842e6e136f02bca8b73a9f + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11.faa + md5sum: 836e9a80d33d8b89168f07ddc13ee991 + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11.fna + md5sum: 20eeb75f86842e6e136f02bca8b73a9f + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11_err.log + md5sum: 538510cfc7483498210f01e53fe035ad + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11_out.log + md5sum: 61050b0706addc9498b2088a2d6efa9a + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/.bbtools_output/.checkpoint + contains: + - "Tool: bbtools" + - "Completed" + - "jobs" + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/busco_sequences/fragmented_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/busco_sequences/multi_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/busco_sequences/single_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/full_table.tsv + md5sum: c56edab1dc1522e993c25ae2b730799f + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/hmmer_output.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/missing_busco_list.tsv + md5sum: b533ef30270f27160acce85a22d01bf5 + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/short_summary.json + contains: + - "one_line_summary" + - "mode" + - "lineage_dataset" + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/short_summary.txt + contains: + - "# BUSCO version is:" + - "Results:" + - "busco:" + - path: output/busco/test-bacteria_odb10-busco/logs/busco.log + contains: + - "DEBUG:busco.run_BUSCO" + - "Results from dataset" + - "how to cite BUSCO" + - path: output/busco/versions.yml + +- name: busco test_busco_genome_multi_fasta + command: nextflow run ./tests/modules/nf-core/busco -entry test_busco_genome_multi_fasta -c ./tests/config/nextflow.config + tags: + - busco + files: + - path: output/busco/short_summary.specific.bacteria_odb10.genome.fasta.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/short_summary.specific.bacteria_odb10.genome.fasta.txt + contains: + - "BUSCO version" + - "The lineage dataset is" + - "BUSCO was run in mode" + - "Complete BUSCOs" + - "Missing BUSCOs" + - "Dependencies and versions" + - path: output/busco/short_summary.specific.bacteria_odb10.genome.fna.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/short_summary.specific.bacteria_odb10.genome.fna.txt + contains: + - "BUSCO version" + - "The lineage dataset is" + - "BUSCO was run in mode" + - "Complete BUSCOs" + - "Missing BUSCOs" + - "Dependencies and versions" + - path: output/busco/test-bacteria_odb10-busco.batch_summary.txt + md5sum: 8c64c1a28b086ef2ee444f99cbed5f7d + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/logs/bbtools_err.log + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/logs/bbtools_out.log + md5sum: 8f047bdb33264d22a83920bc2c63f29a + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/logs/hmmsearch_err.log + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/logs/hmmsearch_out.log + contains: + - "# hmmsearch :: search profile(s) against a sequence database" + - "# target sequence database:" + - "Internal pipeline statistics summary:" + - "[ok]" + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/logs/prodigal_err.log + md5sum: c1fdc6977332f53dfe7f632733bb4585 + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/logs/prodigal_out.log + md5sum: 50752acb1c5a20be886bfdfc06635bcb + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/.checkpoint + contains: + - "Tool: prodigal" + - "Completed" + - "jobs" + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/predicted.faa + md5sum: 8166471fc5f08c82fd5643ab42327f9d + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/predicted.fna + md5sum: ddc508a18f60e7f3314534df50cdf8ca + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11.faa + md5sum: 8166471fc5f08c82fd5643ab42327f9d + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11.fna + md5sum: ddc508a18f60e7f3314534df50cdf8ca + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11_err.log + md5sum: c1fdc6977332f53dfe7f632733bb4585 + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11_out.log + md5sum: 50752acb1c5a20be886bfdfc06635bcb + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_4.faa + md5sum: e56fd59c38248dc21ac94355dca98121 + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_4.fna + md5sum: b365f84bf99c68357952e0b98ed7ce42 + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_4_err.log + md5sum: e5f14d7925ba14a0f9850542f3739894 + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_4_out.log + md5sum: d41971bfc1b621d4ffd2633bc47017ea + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/run_bacteria_odb10/.bbtools_output/.checkpoint + contains: + - "Tool: bbtools" + - "Completed" + - "jobs" + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/run_bacteria_odb10/busco_sequences/fragmented_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/run_bacteria_odb10/busco_sequences/multi_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/run_bacteria_odb10/busco_sequences/single_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/run_bacteria_odb10/full_table.tsv + md5sum: c9651b88b10871abc260ee655898e828 + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/run_bacteria_odb10/hmmer_output.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/run_bacteria_odb10/missing_busco_list.tsv + md5sum: 9939309df2da5419de88c32d1435c779 + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/run_bacteria_odb10/short_summary.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/test-bacteria_odb10-busco/genome.fasta/run_bacteria_odb10/short_summary.txt + contains: + - "# BUSCO version is:" + - "Results:" + - "busco:" + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/bbtools_err.log + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/bbtools_out.log + md5sum: 9caf1a1434414c78562eb0bbb9c0e53f + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/hmmsearch_err.log + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/hmmsearch_out.log + contains: + - "# hmmsearch :: search profile(s) against a sequence database" + - "# target sequence database:" + - "Internal pipeline statistics summary:" + - "[ok]" + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/prodigal_err.log + md5sum: 538510cfc7483498210f01e53fe035ad + - path: output/busco/test-bacteria_odb10-busco/genome.fna/logs/prodigal_out.log + md5sum: 61050b0706addc9498b2088a2d6efa9a + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/.checkpoint + contains: + - "Tool: prodigal" + - "Completed" + - "jobs" + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/predicted.faa + md5sum: 836e9a80d33d8b89168f07ddc13ee991 + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/predicted.fna + md5sum: 20eeb75f86842e6e136f02bca8b73a9f + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11.faa + md5sum: 836e9a80d33d8b89168f07ddc13ee991 + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11.fna + md5sum: 20eeb75f86842e6e136f02bca8b73a9f + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11_err.log + md5sum: 538510cfc7483498210f01e53fe035ad + - path: output/busco/test-bacteria_odb10-busco/genome.fna/prodigal_output/predicted_genes/tmp/prodigal_mode_single_code_11_out.log + md5sum: 61050b0706addc9498b2088a2d6efa9a + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/.bbtools_output/.checkpoint + contains: + - "Tool: bbtools" + - "Completed" + - "jobs" + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/busco_sequences/fragmented_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/busco_sequences/multi_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/busco_sequences/single_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/full_table.tsv + md5sum: c56edab1dc1522e993c25ae2b730799f + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/hmmer_output.tar.gz + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/missing_busco_list.tsv + md5sum: b533ef30270f27160acce85a22d01bf5 + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/short_summary.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/test-bacteria_odb10-busco/genome.fna/run_bacteria_odb10/short_summary.txt + contains: + - "# BUSCO version is:" + - "Results:" + - "busco:" + - path: output/busco/test-bacteria_odb10-busco/logs/busco.log + contains: + - "DEBUG:busco.run_BUSCO" + - "Results from dataset" + - "how to cite BUSCO" + - path: output/busco/versions.yml + +- name: busco test_busco_eukaryote_metaeuk + command: nextflow run ./tests/modules/nf-core/busco -entry test_busco_eukaryote_metaeuk -c ./tests/config/nextflow.config + tags: + - busco + files: + - path: output/busco/short_summary.specific.eukaryota_odb10.genome.fasta.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/short_summary.specific.eukaryota_odb10.genome.fasta.txt + contains: + - "BUSCO version" + - "The lineage dataset is" + - "BUSCO was run in mode" + - "Complete BUSCOs" + - "Missing BUSCOs" + - "Dependencies and versions" + - path: output/busco/test-eukaryota_odb10-busco.batch_summary.txt + md5sum: ff6d8277e452a83ce9456bbee666feb6 + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/bbtools_err.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/bbtools_out.log + md5sum: e63debaa653f18f7405d936050abc093 + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/hmmsearch_err.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/hmmsearch_out.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/metaeuk_run1_err.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/metaeuk_run1_out.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/metaeuk_run2_err.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/metaeuk_run2_out.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/.bbtools_output/.checkpoint + contains: + - "Tool: bbtools" + - "Completed" + - "jobs" + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/busco_sequences/fragmented_busco_sequences.tar.gz + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/busco_sequences/multi_copy_busco_sequences.tar.gz + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/busco_sequences/single_copy_busco_sequences.tar.gz + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/full_table.tsv + md5sum: bd880e90b9e5620a58943a3e0f9ff16b + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/hmmer_output.tar.gz + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/.checkpoint + contains: + - "Tool: metaeuk" + - "Completed" + - "jobs" + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/combined_pred_proteins.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/initial_results/genome.fasta.codon.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/initial_results/genome.fasta.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/initial_results/genome.fasta.gff + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/initial_results/genome.fasta.headersMap.tsv + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/refseq_db_rerun.faa + md5sum: d80b8fa4cb5ed0d47d63d6aa93635bc2 + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/rerun_results/genome.fasta.codon.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/rerun_results/genome.fasta.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/rerun_results/genome.fasta.gff + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/rerun_results/genome.fasta.headersMap.tsv + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/missing_busco_list.tsv + md5sum: 1e8e79c540fd2e69ba0d2659d9eb2988 + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/short_summary.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/short_summary.txt + contains: + - "# BUSCO version is:" + - "Results:" + - "busco:" + - path: output/busco/test-eukaryota_odb10-busco/logs/busco.log + contains: + - "DEBUG:busco.run_BUSCO" + - "Results from dataset" + - "how to cite BUSCO" + - path: output/busco/versions.yml + +- name: busco test_busco_eukaryote_augustus + command: nextflow run ./tests/modules/nf-core/busco -entry test_busco_eukaryote_augustus -c ./tests/config/nextflow.config + tags: + - busco + files: + - path: output/busco/short_summary.specific.eukaryota_odb10.genome.fasta.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/short_summary.specific.eukaryota_odb10.genome.fasta.txt + contains: + - "BUSCO version" + - "The lineage dataset is" + - "BUSCO was run in mode" + - "Complete BUSCOs" + - "Missing BUSCOs" + - "Dependencies and versions" + - path: output/busco/test-eukaryota_odb10-busco.batch_summary.txt + md5sum: ff6d8277e452a83ce9456bbee666feb6 + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/bbtools_err.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/bbtools_out.log + md5sum: e63debaa653f18f7405d936050abc093 + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/hmmsearch_err.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/hmmsearch_out.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/metaeuk_run1_err.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/metaeuk_run1_out.log + contains: + - "metaeuk" + - "easy-predict" + - "Compute score and coverage" + - "Time for processing:" + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/metaeuk_run2_err.log + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/logs/metaeuk_run2_out.log + contains: + - "metaeuk" + - "easy-predict" + - "Compute score and coverage" + - "Time for processing:" + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/.bbtools_output/.checkpoint + contains: + - "Tool: bbtools" + - "Completed" + - "jobs" + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/busco_sequences/fragmented_busco_sequences.tar.gz + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/busco_sequences/multi_copy_busco_sequences.tar.gz + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/busco_sequences/single_copy_busco_sequences.tar.gz + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/full_table.tsv + md5sum: bd880e90b9e5620a58943a3e0f9ff16b + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/hmmer_output.tar.gz + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/.checkpoint + contains: + - "Tool: metaeuk" + - "Completed" + - "jobs" + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/combined_pred_proteins.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/initial_results/genome.fasta.codon.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/initial_results/genome.fasta.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/initial_results/genome.fasta.gff + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/initial_results/genome.fasta.headersMap.tsv + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/refseq_db_rerun.faa + md5sum: d80b8fa4cb5ed0d47d63d6aa93635bc2 + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/rerun_results/genome.fasta.codon.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/rerun_results/genome.fasta.fas + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/rerun_results/genome.fasta.gff + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/metaeuk_output/rerun_results/genome.fasta.headersMap.tsv + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/missing_busco_list.tsv + md5sum: 1e8e79c540fd2e69ba0d2659d9eb2988 + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/short_summary.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/test-eukaryota_odb10-busco/genome.fasta/run_eukaryota_odb10/short_summary.txt + contains: + - "# BUSCO version is:" + - "Results:" + - "busco:" + - path: output/busco/test-eukaryota_odb10-busco/logs/busco.log + contains: + - "DEBUG:busco.run_BUSCO" + - "Results from dataset" + - "how to cite BUSCO" + - path: output/busco/versions.yml + +- name: busco test_busco_protein + command: nextflow run ./tests/modules/nf-core/busco -entry test_busco_protein -c ./tests/config/nextflow.config + tags: + - busco + files: + - path: output/busco/short_summary.specific.bacteria_odb10.proteome.fasta.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/short_summary.specific.bacteria_odb10.proteome.fasta.txt + contains: + - "BUSCO version" + - "The lineage dataset is" + - "BUSCO was run in mode" + - "Complete BUSCOs" + - "Missing BUSCOs" + - "Dependencies and versions" + - path: output/busco/test-bacteria_odb10-busco.batch_summary.txt + md5sum: 7a65e6cbb6c56a2ea4e739ae0aa3297d + - path: output/busco/test-bacteria_odb10-busco/logs/busco.log + contains: + - "DEBUG:busco.run_BUSCO" + - "Results from dataset" + - "how to cite BUSCO" + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/logs/hmmsearch_err.log + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/logs/hmmsearch_out.log + contains: + - "# hmmsearch :: search profile(s) against a sequence database" + - "# target sequence database:" + - "Internal pipeline statistics summary:" + - "[ok]" + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/run_bacteria_odb10/busco_sequences/fragmented_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/run_bacteria_odb10/busco_sequences/multi_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/run_bacteria_odb10/busco_sequences/single_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/run_bacteria_odb10/full_table.tsv + md5sum: 0e34f1011cd83ea1d5d5103ec62b8922 + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/run_bacteria_odb10/hmmer_output.tar.gz + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/run_bacteria_odb10/missing_busco_list.tsv + md5sum: 9939309df2da5419de88c32d1435c779 + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/run_bacteria_odb10/short_summary.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/test-bacteria_odb10-busco/proteome.fasta/run_bacteria_odb10/short_summary.txt + contains: + - "# BUSCO version is:" + - "Results:" + - "busco:" + - path: output/busco/versions.yml + +- name: busco test_busco_transcriptome + command: nextflow run ./tests/modules/nf-core/busco -entry test_busco_transcriptome -c ./tests/config/nextflow.config + tags: + - busco + files: + - path: output/busco/short_summary.specific.bacteria_odb10.test1.contigs.fa.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/short_summary.specific.bacteria_odb10.test1.contigs.fa.txt + contains: + - "BUSCO version" + - "The lineage dataset is" + - "BUSCO was run in mode" + - "Complete BUSCOs" + - "Missing BUSCOs" + - "Dependencies and versions" + - path: output/busco/test-bacteria_odb10-busco.batch_summary.txt + md5sum: 46118ecf60d1b87d22b96d80f4f03632 + - path: output/busco/test-bacteria_odb10-busco/logs/busco.log + contains: + - "DEBUG:busco.run_BUSCO" + - "Results from dataset" + - "how to cite BUSCO" + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/blast_db/.checkpoint + contains: + - "Tool: makeblastdb" + - "Completed" + - "jobs" + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/blast_db/test1.contigs.fa.ndb + md5sum: 3788c017fe5e6f0f58224e9cdd21822b + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/blast_db/test1.contigs.fa.nhr + md5sum: 8ecd2ce392bb5e25ddbe1d85f879582e + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/blast_db/test1.contigs.fa.nin + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/blast_db/test1.contigs.fa.njs + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/blast_db/test1.contigs.fa.not + md5sum: 0c340e376c7e85d19f82ec1a833e6a6e + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/blast_db/test1.contigs.fa.nsq + md5sum: 532d5c0a7ea00fe95ca3c97cb3be6198 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/blast_db/test1.contigs.fa.ntf + md5sum: de1250813f0c7affc6d12dac9d0fb6bb + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/blast_db/test1.contigs.fa.nto + md5sum: ff74bd41f9cc9b011c63a32c4f7693bf + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/logs/hmmsearch_err.log + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/logs/hmmsearch_out.log + contains: + - "# hmmsearch :: search profile(s) against a sequence database" + - "# target sequence database:" + - "Internal pipeline statistics summary:" + - "[ok]" + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/logs/makeblastdb_err.log + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/logs/makeblastdb_out.log + contains: + - "Building a new DB" + - "Adding sequences from FASTA" + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/logs/tblastn_err.log + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/logs/tblastn_out.log + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/.checkpoint + contains: + - "Tool: tblastn" + - "Completed" + - "jobs" + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/coordinates.tsv + md5sum: cc30eed321944af293452bdbcfc24292 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_101.temp + md5sum: 73e9c65fc83fedc58f57f09b08f08238 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_119.temp + md5sum: 7fa4cc7955ec0cc36330a221c579b975 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_129.temp + md5sum: 6f1601c875d019e3f6f1f98ed8e988d4 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_138.temp + md5sum: 3f8e034686cd240c2330650d791bcae2 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_143.temp + md5sum: df3dfa8e9ba30ed70cf75b5e7abf2179 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_172.temp + md5sum: 7d463e0e6cf7169bc9077d8dc776dda1 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_178.temp + md5sum: 2288edf7fa4f88f51b4cf4d94086f77e + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_188.temp + md5sum: 029906abbad6d87fc57830dd548cac24 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_195.temp + md5sum: 4937f3b348774a31b1160a00297c29cc + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_210.temp + md5sum: afcb20ba4c466479d6b91c8c62251e1f + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_232.temp + md5sum: 2e1e823ce017345bd998191a39fa9924 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_268.temp + md5sum: 08c2d82c34ecffbe1c638b410349412e + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_29.temp + md5sum: cd9b63cf93524284781535c888313764 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_44.temp + md5sum: d1929b742b24ebe379bf4801ca882dca + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_58.temp + md5sum: 69215765b010c05336538cb322c900b3 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_72.temp + md5sum: 6feaa1cc3b0899a147ea9d466878f3e3 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_80.temp + md5sum: 13625eae14e860a96ce17cd4e37e9d01 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_81.temp + md5sum: e14b2484649b0dbc8926815c207b806d + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_93.temp + md5sum: 6902c93691df00e690faea914c71839e + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/sequences/k141_97.temp + md5sum: 0a0d9d38a83acbd5ad43c29cdf429988 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/blast_output/tblastn.tsv + contains: + - "TBLASTN" + - "BLAST processed" + - "queries" + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/busco_sequences/fragmented_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/busco_sequences/multi_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/busco_sequences/single_copy_busco_sequences.tar.gz + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/full_table.tsv + md5sum: 24df25199e13c88bd892fc3e7b541ca0 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/hmmer_output.tar.gz + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/missing_busco_list.tsv + md5sum: e7232e2b8cca4fdfdd9e363b39ebbc81 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/short_summary.json + contains: + - "one_line_summary" + - "mode" + - "dataset" + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/short_summary.txt + contains: + - "# BUSCO version is:" + - "Results:" + - "busco:" + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/run_bacteria_odb10/single_copy_proteins.faa + md5sum: e04b9465733577ae6e4bccb7aa01e720 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1024388at2.faa + md5sum: 7333c39a20258f20c7019ea0cd83157c + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1054741at2.faa + md5sum: ebb481e77a824685fbe04d8a2f3a0d7d + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1093223at2.faa + md5sum: 34621c7d499034e8f8e6b92fd4020a93 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1151822at2.faa + md5sum: aa89ca381c1c70c9c4e1380351ca7c2a + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/143460at2.faa + md5sum: f2e91d78b8dd3722840378789f29e8c8 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1491686at2.faa + md5sum: 73c25aef5c9cba7f4151804941b146ea + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1504821at2.faa + md5sum: cda556018d1f84ebe517e89f6fc107d0 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1574817at2.faa + md5sum: a9096c9fb8b25c78a72871ab0463acdc + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1592033at2.faa + md5sum: e463d25ce186c0cebfd749474f3a4c64 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1623045at2.faa + md5sum: f2cfd241590c6d8377286d6135480937 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1661836at2.faa + md5sum: 586569546fb9861502468e3d9ba2775c + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1674344at2.faa + md5sum: 24c658bee14ad84b062d81ad96642eb8 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1698718at2.faa + md5sum: 0b8e26ddf5149bbd8805be7af125208d + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/1990650at2.faa + md5sum: 159320712ee01fb2ccb31a25df44eead + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/223233at2.faa + md5sum: 812629c0b06ac3d18661c2ca78de0c08 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/402899at2.faa + md5sum: f7ff4e1591342d30b77392a2e84b57d9 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/505485at2.faa + md5sum: 7b34a24fc49c540d46fcf96ff5129564 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/665824at2.faa + md5sum: 4cff2df64f6bcaff8bc19c234c8bcccd + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/776861at2.faa + md5sum: 613af7a3fea30ea2bece66f603b9284a + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/874197at2.faa + md5sum: a7cd1b13c9ef91c7ef4e31614166f197 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/932854at2.faa + md5sum: fe313ffd5efdb0fed887a04fba352552 + - path: output/busco/test-bacteria_odb10-busco/test1.contigs.fa/translated_proteins/95696at2.faa + md5sum: 4e1f30a2fea4dfbf9bb7fae2700622a0 + - path: output/busco/versions.yml diff --git a/modules/nf-core/busco/busco/tests/tags.yml b/modules/nf-core/busco/busco/tests/tags.yml new file mode 100644 index 00000000..7c4d2835 --- /dev/null +++ b/modules/nf-core/busco/busco/tests/tags.yml @@ -0,0 +1,2 @@ +busco/busco: + - "modules/nf-core/busco/busco/**" diff --git a/modules/nf-core/busco/meta.yml b/modules/nf-core/busco/meta.yml deleted file mode 100644 index 90b30d4d..00000000 --- a/modules/nf-core/busco/meta.yml +++ /dev/null @@ -1,96 +0,0 @@ -name: busco -description: Benchmarking Universal Single Copy Orthologs -keywords: - - quality control - - genome - - transcriptome - - proteome -tools: - - busco: - description: BUSCO provides measures for quantitative assessment of genome assembly, gene set, and transcriptome completeness based on evolutionarily informed expectations of gene content from near-universal single-copy orthologs selected from OrthoDB. - homepage: https://busco.ezlab.org/ - documentation: https://busco.ezlab.org/busco_userguide.html - tool_dev_url: https://gitlab.com/ezlab/busco - doi: "10.1007/978-1-4939-9173-0_14" - licence: ["MIT"] -input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - fasta: - type: file - description: Nucleic or amino acid sequence file in FASTA format. - pattern: "*.{fasta,fna,fa,fasta.gz,fna.gz,fa.gz}" - - mode: - type: string - description: The mode to run Busco in. One of genome, proteins, or transcriptome - pattern: "{genome,proteins,transcriptome}" - - lineage: - type: string - description: The BUSCO lineage to use, or "auto" to automatically select lineage - - busco_lineages_path: - type: directory - description: Path to local BUSCO lineages directory. - - config_file: - type: file - description: Path to BUSCO config file. -output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - batch_summary: - type: file - description: Summary of all sequence files analyzed - pattern: "*-busco.batch_summary.txt" - - short_summaries_txt: - type: file - description: Short Busco summary in plain text format - pattern: "short_summary.*.txt" - - short_summaries_json: - type: file - description: Short Busco summary in JSON format - pattern: "short_summary.*.json" - - busco_dir: - type: directory - description: BUSCO lineage specific output - pattern: "*-busco" - - full_table: - type: file - description: Full BUSCO results table - pattern: "full_table.tsv" - - missing_busco_list: - type: file - description: List of missing BUSCOs - pattern: "missing_busco_list.tsv" - - single_copy_proteins: - type: file - description: Fasta file of single copy proteins (transcriptome mode) - pattern: "single_copy_proteins.faa" - - seq_dir: - type: directory - description: BUSCO sequence directory - pattern: "busco_sequences" - - translated_proteins: - type: directory - description: Six frame translations of each transcript made by the transcriptome mode - pattern: "translated_proteins" - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" -authors: - - "@priyanka-surana" - - "@charles-plessy" - - "@mahesh-panchal" - - "@muffato" - - "@jvhagey" -maintainers: - - "@priyanka-surana" - - "@charles-plessy" - - "@mahesh-panchal" - - "@muffato" - - "@jvhagey" diff --git a/modules/nf-core/bwamem2/index/environment.yml b/modules/nf-core/bwamem2/index/environment.yml index 26b43917..15cee238 100644 --- a/modules/nf-core/bwamem2/index/environment.yml +++ b/modules/nf-core/bwamem2/index/environment.yml @@ -1,7 +1,5 @@ -name: bwamem2_index channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::bwa-mem2=2.2.1 diff --git a/modules/nf-core/bwamem2/index/meta.yml b/modules/nf-core/bwamem2/index/meta.yml index c14a1092..74f54ef0 100644 --- a/modules/nf-core/bwamem2/index/meta.yml +++ b/modules/nf-core/bwamem2/index/meta.yml @@ -13,29 +13,32 @@ tools: homepage: https://github.com/bwa-mem2/bwa-mem2 documentation: https://github.com/bwa-mem2/bwa-mem2#usage licence: ["MIT"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - fasta: - type: file - description: Input genome fasta file + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - fasta: + type: file + description: Input genome fasta file output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - index: - type: file - description: BWA genome index files - pattern: "*.{0123,amb,ann,bwt.2bit.64,pac}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - bwamem2: + type: file + description: BWA genome index files + pattern: "*.{0123,amb,ann,bwt.2bit.64,pac}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@maxulysse" maintainers: diff --git a/modules/nf-core/bwamem2/index/tests/main.nf.test b/modules/nf-core/bwamem2/index/tests/main.nf.test new file mode 100644 index 00000000..dbf11132 --- /dev/null +++ b/modules/nf-core/bwamem2/index/tests/main.nf.test @@ -0,0 +1,31 @@ +nextflow_process { + + name "Test Process BWAMEM2_INDEX" + tag "modules_nfcore" + tag "modules" + tag "bwamem2" + tag "bwamem2/index" + script "../main.nf" + process "BWAMEM2_INDEX" + + test("BWAMEM2 index") { + + when { + process { + """ + input[0] = [ + [id: 'test'], + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } +} diff --git a/modules/nf-core/bwamem2/index/tests/main.nf.test.snap b/modules/nf-core/bwamem2/index/tests/main.nf.test.snap new file mode 100644 index 00000000..69b268ee --- /dev/null +++ b/modules/nf-core/bwamem2/index/tests/main.nf.test.snap @@ -0,0 +1,47 @@ +{ + "BWAMEM2 index": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + [ + "genome.fasta.0123:md5,b02870de80106104abcb03cd9463e7d8", + "genome.fasta.amb:md5,3a68b8b2287e07dd3f5f95f4344ba76e", + "genome.fasta.ann:md5,c32e11f6c859f166c7525a9c1d583567", + "genome.fasta.bwt.2bit.64:md5,d097a1b82dee375d41a1ea69895a9216", + "genome.fasta.pac:md5,983e3d2cd6f36e2546e6d25a0da78d66" + ] + ] + ], + "1": [ + "versions.yml:md5,9ffd13d12e7108ed15c58566bc4717d6" + ], + "index": [ + [ + { + "id": "test" + }, + [ + "genome.fasta.0123:md5,b02870de80106104abcb03cd9463e7d8", + "genome.fasta.amb:md5,3a68b8b2287e07dd3f5f95f4344ba76e", + "genome.fasta.ann:md5,c32e11f6c859f166c7525a9c1d583567", + "genome.fasta.bwt.2bit.64:md5,d097a1b82dee375d41a1ea69895a9216", + "genome.fasta.pac:md5,983e3d2cd6f36e2546e6d25a0da78d66" + ] + ] + ], + "versions": [ + "versions.yml:md5,9ffd13d12e7108ed15c58566bc4717d6" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.02.0" + }, + "timestamp": "2024-03-18T12:59:39.132616" + } +} \ No newline at end of file diff --git a/modules/nf-core/bwamem2/index/tests/tags.yml b/modules/nf-core/bwamem2/index/tests/tags.yml new file mode 100644 index 00000000..3953018e --- /dev/null +++ b/modules/nf-core/bwamem2/index/tests/tags.yml @@ -0,0 +1,2 @@ +bwamem2/index: + - modules/nf-core/bwamem2/index/** diff --git a/modules/nf-core/cat/cat/environment.yml b/modules/nf-core/cat/cat/environment.yml index 17a04ef2..9b01c865 100644 --- a/modules/nf-core/cat/cat/environment.yml +++ b/modules/nf-core/cat/cat/environment.yml @@ -1,7 +1,5 @@ -name: cat_cat channels: - conda-forge - bioconda - - defaults dependencies: - conda-forge::pigz=2.3.4 diff --git a/modules/nf-core/cat/cat/main.nf b/modules/nf-core/cat/cat/main.nf index adbdbd7b..2862c64c 100644 --- a/modules/nf-core/cat/cat/main.nf +++ b/modules/nf-core/cat/cat/main.nf @@ -76,4 +76,3 @@ def getFileSuffix(filename) { def match = filename =~ /^.*?((\.\w{1,5})?(\.\w{1,5}\.gz$))/ return match ? match[0][1] : filename.substring(filename.lastIndexOf('.')) } - diff --git a/modules/nf-core/cat/cat/meta.yml b/modules/nf-core/cat/cat/meta.yml index 00a8db0b..81778a06 100644 --- a/modules/nf-core/cat/cat/meta.yml +++ b/modules/nf-core/cat/cat/meta.yml @@ -9,25 +9,32 @@ tools: description: Just concatenation documentation: https://man7.org/linux/man-pages/man1/cat.1.html licence: ["GPL-3.0-or-later"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - files_in: - type: file - description: List of compressed / uncompressed files - pattern: "*" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - files_in: + type: file + description: List of compressed / uncompressed files + pattern: "*" output: - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - file_out: - type: file - description: Concatenated file. Will be gzipped if file_out ends with ".gz" - pattern: "${file_out}" + - meta: + type: file + description: Concatenated file. Will be gzipped if file_out ends with ".gz" + pattern: "${file_out}" + - ${prefix}: + type: file + description: Concatenated file. Will be gzipped if file_out ends with ".gz" + pattern: "${file_out}" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@erikrikarddaniel" - "@FriederikeHanssen" diff --git a/modules/nf-core/cat/cat/tests/main.nf.test b/modules/nf-core/cat/cat/tests/main.nf.test index aaae04f9..9cb16178 100644 --- a/modules/nf-core/cat/cat/tests/main.nf.test +++ b/modules/nf-core/cat/cat/tests/main.nf.test @@ -19,8 +19,8 @@ nextflow_process { [ [ id:'genome', single_end:true ], [ - file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true), - file(params.test_data['sarscov2']['genome']['genome_sizes'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.sizes', checkIfExists: true) ] ] """ @@ -29,7 +29,8 @@ nextflow_process { then { assertAll( { assert !process.success }, - { assert process.stdout.toString().contains("The name of the input file can't be the same as for the output prefix") } + { assert process.stdout.toString().contains("The name of the input file can't be the same as for the output prefix") }, + { assert snapshot(process.out.versions).match() } ) } } @@ -45,8 +46,8 @@ nextflow_process { [ [ id:'test', single_end:true ], [ - file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true), - file(params.test_data['sarscov2']['genome']['genome_sizes'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.sizes', checkIfExists: true) ] ] """ @@ -72,8 +73,8 @@ nextflow_process { [ [ id:'test', single_end:true ], [ - file(params.test_data['sarscov2']['genome']['genome_gff3_gz'], checkIfExists: true), - file(params.test_data['sarscov2']['genome']['contigs_genome_maf_gz'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.gff3.gz', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/alignment/last/contigs.genome.maf.gz', checkIfExists: true) ] ] """ @@ -83,7 +84,12 @@ nextflow_process { def lines = path(process.out.file_out.get(0).get(1)).linesGzip assertAll( { assert process.success }, - { assert snapshot(process.out).match() } + { assert snapshot( + lines[0..5], + lines.size(), + process.out.versions + ).match() + } ) } } @@ -101,8 +107,8 @@ nextflow_process { [ [ id:'test', single_end:true ], [ - file(params.test_data['sarscov2']['genome']['genome_gff3_gz'], checkIfExists: true), - file(params.test_data['sarscov2']['genome']['contigs_genome_maf_gz'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.gff3.gz', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/alignment/last/contigs.genome.maf.gz', checkIfExists: true) ] ] """ @@ -130,8 +136,8 @@ nextflow_process { [ [ id:'test', single_end:true ], [ - file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true), - file(params.test_data['sarscov2']['genome']['genome_sizes'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.sizes', checkIfExists: true) ] ] """ @@ -141,7 +147,12 @@ nextflow_process { def lines = path(process.out.file_out.get(0).get(1)).linesGzip assertAll( { assert process.success }, - { assert snapshot(process.out).match() } + { assert snapshot( + lines[0..5], + lines.size(), + process.out.versions + ).match() + } ) } } @@ -158,7 +169,7 @@ nextflow_process { [ [ id:'test', single_end:true ], [ - file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) ] ] """ @@ -168,10 +179,13 @@ nextflow_process { def lines = path(process.out.file_out.get(0).get(1)).linesGzip assertAll( { assert process.success }, - { assert snapshot(lines[0..5]).match("test_cat_one_file_unzipped_zipped_lines") }, - { assert snapshot(lines.size()).match("test_cat_one_file_unzipped_zipped_size")} + { assert snapshot( + lines[0..5], + lines.size(), + process.out.versions + ).match() + } ) } } } - diff --git a/modules/nf-core/cat/cat/tests/main.nf.test.snap b/modules/nf-core/cat/cat/tests/main.nf.test.snap index 0c9bfe8d..b7623ee6 100644 --- a/modules/nf-core/cat/cat/tests/main.nf.test.snap +++ b/modules/nf-core/cat/cat/tests/main.nf.test.snap @@ -28,6 +28,10 @@ ] } ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.3" + }, "timestamp": "2023-10-16T14:32:18.500464399" }, "test_cat_zipped_unzipped": { @@ -59,40 +63,46 @@ ] } ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.3" + }, "timestamp": "2023-10-16T14:32:49.642741302" }, "test_cat_zipped_zipped": { "content": [ - { - "0": [ - [ - { - "id": "test", - "single_end": true - }, - "test.gff3.gz:md5,c439d3b60e7bc03e8802a451a0d9a5d9" - ] - ], - "1": [ - "versions.yml:md5,115ed6177ebcff24eb99d503fa5ef894" - ], - "file_out": [ - [ - { - "id": "test", - "single_end": true - }, - "test.gff3.gz:md5,c439d3b60e7bc03e8802a451a0d9a5d9" - ] - ], - "versions": [ - "versions.yml:md5,115ed6177ebcff24eb99d503fa5ef894" - ] - } + [ + "MT192765.1\tGenbank\ttranscript\t259\t29667\t.\t+\t.\tID=unknown_transcript_1;geneID=orf1ab;gene_name=orf1ab", + "MT192765.1\tGenbank\tgene\t259\t21548\t.\t+\t.\tParent=unknown_transcript_1", + "MT192765.1\tGenbank\tCDS\t259\t13461\t.\t+\t0\tParent=unknown_transcript_1;exception=\"ribosomal slippage\";gbkey=CDS;gene=orf1ab;note=\"pp1ab;translated=by -1 ribosomal frameshift\";product=\"orf1ab polyprotein\";protein_id=QIK50426.1", + "MT192765.1\tGenbank\tCDS\t13461\t21548\t.\t+\t0\tParent=unknown_transcript_1;exception=\"ribosomal slippage\";gbkey=CDS;gene=orf1ab;note=\"pp1ab;translated=by -1 ribosomal frameshift\";product=\"orf1ab polyprotein\";protein_id=QIK50426.1", + "MT192765.1\tGenbank\tCDS\t21556\t25377\t.\t+\t0\tParent=unknown_transcript_1;gbkey=CDS;gene=S;note=\"structural protein\";product=\"surface glycoprotein\";protein_id=QIK50427.1", + "MT192765.1\tGenbank\tgene\t21556\t25377\t.\t+\t.\tParent=unknown_transcript_1" + ], + 78, + [ + "versions.yml:md5,115ed6177ebcff24eb99d503fa5ef894" + ] ], - "timestamp": "2024-01-12T14:02:02.999254641" + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.3" + }, + "timestamp": "2024-07-22T11:51:46.802978" }, - "test_cat_one_file_unzipped_zipped_lines": { + "test_cat_name_conflict": { + "content": [ + [ + + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.3" + }, + "timestamp": "2024-07-22T11:51:29.45394" + }, + "test_cat_one_file_unzipped_zipped": { "content": [ [ ">MT192765.1 Severe acute respiratory syndrome coronavirus 2 isolate SARS-CoV-2/human/USA/PC00101P/2020, complete genome", @@ -101,45 +111,37 @@ "TAACTCGTCTATCTTCTGCAGGCTGCTTACGGTTTCGTCCGTGTTGCAGCCGATCATCAGCACATCTAGGTTTTGTCCGG", "GTGTGACCGAAAGGTAAGATGGAGAGCCTTGTCCCTGGTTTCAACGAGAAAACACACGTCCAACTCAGTTTGCCTGTTTT", "ACAGGTTCGCGACGTGCTCGTACGTGGCTTTGGAGACTCCGTGGAGGAGGTCTTATCAGAGGCACGTCAACATCTTAAAG" + ], + 374, + [ + "versions.yml:md5,115ed6177ebcff24eb99d503fa5ef894" ] ], - "timestamp": "2023-10-16T14:33:21.39642399" + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.3" + }, + "timestamp": "2024-07-22T11:52:02.774016" }, "test_cat_unzipped_zipped": { "content": [ - { - "0": [ - [ - { - "id": "test", - "single_end": true - }, - "cat.txt.gz:md5,f44b33a0e441ad58b2d3700270e2dbe2" - ] - ], - "1": [ - "versions.yml:md5,115ed6177ebcff24eb99d503fa5ef894" - ], - "file_out": [ - [ - { - "id": "test", - "single_end": true - }, - "cat.txt.gz:md5,f44b33a0e441ad58b2d3700270e2dbe2" - ] - ], - "versions": [ - "versions.yml:md5,115ed6177ebcff24eb99d503fa5ef894" - ] - } - ], - "timestamp": "2024-01-12T14:08:26.948048418" - }, - "test_cat_one_file_unzipped_zipped_size": { - "content": [ - 374 + [ + ">MT192765.1 Severe acute respiratory syndrome coronavirus 2 isolate SARS-CoV-2/human/USA/PC00101P/2020, complete genome", + "GTTTATACCTTCCCAGGTAACAAACCAACCAACTTTCGATCTCTTGTAGATCTGTTCTCTAAACGAACTTTAAAATCTGT", + "GTGGCTGTCACTCGGCTGCATGCTTAGTGCACTCACGCAGTATAATTAATAACTAATTACTGTCGTTGACAGGACACGAG", + "TAACTCGTCTATCTTCTGCAGGCTGCTTACGGTTTCGTCCGTGTTGCAGCCGATCATCAGCACATCTAGGTTTTGTCCGG", + "GTGTGACCGAAAGGTAAGATGGAGAGCCTTGTCCCTGGTTTCAACGAGAAAACACACGTCCAACTCAGTTTGCCTGTTTT", + "ACAGGTTCGCGACGTGCTCGTACGTGGCTTTGGAGACTCCGTGGAGGAGGTCTTATCAGAGGCACGTCAACATCTTAAAG" + ], + 375, + [ + "versions.yml:md5,115ed6177ebcff24eb99d503fa5ef894" + ] ], - "timestamp": "2024-01-12T14:10:22.445700266" + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.3" + }, + "timestamp": "2024-07-22T11:51:57.581523" } -} +} \ No newline at end of file diff --git a/modules/nf-core/cooler/cload/environment.yml b/modules/nf-core/cooler/cload/environment.yml index 03abee73..f8165ca9 100644 --- a/modules/nf-core/cooler/cload/environment.yml +++ b/modules/nf-core/cooler/cload/environment.yml @@ -1,7 +1,5 @@ -name: cooler_cload channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::cooler=0.9.2 diff --git a/modules/nf-core/cooler/cload/meta.yml b/modules/nf-core/cooler/cload/meta.yml index fa5474ae..1bb9f748 100644 --- a/modules/nf-core/cooler/cload/meta.yml +++ b/modules/nf-core/cooler/cload/meta.yml @@ -13,41 +13,45 @@ tools: tool_dev_url: https://github.com/open2c/cooler doi: "10.1093/bioinformatics/btz540" licence: ["BSD-3-clause"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - pairs: - type: file - description: Path to contacts (i.e. read pairs) file. - - index: - type: file - description: Path to index file of the contacts. - - cool_bin: - type: integer - description: Bins size in bp - - chromsizes: - type: file - description: Path to a chromsizes file. + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - pairs: + type: file + description: Path to contacts (i.e. read pairs) file. + - index: + type: file + description: Path to index file of the contacts. + - cool_bin: + type: integer + description: Bins size in bp + - - chromsizes: + type: file + description: Path to a chromsizes file. output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - version: - type: file - description: File containing software version - pattern: "versions.yml" - cool: - type: file - description: Output COOL file path - pattern: "*.cool" - - cool_bin: - type: integer - description: Bins size in bp + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.cool": + type: file + description: Output COOL file path + pattern: "*.cool" + - cool_bin: + type: file + description: Output COOL file path + pattern: "*.cool" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@jianhong" - "@muffato" diff --git a/modules/nf-core/cooler/zoomify/environment.yml b/modules/nf-core/cooler/zoomify/environment.yml index 2288f376..f8165ca9 100644 --- a/modules/nf-core/cooler/zoomify/environment.yml +++ b/modules/nf-core/cooler/zoomify/environment.yml @@ -1,7 +1,5 @@ -name: cooler_zoomify channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::cooler=0.9.2 diff --git a/modules/nf-core/cooler/zoomify/meta.yml b/modules/nf-core/cooler/zoomify/meta.yml index d87aaf29..3f928781 100644 --- a/modules/nf-core/cooler/zoomify/meta.yml +++ b/modules/nf-core/cooler/zoomify/meta.yml @@ -12,30 +12,33 @@ tools: tool_dev_url: https://github.com/open2c/cooler doi: "10.1093/bioinformatics/btz540" licence: ["BSD-3-clause"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - cool: - type: file - description: Path to COOL file - pattern: "*.{cool,mcool}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - cool: + type: file + description: Path to COOL file + pattern: "*.{cool,mcool}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - mcool: - type: file - description: Output mcool file - pattern: "*.mcool" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.mcool": + type: file + description: Output mcool file + pattern: "*.mcool" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@jianhong" maintainers: diff --git a/modules/nf-core/custom/dumpsoftwareversions/environment.yml b/modules/nf-core/custom/dumpsoftwareversions/environment.yml index 9b3272bc..9d79af93 100644 --- a/modules/nf-core/custom/dumpsoftwareversions/environment.yml +++ b/modules/nf-core/custom/dumpsoftwareversions/environment.yml @@ -1,7 +1,5 @@ -name: custom_dumpsoftwareversions channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::multiqc=1.19 + - bioconda::multiqc=1.20 diff --git a/modules/nf-core/custom/dumpsoftwareversions/main.nf b/modules/nf-core/custom/dumpsoftwareversions/main.nf index f2187611..105f9265 100644 --- a/modules/nf-core/custom/dumpsoftwareversions/main.nf +++ b/modules/nf-core/custom/dumpsoftwareversions/main.nf @@ -4,8 +4,8 @@ process CUSTOM_DUMPSOFTWAREVERSIONS { // Requires `pyyaml` which does not have a dedicated container but is in the MultiQC container conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/multiqc:1.19--pyhdfd78af_0' : - 'biocontainers/multiqc:1.19--pyhdfd78af_0' }" + 'https://depot.galaxyproject.org/singularity/multiqc:1.20--pyhdfd78af_0' : + 'biocontainers/multiqc:1.20--pyhdfd78af_0' }" input: path versions diff --git a/modules/nf-core/custom/dumpsoftwareversions/meta.yml b/modules/nf-core/custom/dumpsoftwareversions/meta.yml index 5f15a5fd..dc1e412f 100644 --- a/modules/nf-core/custom/dumpsoftwareversions/meta.yml +++ b/modules/nf-core/custom/dumpsoftwareversions/meta.yml @@ -1,34 +1,40 @@ # yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/meta-schema.json name: custom_dumpsoftwareversions -description: Custom module used to dump software versions within the nf-core pipeline template +description: Custom module used to dump software versions within the nf-core pipeline + template keywords: - custom - dump - version tools: - custom: - description: Custom module used to dump software versions within the nf-core pipeline template + description: Custom module used to dump software versions within the nf-core pipeline + template homepage: https://github.com/nf-core/tools documentation: https://github.com/nf-core/tools licence: ["MIT"] + identifier: "" input: - - versions: - type: file - description: YML file containing software versions - pattern: "*.yml" + - - versions: + type: file + description: YML file containing software versions + pattern: "*.yml" output: - yml: - type: file - description: Standard YML file containing software versions - pattern: "software_versions.yml" + - software_versions.yml: + type: file + description: Standard YML file containing software versions + pattern: "software_versions.yml" - mqc_yml: - type: file - description: MultiQC custom content YML file containing software versions - pattern: "software_versions_mqc.yml" + - software_versions_mqc.yml: + type: file + description: MultiQC custom content YML file containing software versions + pattern: "software_versions_mqc.yml" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@drpatelh" - "@grst" diff --git a/modules/nf-core/custom/dumpsoftwareversions/templates/dumpsoftwareversions.py b/modules/nf-core/custom/dumpsoftwareversions/templates/dumpsoftwareversions.py index da033408..b83b32c4 100755 --- a/modules/nf-core/custom/dumpsoftwareversions/templates/dumpsoftwareversions.py +++ b/modules/nf-core/custom/dumpsoftwareversions/templates/dumpsoftwareversions.py @@ -3,11 +3,11 @@ """Provide functions to merge multiple versions.yml files.""" - -import yaml import platform from textwrap import dedent +import yaml + def _make_versions_html(versions): """Generate a tabular HTML output of all versions for MultiQC.""" diff --git a/modules/nf-core/custom/getchromsizes/custom-getchromsizes.diff b/modules/nf-core/custom/getchromsizes/custom-getchromsizes.diff index 216925e6..b05f08cc 100644 --- a/modules/nf-core/custom/getchromsizes/custom-getchromsizes.diff +++ b/modules/nf-core/custom/getchromsizes/custom-getchromsizes.diff @@ -1,15 +1,10 @@ Changes in module 'nf-core/custom/getchromsizes' +'modules/nf-core/custom/getchromsizes/meta.yml' is unchanged +Changes in 'custom/getchromsizes/main.nf': --- modules/nf-core/custom/getchromsizes/main.nf +++ modules/nf-core/custom/getchromsizes/main.nf -@@ -1,5 +1,5 @@ - process CUSTOM_GETCHROMSIZES { -- tag "$fasta" -+ tag "$meta.id" - label 'process_single' - - conda "bioconda::samtools=1.16.1" -@@ -8,22 +8,26 @@ - 'biocontainers/samtools:1.16.1--h6899075_1' }" +@@ -8,22 +8,25 @@ + 'biocontainers/samtools:1.21--h50ea8bc_0' }" input: - tuple val(meta), path(fasta) @@ -21,18 +16,16 @@ Changes in module 'nf-core/custom/getchromsizes' - tuple val(meta), path ("*.fai") , emit: fai - tuple val(meta), path ("*.gzi") , emit: gzi, optional: true - path "versions.yml" , emit: versions -+ tuple val(meta), path ("*.${suffix}") , emit: sizes -+ tuple val(meta), path ("*.fa") , emit: fasta -+ tuple val(meta), path ("*.fai") , emit: fai -+ tuple val(meta), path ("*.gzi") , emit: gzi, optional: true -+ path "versions.yml" , emit: versions ++ tuple val(meta), path ("*.${suffix}"), emit: sizes ++ tuple val(meta), path ("*.fai"), emit: fai ++ tuple val(meta), path ("*.gzi"), emit: gzi, optional: true ++ path "versions.yml", emit: versions when: task.ext.when == null || task.ext.when script: -- def args = task.ext.args ?: '' -+ def args = task.ext.args ?: '' + def args = task.ext.args ?: '' + def prefix = task.ext.prefix ?: "${meta.id}" """ - samtools faidx $fasta @@ -43,20 +36,9 @@ Changes in module 'nf-core/custom/getchromsizes' cat <<-END_VERSIONS > versions.yml "${task.process}": -@@ -32,9 +36,12 @@ - """ - - stub: -+ def prefix = task.ext.prefix ?: "${meta.id}" -+ def suffix = 'temp.genome' - """ -- touch ${fasta}.fai -- touch ${fasta}.sizes -+ ln -s ${fasta} ${prefix}.fa -+ touch ${prefix}.fa.fai -+ touch ${prefix}.${suffix} - - cat <<-END_VERSIONS > versions.yml - "${task.process}": +'modules/nf-core/custom/getchromsizes/environment.yml' is unchanged +'modules/nf-core/custom/getchromsizes/tests/main.nf.test' is unchanged +'modules/nf-core/custom/getchromsizes/tests/tags.yml' is unchanged +'modules/nf-core/custom/getchromsizes/tests/main.nf.test.snap' is unchanged ************************************************************ diff --git a/modules/nf-core/custom/getchromsizes/environment.yml b/modules/nf-core/custom/getchromsizes/environment.yml index 2a01695f..2bcd47ee 100644 --- a/modules/nf-core/custom/getchromsizes/environment.yml +++ b/modules/nf-core/custom/getchromsizes/environment.yml @@ -1,7 +1,7 @@ -name: custom_getchromsizes channels: - conda-forge - bioconda - - defaults + dependencies: - - bioconda::samtools=1.16.1 + - bioconda::htslib=1.21 + - bioconda::samtools=1.21 diff --git a/modules/nf-core/custom/getchromsizes/main.nf b/modules/nf-core/custom/getchromsizes/main.nf index 146f501e..263ddc7d 100644 --- a/modules/nf-core/custom/getchromsizes/main.nf +++ b/modules/nf-core/custom/getchromsizes/main.nf @@ -1,28 +1,27 @@ process CUSTOM_GETCHROMSIZES { - tag "$meta.id" + tag "$fasta" label 'process_single' conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/samtools:1.16.1--h6899075_1' : - 'biocontainers/samtools:1.16.1--h6899075_1' }" + 'https://depot.galaxyproject.org/singularity/samtools:1.21--h50ea8bc_0' : + 'biocontainers/samtools:1.21--h50ea8bc_0' }" input: tuple val(meta), path(fasta, stageAs: 'input/*') val suffix output: - tuple val(meta), path ("*.${suffix}") , emit: sizes - tuple val(meta), path ("*.fa") , emit: fasta - tuple val(meta), path ("*.fai") , emit: fai - tuple val(meta), path ("*.gzi") , emit: gzi, optional: true - path "versions.yml" , emit: versions + tuple val(meta), path ("*.${suffix}"), emit: sizes + tuple val(meta), path ("*.fai"), emit: fai + tuple val(meta), path ("*.gzi"), emit: gzi, optional: true + path "versions.yml", emit: versions when: task.ext.when == null || task.ext.when script: - def args = task.ext.args ?: '' + def args = task.ext.args ?: '' def prefix = task.ext.prefix ?: "${meta.id}" """ ln -s ${fasta} ${prefix}.fa @@ -36,12 +35,12 @@ process CUSTOM_GETCHROMSIZES { """ stub: - def prefix = task.ext.prefix ?: "${meta.id}" - def suffix = 'temp.genome' """ - ln -s ${fasta} ${prefix}.fa - touch ${prefix}.fa.fai - touch ${prefix}.${suffix} + touch ${fasta}.fai + touch ${fasta}.sizes + if [[ "${fasta.extension}" == "gz" ]]; then + touch ${fasta}.gzi + fi cat <<-END_VERSIONS > versions.yml "${task.process}": diff --git a/modules/nf-core/custom/getchromsizes/meta.yml b/modules/nf-core/custom/getchromsizes/meta.yml index 529be07e..2a0e8da0 100644 --- a/modules/nf-core/custom/getchromsizes/meta.yml +++ b/modules/nf-core/custom/getchromsizes/meta.yml @@ -12,38 +12,53 @@ tools: tool_dev_url: https://github.com/samtools/samtools doi: 10.1093/bioinformatics/btp352 licence: ["MIT"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - fasta: - type: file - description: FASTA file - pattern: "*.{fa,fasta,fna,fas}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - fasta: + type: file + description: FASTA file + pattern: "*.{fa,fasta,fna,fas}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - sizes: - type: file - description: File containing chromosome lengths - pattern: "*.{sizes}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.sizes": + type: file + description: File containing chromosome lengths + pattern: "*.{sizes}" - fai: - type: file - description: FASTA index file - pattern: "*.{fai}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.fai": + type: file + description: FASTA index file + pattern: "*.{fai}" - gzi: - type: file - description: Optional gzip index file for compressed inputs - pattern: "*.gzi" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.gzi": + type: file + description: Optional gzip index file for compressed inputs + pattern: "*.gzi" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@tamara-hodgetts" - "@chris-cheshire" diff --git a/modules/nf-core/custom/getchromsizes/tests/main.nf.test b/modules/nf-core/custom/getchromsizes/tests/main.nf.test index 9f6b5640..2f741a4b 100644 --- a/modules/nf-core/custom/getchromsizes/tests/main.nf.test +++ b/modules/nf-core/custom/getchromsizes/tests/main.nf.test @@ -12,9 +12,6 @@ nextflow_process { test("test_custom_getchromsizes") { when { - params { - outdir = "$outputDir" - } process { """ input[0] = Channel.of([ @@ -31,15 +28,11 @@ nextflow_process { { assert snapshot(process.out).match() } ) } - } test("test_custom_getchromsizes_bgzip") { when { - params { - outdir = "$outputDir" - } process { """ input[0] = Channel.of([ @@ -56,7 +49,51 @@ nextflow_process { { assert snapshot(process.out).match() } ) } + } + + test("test_custom_getchromsizes - stub") { + + options "-stub" + + when { + process { + """ + input[0] = Channel.of([ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ]) + """ + } + } + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } } + test("test_custom_getchromsizes_bgzip - stub") { + + options "-stub" + + when { + process { + """ + input[0] = Channel.of([ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta.gz', checkIfExists: true) + ]) + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } } diff --git a/modules/nf-core/custom/getchromsizes/tests/main.nf.test.snap b/modules/nf-core/custom/getchromsizes/tests/main.nf.test.snap index 2e560bd3..d2072e22 100644 --- a/modules/nf-core/custom/getchromsizes/tests/main.nf.test.snap +++ b/modules/nf-core/custom/getchromsizes/tests/main.nf.test.snap @@ -1,4 +1,69 @@ { + "test_custom_getchromsizes_bgzip - stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "genome.fasta.gz.sizes:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + [ + { + "id": "test" + }, + "genome.fasta.gz.fai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "2": [ + [ + { + "id": "test" + }, + "genome.fasta.gz.gzi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "3": [ + "versions.yml:md5,f68edc4d5999d8cb07147b9be115d60b" + ], + "fai": [ + [ + { + "id": "test" + }, + "genome.fasta.gz.fai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "gzi": [ + [ + { + "id": "test" + }, + "genome.fasta.gz.gzi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "sizes": [ + [ + { + "id": "test" + }, + "genome.fasta.gz.sizes:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,f68edc4d5999d8cb07147b9be115d60b" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-10-18T10:11:37.459094476" + }, "test_custom_getchromsizes": { "content": [ { @@ -22,7 +87,7 @@ ], "3": [ - "versions.yml:md5,45a83c5f3dddbc5dcab30035169f7ce8" + "versions.yml:md5,f68edc4d5999d8cb07147b9be115d60b" ], "fai": [ [ @@ -44,11 +109,15 @@ ] ], "versions": [ - "versions.yml:md5,45a83c5f3dddbc5dcab30035169f7ce8" + "versions.yml:md5,f68edc4d5999d8cb07147b9be115d60b" ] } ], - "timestamp": "2024-01-17T17:48:35.562918" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-10-18T10:11:01.526301351" }, "test_custom_getchromsizes_bgzip": { "content": [ @@ -78,7 +147,7 @@ ] ], "3": [ - "versions.yml:md5,45a83c5f3dddbc5dcab30035169f7ce8" + "versions.yml:md5,f68edc4d5999d8cb07147b9be115d60b" ], "fai": [ [ @@ -105,10 +174,69 @@ ] ], "versions": [ - "versions.yml:md5,45a83c5f3dddbc5dcab30035169f7ce8" + "versions.yml:md5,f68edc4d5999d8cb07147b9be115d60b" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-10-18T10:11:14.32456385" + }, + "test_custom_getchromsizes - stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "genome.fasta.sizes:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + [ + { + "id": "test" + }, + "genome.fasta.fai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,f68edc4d5999d8cb07147b9be115d60b" + ], + "fai": [ + [ + { + "id": "test" + }, + "genome.fasta.fai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "gzi": [ + + ], + "sizes": [ + [ + { + "id": "test" + }, + "genome.fasta.sizes:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,f68edc4d5999d8cb07147b9be115d60b" ] } ], - "timestamp": "2024-01-17T17:49:02.562311" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-10-18T10:11:26.505487624" } } \ No newline at end of file diff --git a/modules/nf-core/fastk/fastk/environment.yml b/modules/nf-core/fastk/fastk/environment.yml index 54c932fe..14207ff6 100644 --- a/modules/nf-core/fastk/fastk/environment.yml +++ b/modules/nf-core/fastk/fastk/environment.yml @@ -1,5 +1,5 @@ -name: fastk_fastk channels: - conda-forge - bioconda - - defaults +dependencies: + - bioconda::false_flag # False flag to pass nf-core/lint diff --git a/modules/nf-core/fastk/fastk/fastk-fastk.diff b/modules/nf-core/fastk/fastk/fastk-fastk.diff index 0a0cba9e..80906fac 100644 --- a/modules/nf-core/fastk/fastk/fastk-fastk.diff +++ b/modules/nf-core/fastk/fastk/fastk-fastk.diff @@ -6,9 +6,25 @@ Changes in module 'nf-core/fastk/fastk' // WARN: Version information not provided by tool on CLI. Please update version string below when bumping container versions. - container 'ghcr.io/nbisweden/fastk_genescopefk_merquryfk:1.2' -+ container 'quay.io/sanger-tol/fastk:1.0.1-c1' ++ container 'quay.io/sanger-tol/fastk:1.0.2-c1' input: tuple val(meta), path(reads) +@@ -29,7 +29,6 @@ + FastK \\ + $args \\ + -T$task.cpus \\ +- -M${task.memory.toGiga()} \\ + -N${prefix}_fk \\ + $reads + +@@ -62,7 +61,6 @@ + "FastK \\ + $args \\ + -T$task.cpus \\ +- -M${task.memory.toGiga()} \\ + -N${prefix}_fk \\ + $reads" + ************************************************************ diff --git a/modules/nf-core/fastk/fastk/main.nf b/modules/nf-core/fastk/fastk/main.nf index 94d5f592..256e80de 100644 --- a/modules/nf-core/fastk/fastk/main.nf +++ b/modules/nf-core/fastk/fastk/main.nf @@ -3,7 +3,7 @@ process FASTK_FASTK { label 'process_medium' // WARN: Version information not provided by tool on CLI. Please update version string below when bumping container versions. - container 'quay.io/sanger-tol/fastk:1.0.1-c1' + container 'quay.io/sanger-tol/fastk:1.0.2-c1' input: tuple val(meta), path(reads) @@ -29,10 +29,41 @@ process FASTK_FASTK { FastK \\ $args \\ -T$task.cpus \\ - -M${task.memory.toGiga()} \\ -N${prefix}_fk \\ $reads + find . -name '*.ktab*' \\ + | xargs chmod a+r + + cat <<-END_VERSIONS > versions.yml + "${task.process}": + fastk: $FASTK_VERSION + END_VERSIONS + """ + + stub: + // Exit if running this module with -profile conda / -profile mamba + if (workflow.profile.tokenize(',').intersect(['conda', 'mamba']).size() >= 1) { + error "FASTK_FASTK module does not support Conda. Please use Docker / Singularity / Podman instead." + } + def args = task.ext.args ?: '' + def prefix = task.ext.prefix ?: "${meta.id}" + def FASTK_VERSION = 'f18a4e6d2207539f7b84461daebc54530a9559b0' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. + + def touch_ktab = args.contains('-t') ? "touch ${prefix}_fk.ktab .${prefix}_fk.ktab.1" : '' + def touch_prof = args.contains('-p') ? "touch ${prefix}_fk.prof .${prefix}_fk.pidx.1" : '' + """ + touch ${prefix}_fk.hist + $touch_ktab + $touch_prof + + echo \\ + "FastK \\ + $args \\ + -T$task.cpus \\ + -N${prefix}_fk \\ + $reads" + cat <<-END_VERSIONS > versions.yml "${task.process}": fastk: $FASTK_VERSION diff --git a/modules/nf-core/fastk/fastk/meta.yml b/modules/nf-core/fastk/fastk/meta.yml index 7c7f4260..4abc1033 100644 --- a/modules/nf-core/fastk/fastk/meta.yml +++ b/modules/nf-core/fastk/fastk/meta.yml @@ -9,41 +9,57 @@ tools: description: "A fast K-mer counter for high-fidelity shotgun datasets" homepage: "https://github.com/thegenemyers/FASTK" tool_dev_url: "https://github.com/thegenemyers/FASTK" - licence: "https://github.com/thegenemyers/FASTK/blob/master/LICENSE" + licence: ["https://github.com/thegenemyers/FASTK/blob/master/LICENSE"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - reads: - type: file - description: | - List of input FastQ files of size 1 and 2 for single-end and paired-end data, - respectively. + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - reads: + type: file + description: | + List of input FastQ files of size 1 and 2 for single-end and paired-end data, + respectively. output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - hist: - type: file - description: Histogram of k-mers - pattern: "*.hist" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.hist": + type: file + description: Histogram of k-mers + pattern: "*.hist" - ktab: - type: file - description: A sorted table of all canonical k‑mers along with their counts. - pattern: "*.ktab" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - '*.ktab*", hidden: true': + type: file + description: A sorted table of all canonical k‑mers along with their counts. + pattern: "*.ktab" - prof: - type: file - description: A k‑mer count profile of each sequence in the input data set. - pattern: "*.prof" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - '*.{prof,pidx}*", hidden: true': + type: file + description: A k‑mer count profile of each sequence in the input data set. + pattern: "*.prof" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@mahesh-panchal" maintainers: - "@mahesh-panchal" + - "@gallvp" diff --git a/modules/nf-core/fastk/fastk/tests/main.nf.test b/modules/nf-core/fastk/fastk/tests/main.nf.test new file mode 100644 index 00000000..856c36a8 --- /dev/null +++ b/modules/nf-core/fastk/fastk/tests/main.nf.test @@ -0,0 +1,110 @@ +nextflow_process { + + name "Test Process FASTK_FASTK" + script "../main.nf" + config './nextflow.config' + process "FASTK_FASTK" + + tag "modules" + tag "modules_nfcore" + tag "fastk" + tag "fastk/fastk" + + test("test_fastk_fastk_single_end") { + when { + process { + """ + input[0] = [ + [ id:'test' , single_end: true ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true) + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("test_fastk_fastk_paired_end") { + + when { + process { + """ + input[0] = [ + [ id:'test' , single_end: false ], // meta map + [ + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_2.fastq.gz', checkIfExists: true) + ] + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("test_fastk_fastk_single_end_stub") { + + options '-stub' + + when { + process { + """ + input[0] = [ + [ id:'test' , single_end: true ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true) + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("test_fastk_fastk_paired_end_stub") { + + options '-stub' + + when { + process { + """ + input[0] = [ + [ id:'test' , single_end: false ], // meta map + [ + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_2.fastq.gz', checkIfExists: true) + ] + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + +} diff --git a/modules/nf-core/fastk/fastk/tests/main.nf.test.snap b/modules/nf-core/fastk/fastk/tests/main.nf.test.snap new file mode 100644 index 00000000..1e3fc4da --- /dev/null +++ b/modules/nf-core/fastk/fastk/tests/main.nf.test.snap @@ -0,0 +1,346 @@ +{ + "test_fastk_fastk_single_end_stub": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": true + }, + "test_fk.hist:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + [ + { + "id": "test", + "single_end": true + }, + [ + ".test_fk.ktab.1:md5,d41d8cd98f00b204e9800998ecf8427e", + "test_fk.ktab:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ] + ], + "2": [ + [ + { + "id": "test", + "single_end": true + }, + [ + ".test_fk.pidx.1:md5,d41d8cd98f00b204e9800998ecf8427e", + "test_fk.prof:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ] + ], + "3": [ + "versions.yml:md5,c216a1608924d1662d2086e1de1d5abd" + ], + "hist": [ + [ + { + "id": "test", + "single_end": true + }, + "test_fk.hist:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "ktab": [ + [ + { + "id": "test", + "single_end": true + }, + [ + ".test_fk.ktab.1:md5,d41d8cd98f00b204e9800998ecf8427e", + "test_fk.ktab:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ] + ], + "prof": [ + [ + { + "id": "test", + "single_end": true + }, + [ + ".test_fk.pidx.1:md5,d41d8cd98f00b204e9800998ecf8427e", + "test_fk.prof:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ] + ], + "versions": [ + "versions.yml:md5,c216a1608924d1662d2086e1de1d5abd" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-18T19:39:28.510263" + }, + "test_fastk_fastk_single_end": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": true + }, + "test_fk.hist:md5,c80e12f7321e62dba4b437d7bff36ec0" + ] + ], + "1": [ + [ + { + "id": "test", + "single_end": true + }, + [ + ".test_fk.ktab.1:md5,ceeacd0cb3aa69bf9b2a402830b40e26", + ".test_fk.ktab.2:md5,f2629fd15b285aed3dc2d5fe546edf3f", + "test_fk.ktab:md5,a605a58931a4b5029469e1c2575c8cee" + ] + ] + ], + "2": [ + [ + { + "id": "test", + "single_end": true + }, + [ + ".test_fk.pidx.1:md5,90bc384f61d2ecdb4586ab52ab04fddf", + ".test_fk.prof.1:md5,ebd48923a724cf79934f0b2ed42ba73d", + "test_fk.prof:md5,43d426c95d277b8148406624d513bd40" + ] + ] + ], + "3": [ + "versions.yml:md5,c216a1608924d1662d2086e1de1d5abd" + ], + "hist": [ + [ + { + "id": "test", + "single_end": true + }, + "test_fk.hist:md5,c80e12f7321e62dba4b437d7bff36ec0" + ] + ], + "ktab": [ + [ + { + "id": "test", + "single_end": true + }, + [ + ".test_fk.ktab.1:md5,ceeacd0cb3aa69bf9b2a402830b40e26", + ".test_fk.ktab.2:md5,f2629fd15b285aed3dc2d5fe546edf3f", + "test_fk.ktab:md5,a605a58931a4b5029469e1c2575c8cee" + ] + ] + ], + "prof": [ + [ + { + "id": "test", + "single_end": true + }, + [ + ".test_fk.pidx.1:md5,90bc384f61d2ecdb4586ab52ab04fddf", + ".test_fk.prof.1:md5,ebd48923a724cf79934f0b2ed42ba73d", + "test_fk.prof:md5,43d426c95d277b8148406624d513bd40" + ] + ] + ], + "versions": [ + "versions.yml:md5,c216a1608924d1662d2086e1de1d5abd" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-18T19:31:31.128388" + }, + "test_fastk_fastk_paired_end": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test_fk.hist:md5,4f75b550d87ed4f26a2b10a05ac7e98c" + ] + ], + "1": [ + [ + { + "id": "test", + "single_end": false + }, + [ + ".test_fk.ktab.1:md5,7f28fb44940fda799797e3069f5d7263", + ".test_fk.ktab.2:md5,c14a85c128926ace78372f09029977b1", + "test_fk.ktab:md5,fddd5be0c36ad1d2131b8d8774f7657a" + ] + ] + ], + "2": [ + [ + { + "id": "test", + "single_end": false + }, + [ + ".test_fk.pidx.1:md5,e7e760f714070a4afefb38ffff559684", + ".test_fk.pidx.2:md5,a549612bbdba2506eb3311237638c4b0", + ".test_fk.prof.1:md5,46a5fd9e297262b058f8c1fd062fcf56", + ".test_fk.prof.2:md5,80326a7406f41ccf2e51e341fc804132", + "test_fk.prof:md5,d3c7d8decd4ea6e298291b8be0e2de85" + ] + ] + ], + "3": [ + "versions.yml:md5,c216a1608924d1662d2086e1de1d5abd" + ], + "hist": [ + [ + { + "id": "test", + "single_end": false + }, + "test_fk.hist:md5,4f75b550d87ed4f26a2b10a05ac7e98c" + ] + ], + "ktab": [ + [ + { + "id": "test", + "single_end": false + }, + [ + ".test_fk.ktab.1:md5,7f28fb44940fda799797e3069f5d7263", + ".test_fk.ktab.2:md5,c14a85c128926ace78372f09029977b1", + "test_fk.ktab:md5,fddd5be0c36ad1d2131b8d8774f7657a" + ] + ] + ], + "prof": [ + [ + { + "id": "test", + "single_end": false + }, + [ + ".test_fk.pidx.1:md5,e7e760f714070a4afefb38ffff559684", + ".test_fk.pidx.2:md5,a549612bbdba2506eb3311237638c4b0", + ".test_fk.prof.1:md5,46a5fd9e297262b058f8c1fd062fcf56", + ".test_fk.prof.2:md5,80326a7406f41ccf2e51e341fc804132", + "test_fk.prof:md5,d3c7d8decd4ea6e298291b8be0e2de85" + ] + ] + ], + "versions": [ + "versions.yml:md5,c216a1608924d1662d2086e1de1d5abd" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-18T19:31:35.565502" + }, + "test_fastk_fastk_paired_end_stub": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test_fk.hist:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + [ + { + "id": "test", + "single_end": false + }, + [ + ".test_fk.ktab.1:md5,d41d8cd98f00b204e9800998ecf8427e", + "test_fk.ktab:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ] + ], + "2": [ + [ + { + "id": "test", + "single_end": false + }, + [ + ".test_fk.pidx.1:md5,d41d8cd98f00b204e9800998ecf8427e", + "test_fk.prof:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ] + ], + "3": [ + "versions.yml:md5,c216a1608924d1662d2086e1de1d5abd" + ], + "hist": [ + [ + { + "id": "test", + "single_end": false + }, + "test_fk.hist:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "ktab": [ + [ + { + "id": "test", + "single_end": false + }, + [ + ".test_fk.ktab.1:md5,d41d8cd98f00b204e9800998ecf8427e", + "test_fk.ktab:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ] + ], + "prof": [ + [ + { + "id": "test", + "single_end": false + }, + [ + ".test_fk.pidx.1:md5,d41d8cd98f00b204e9800998ecf8427e", + "test_fk.prof:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ] + ], + "versions": [ + "versions.yml:md5,c216a1608924d1662d2086e1de1d5abd" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-05-18T19:39:32.570957" + } +} \ No newline at end of file diff --git a/modules/nf-core/fastk/fastk/tests/nextflow.config b/modules/nf-core/fastk/fastk/tests/nextflow.config new file mode 100644 index 00000000..c89ce9d5 --- /dev/null +++ b/modules/nf-core/fastk/fastk/tests/nextflow.config @@ -0,0 +1,3 @@ +process { + ext.args = '-t -p' +} diff --git a/modules/nf-core/fastk/fastk/tests/tags.yml b/modules/nf-core/fastk/fastk/tests/tags.yml new file mode 100644 index 00000000..82f9df82 --- /dev/null +++ b/modules/nf-core/fastk/fastk/tests/tags.yml @@ -0,0 +1,2 @@ +fastk/fastk: + - "modules/nf-core/fastk/fastk/**" diff --git a/modules/nf-core/gnu/sort/environment.yml b/modules/nf-core/gnu/sort/environment.yml index 45d23356..babcfb55 100644 --- a/modules/nf-core/gnu/sort/environment.yml +++ b/modules/nf-core/gnu/sort/environment.yml @@ -1,7 +1,7 @@ -name: GNU_SORT +--- +# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::coreutils=8.25 + - conda-forge::coreutils=9.3 diff --git a/modules/nf-core/gnu/sort/main.nf b/modules/nf-core/gnu/sort/main.nf index 108a70a8..e1167666 100644 --- a/modules/nf-core/gnu/sort/main.nf +++ b/modules/nf-core/gnu/sort/main.nf @@ -1,11 +1,11 @@ process GNU_SORT { - tag "${meta.id}" + tag "$meta.id" label "process_low" conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/coreutils:8.25--1' : - 'biocontainers/coreutils:8.25--1' }" + 'https://depot.galaxyproject.org/singularity/coreutils:9.3': + 'biocontainers/coreutils:9.3' }" input: tuple val(meta), path(input) @@ -22,7 +22,7 @@ process GNU_SORT { def prefix = task.ext.prefix ?: "${meta.id}" suffix = task.ext.suffix ?: "${input.extension}" output_file = "${prefix}.${suffix}" - def VERSION = "9.1" // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. + def VERSION = "9.3" // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. if ("$input" == "$output_file") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!" """ sort ${args} ${input} > ${output_file} @@ -34,15 +34,14 @@ process GNU_SORT { """ stub: - def args = task.ext.args ?: '' def prefix = task.ext.prefix ?: "${meta.id}" suffix = task.ext.suffix ?: "${input.extension}" output_file = "${prefix}.${suffix}" - def VERSION = "9.1" + def VERSION = "9.3" if ("$input" == "$output_file") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!" """ - sort ${args} ${input} > ${output_file} + touch ${output_file} cat <<-END_VERSIONS > versions.yml "${task.process}": diff --git a/modules/nf-core/gnu/sort/meta.yml b/modules/nf-core/gnu/sort/meta.yml index 014bcd52..c555dbb5 100644 --- a/modules/nf-core/gnu/sort/meta.yml +++ b/modules/nf-core/gnu/sort/meta.yml @@ -1,4 +1,4 @@ -name: "GNU_SORT" +name: "gnu_sort" description: | Writes a sorted concatenation of file/s keywords: @@ -11,30 +11,29 @@ tools: homepage: "https://github.com/vgl-hub/gfastats" documentation: "https://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html" licence: ["GPL"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - input: - type: file - description: Draft assembly file - pattern: "*.{txt,bed,interval,genome,bins}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - input: + type: file + description: Draft assembly file + pattern: "*.{txt,bed,interval,genome,bins}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - sorted: - type: file - description: The sorted txt file generated by sort - pattern: "*.{txt,bed,interval,genome,bins}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@DLBPointon" maintainers: diff --git a/modules/nf-core/gnu/sort/tests/main.nf.test b/modules/nf-core/gnu/sort/tests/main.nf.test new file mode 100644 index 00000000..e4030187 --- /dev/null +++ b/modules/nf-core/gnu/sort/tests/main.nf.test @@ -0,0 +1,120 @@ +nextflow_process { + + name "Test Process GNU_SORT" + script "modules/nf-core/gnu/sort/main.nf" + process "GNU_SORT" + + tag "modules" + tag "modules_nfcore" + tag "gnu" + tag "gnu/sort" + + test("unsorted_genome_sort") { + config "./sort_simple_bed.config" + + when { + process { + """ + input[0] = [ + [id:'genome_test'], + file(params.test_data['generic']['unsorted_data']['unsorted_text']['genome_file'], + checkIfExists: true) + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() }, + { assert snapshot( + file(process.out.sorted[0][1]).name + ).match("genome_sort") + } + ) + } + + } + + test("unsorted_intervals_sort") { + config "./sort_simple_bed.config" + when { + process { + """ + input[0] = [ + [id:'test'], + file(params.test_data['generic']['unsorted_data']['unsorted_text']['intervals'], + checkIfExists: true) + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() }, + { assert snapshot( + file(process.out.sorted[0][1]).name + ).match("interval_sort") + } + ) + } + + } + + test("unsorted_csv_sort") { + config "./sort_complex.config" + + when { + process { + """ + input[0] = [ + [id:'test'], + file(params.test_data['generic']['unsorted_data']['unsorted_text']['numbers_csv'], + checkIfExists: true) + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() }, + { assert snapshot( + file(process.out.sorted[0][1]).name + ).match("csv_sort") + } + ) + } + + } + + test("unsorted_csv_sort_stub") { + config "./sort_complex.config" + options "-stub" + + when { + process { + """ + input[0] = [ + [id:'test'], + file(params.test_data['generic']['unsorted_data']['unsorted_text']['numbers_csv'], + checkIfExists: true) + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() }, + ) + } + + } + +} diff --git a/modules/nf-core/gnu/sort/tests/main.nf.test.snap b/modules/nf-core/gnu/sort/tests/main.nf.test.snap new file mode 100644 index 00000000..63891bc4 --- /dev/null +++ b/modules/nf-core/gnu/sort/tests/main.nf.test.snap @@ -0,0 +1,164 @@ +{ + "unsorted_csv_sort": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.csv.sorted:md5,0b52d1b4c4a0c6e972c6f94aafd75a1d" + ] + ], + "1": [ + "versions.yml:md5,dd412503ec9dd665203e083ea44326cb" + ], + "sorted": [ + [ + { + "id": "test" + }, + "test.csv.sorted:md5,0b52d1b4c4a0c6e972c6f94aafd75a1d" + ] + ], + "versions": [ + "versions.yml:md5,dd412503ec9dd665203e083ea44326cb" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-14T11:13:44.714632791" + }, + "interval_sort": { + "content": [ + "test.bed.sorted" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-14T11:13:37.962807086" + }, + "unsorted_csv_sort_stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.csv.sorted:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,dd412503ec9dd665203e083ea44326cb" + ], + "sorted": [ + [ + { + "id": "test" + }, + "test.csv.sorted:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,dd412503ec9dd665203e083ea44326cb" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-14T11:13:51.456258705" + }, + "csv_sort": { + "content": [ + "test.csv.sorted" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-14T11:13:44.725431761" + }, + "unsorted_genome_sort": { + "content": [ + { + "0": [ + [ + { + "id": "genome_test" + }, + "genome_test.bed.sorted:md5,fd97f7efafdbbfa71d9b560f10b4b048" + ] + ], + "1": [ + "versions.yml:md5,dd412503ec9dd665203e083ea44326cb" + ], + "sorted": [ + [ + { + "id": "genome_test" + }, + "genome_test.bed.sorted:md5,fd97f7efafdbbfa71d9b560f10b4b048" + ] + ], + "versions": [ + "versions.yml:md5,dd412503ec9dd665203e083ea44326cb" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-14T11:13:31.041778719" + }, + "genome_sort": { + "content": [ + "genome_test.bed.sorted" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-14T11:13:31.060201722" + }, + "unsorted_intervals_sort": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.bed.sorted:md5,abbce903ef263d38b2f71856387799ab" + ] + ], + "1": [ + "versions.yml:md5,dd412503ec9dd665203e083ea44326cb" + ], + "sorted": [ + [ + { + "id": "test" + }, + "test.bed.sorted:md5,abbce903ef263d38b2f71856387799ab" + ] + ], + "versions": [ + "versions.yml:md5,dd412503ec9dd665203e083ea44326cb" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-14T11:13:37.951397547" + } +} \ No newline at end of file diff --git a/modules/nf-core/gnu/sort/tests/sort_complex.config b/modules/nf-core/gnu/sort/tests/sort_complex.config new file mode 100644 index 00000000..103eaaf6 --- /dev/null +++ b/modules/nf-core/gnu/sort/tests/sort_complex.config @@ -0,0 +1,6 @@ +process { + withName: GNU_SORT { + ext.args = { "-t ';' -g -k 1,1 -k 2,2" } + ext.suffix = { "csv.sorted" } + } +} \ No newline at end of file diff --git a/modules/nf-core/gnu/sort/tests/sort_simple_bed.config b/modules/nf-core/gnu/sort/tests/sort_simple_bed.config new file mode 100644 index 00000000..d7d52e0f --- /dev/null +++ b/modules/nf-core/gnu/sort/tests/sort_simple_bed.config @@ -0,0 +1,6 @@ +process { + withName: GNU_SORT { + ext.args = { "-k1,1 -k2,2n" } + ext.suffix = { "bed.sorted" } + } +} \ No newline at end of file diff --git a/modules/nf-core/gnu/sort/tests/sort_simple_genome.config b/modules/nf-core/gnu/sort/tests/sort_simple_genome.config new file mode 100644 index 00000000..4dcec385 --- /dev/null +++ b/modules/nf-core/gnu/sort/tests/sort_simple_genome.config @@ -0,0 +1,6 @@ +process { + withName: GNU_SORT { + ext.args = { "-k1,1 -k2,2n" } + ext.suffix = { "genome.sorted" } + } +} \ No newline at end of file diff --git a/modules/nf-core/gnu/sort/tests/tags.yml b/modules/nf-core/gnu/sort/tests/tags.yml new file mode 100644 index 00000000..ac40e376 --- /dev/null +++ b/modules/nf-core/gnu/sort/tests/tags.yml @@ -0,0 +1,2 @@ +gnu/sort: + - "modules/nf-core/gnu/sort/**" diff --git a/modules/nf-core/merquryfk/merquryfk/environment.yml b/modules/nf-core/merquryfk/merquryfk/environment.yml deleted file mode 100644 index 44a5ee9b..00000000 --- a/modules/nf-core/merquryfk/merquryfk/environment.yml +++ /dev/null @@ -1,5 +0,0 @@ -name: merquryfk_merquryfk -channels: - - conda-forge - - bioconda - - defaults diff --git a/modules/nf-core/merquryfk/merquryfk/main.nf b/modules/nf-core/merquryfk/merquryfk/main.nf index 378a9911..0cb1c63b 100644 --- a/modules/nf-core/merquryfk/merquryfk/main.nf +++ b/modules/nf-core/merquryfk/merquryfk/main.nf @@ -3,29 +3,31 @@ process MERQURYFK_MERQURYFK { label 'process_medium' // WARN: Version information not provided by tool on CLI. Please update version string below when bumping container versions. - container 'quay.io/sanger-tol/fastk:1.0.1-c1' + container 'quay.io/sanger-tol/fastk:1.0.2-c1' input: - tuple val(meta), path(fastk_hist), path(fastk_ktab), path(assembly), path(haplotigs) + tuple val(meta), path(fastk_hist),path(fastk_ktab),path(assembly),path(haplotigs) + path matktab //optional + path patktab //optional output: - tuple val(meta), path("${prefix}.completeness.stats") , emit: stats - tuple val(meta), path("${prefix}.*_only.bed") , emit: bed - tuple val(meta), path("${prefix}.*.qv") , emit: assembly_qv - tuple val(meta), path("${prefix}.*.spectra-cn.fl.png"), emit: spectra_cn_fl_png, optional: true - tuple val(meta), path("${prefix}.*.spectra-cn.fl.pdf"), emit: spectra_cn_fl_pdf, optional: true - tuple val(meta), path("${prefix}.*.spectra-cn.ln.png"), emit: spectra_cn_ln_png, optional: true - tuple val(meta), path("${prefix}.*.spectra-cn.ln.pdf"), emit: spectra_cn_ln_pdf, optional: true - tuple val(meta), path("${prefix}.*.spectra-cn.st.png"), emit: spectra_cn_st_png, optional: true - tuple val(meta), path("${prefix}.*.spectra-cn.st.pdf"), emit: spectra_cn_st_pdf, optional: true - tuple val(meta), path("${prefix}.qv") , emit: qv - tuple val(meta), path("${prefix}.spectra-asm.fl.png") , emit: spectra_asm_fl_png, optional: true - tuple val(meta), path("${prefix}.spectra-asm.fl.pdf") , emit: spectra_asm_fl_pdf, optional: true - tuple val(meta), path("${prefix}.spectra-asm.ln.png") , emit: spectra_asm_ln_png, optional: true - tuple val(meta), path("${prefix}.spectra-asm.ln.pdf") , emit: spectra_asm_ln_pdf, optional: true - tuple val(meta), path("${prefix}.spectra-asm.st.png") , emit: spectra_asm_st_png, optional: true - tuple val(meta), path("${prefix}.spectra-asm.st.pdf") , emit: spectra_asm_st_pdf, optional: true - path "versions.yml" , emit: versions + tuple val(meta), path("${prefix}.completeness.stats") , emit: stats + tuple val(meta), path("${prefix}.*_only.bed") , emit: bed + tuple val(meta), path("${prefix}.*.qv") , emit: assembly_qv + tuple val(meta), path("${prefix}.*.spectra-cn.fl.{png,pdf}") , emit: spectra_cn_fl, optional: true + tuple val(meta), path("${prefix}.*.spectra-cn.ln.{png,pdf}") , emit: spectra_cn_ln, optional: true + tuple val(meta), path("${prefix}.*.spectra-cn.st.{png,pdf}") , emit: spectra_cn_st, optional: true + tuple val(meta), path("${prefix}.qv") , emit: qv + tuple val(meta), path("${prefix}.spectra-asm.fl.{png,pdf}") , emit: spectra_asm_fl, optional: true + tuple val(meta), path("${prefix}.spectra-asm.ln.{png,pdf}") , emit: spectra_asm_ln, optional: true + tuple val(meta), path("${prefix}.spectra-asm.st.{png,pdf}") , emit: spectra_asm_st, optional: true + tuple val(meta), path("${prefix}.phased_block.bed") , emit: phased_block_bed, optional: true + tuple val(meta), path("${prefix}.phased_block.stats") , emit: phased_block_stats, optional: true + tuple val(meta), path("${prefix}.continuity.N.{pdf,png}") , emit: continuity_N, optional: true + tuple val(meta), path("${prefix}.block.N.{pdf,png}") , emit: block_N, optional: true + tuple val(meta), path("${prefix}.block.blob.{pdf,png}") , emit: block_blob, optional: true + tuple val(meta), path("${prefix}.hapmers.blob.{pdf,png}") , emit: hapmers_blob, optional: true + path "versions.yml" , emit: versions when: task.ext.when == null || task.ext.when @@ -37,6 +39,8 @@ process MERQURYFK_MERQURYFK { } def args = task.ext.args ?: '' prefix = task.ext.prefix ?: "${meta.id}" + def mat_ktab = matktab ? "${matktab.find{ it.toString().endsWith(".ktab") }}" : '' + def pat_ktab = patktab ? "${patktab.find{ it.toString().endsWith(".ktab") }}" : '' def FASTK_VERSION = 'f18a4e6d2207539f7b84461daebc54530a9559b0' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. def MERQURY_VERSION = '8ae344092df5dcaf83cfb7f90f662597a9b1fc61' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. """ @@ -44,6 +48,8 @@ process MERQURYFK_MERQURYFK { $args \\ -T$task.cpus \\ ${fastk_ktab.find{ it.toString().endsWith(".ktab") }} \\ + ${mat_ktab} \\ + ${pat_ktab} \\ $assembly \\ $haplotigs \\ $prefix @@ -55,4 +61,20 @@ process MERQURYFK_MERQURYFK { r: \$( R --version | sed '1!d; s/.*version //; s/ .*//' ) END_VERSIONS """ + stub: + prefix = task.ext.prefix ?: "${meta.id}" + def FASTK_VERSION = 'f18a4e6d2207539f7b84461daebc54530a9559b0' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. + def MERQURY_VERSION = '8ae344092df5dcaf83cfb7f90f662597a9b1fc61' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. + """ + touch ${prefix}.completeness.stats + touch ${prefix}.qv + touch ${prefix}._.qv + touch ${prefix}._only.bed + cat <<-END_VERSIONS > versions.yml + "${task.process}": + fastk: $FASTK_VERSION + merquryfk: $MERQURY_VERSION + r: \$( R --version | sed '1!d; s/.*version //; s/ .*//' ) + END_VERSIONS + """ } diff --git a/modules/nf-core/merquryfk/merquryfk/merquryfk-merquryfk.diff b/modules/nf-core/merquryfk/merquryfk/merquryfk-merquryfk.diff index 4ae0d7be..dfa35f41 100644 --- a/modules/nf-core/merquryfk/merquryfk/merquryfk-merquryfk.diff +++ b/modules/nf-core/merquryfk/merquryfk/merquryfk-merquryfk.diff @@ -1,14 +1,29 @@ Changes in module 'nf-core/merquryfk/merquryfk' +'modules/nf-core/merquryfk/merquryfk/meta.yml' is unchanged +Changes in 'merquryfk/merquryfk/main.nf': --- modules/nf-core/merquryfk/merquryfk/main.nf +++ modules/nf-core/merquryfk/merquryfk/main.nf -@@ -3,7 +3,7 @@ +@@ -3,12 +3,12 @@ label 'process_medium' // WARN: Version information not provided by tool on CLI. Please update version string below when bumping container versions. - container 'ghcr.io/nbisweden/fastk_genescopefk_merquryfk:1.2' -+ container 'quay.io/sanger-tol/fastk:1.0.1-c1' ++ container 'quay.io/sanger-tol/fastk:1.0.2-c1' input: - tuple val(meta), path(fastk_hist), path(fastk_ktab), path(assembly), path(haplotigs) + tuple val(meta), path(fastk_hist),path(fastk_ktab),path(assembly),path(haplotigs) + path matktab //optional +- path patktab //optional ++ path patktab + + output: + tuple val(meta), path("${prefix}.completeness.stats") , emit: stats +'modules/nf-core/merquryfk/merquryfk/tests/nextflow.trio.config' is unchanged +'modules/nf-core/merquryfk/merquryfk/tests/nextflow.png.config' is unchanged +'modules/nf-core/merquryfk/merquryfk/tests/main.nf.test' is unchanged +'modules/nf-core/merquryfk/merquryfk/tests/tags.yml' is unchanged +'modules/nf-core/merquryfk/merquryfk/tests/nextflow.config' is unchanged +'modules/nf-core/merquryfk/merquryfk/tests/nextflow.pdf.config' is unchanged +'modules/nf-core/merquryfk/merquryfk/tests/main.nf.test.snap' is unchanged ************************************************************ diff --git a/modules/nf-core/merquryfk/merquryfk/meta.yml b/modules/nf-core/merquryfk/merquryfk/meta.yml index 7d4af796..82bfbec4 100644 --- a/modules/nf-core/merquryfk/merquryfk/meta.yml +++ b/modules/nf-core/merquryfk/merquryfk/meta.yml @@ -9,104 +9,209 @@ tools: description: "FastK based version of Merqury" homepage: "https://github.com/thegenemyers/MERQURY.FK" tool_dev_url: "https://github.com/thegenemyers/MERQURY.FK" - licence: "https://github.com/thegenemyers/MERQURY.FK/blob/main/LICENSE" + licence: ["https://github.com/thegenemyers/MERQURY.FK/blob/main/LICENSE"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - fastk_hist: - type: file - description: A histogram files from the program FastK - pattern: "*.hist" - - fastk_ktab: - type: file - description: Histogram ktab files from the program FastK (option -t) - pattern: "*.ktab*" - - assembly: - type: file - description: Genome (primary) assembly files (fasta format) - pattern: ".fasta" - - haplotigs: - type: file - description: Assembly haplotigs (fasta format) - pattern: ".fasta" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - fastk_hist: + type: file + description: A histogram files from the program FastK + pattern: "*.hist" + - fastk_ktab: + type: file + description: Histogram ktab files from the program FastK (option -t) + pattern: "*.ktab*" + - assembly: + type: file + description: Genome (primary) assembly files (fasta format) + pattern: ".fasta" + - haplotigs: + type: file + description: Assembly haplotigs (fasta format) + pattern: ".fasta" + - - matktab: + type: file + description: trio maternal histogram ktab files from the program FastK (option + -t) + pattern: "*.ktab*" + - - patktab: + type: file + description: trio paternal histogram ktab files from the program FastK (option + -t) + pattern: "*.ktab*" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - stats: - type: file - description: Assembly statistics file - pattern: "*.completeness.stats" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.completeness.stats: + type: file + description: Assembly statistics file + pattern: "*.completeness.stats" - bed: - type: file - description: Assembly only kmer positions not supported by reads in bed format - pattern: "*_only.bed" - - spectra_cn_fl_png: - type: file - description: "Unstacked copy number spectra filled plot in PNG format" - pattern: "*.spectra-cn.fl.png" - - spectra_cn_ln_png: - type: file - description: "Unstacked copy number spectra line plot in PNG format" - pattern: "*.spectra-cn.ln.png" - - spectra_cn_st_png: - type: file - description: "Stacked copy number spectra line plot in PNG format" - pattern: "*.spectra-cn.st.png" - - spectra_asm_fl_png: - type: file - description: "Unstacked assembly spectra filled plot in PNG format" - pattern: "*.spectra-asm.fl.png" - - spectra_asm_ln_png: - type: file - description: "Unstacked assembly spectra line plot in PNG format" - pattern: "*.spectra-asm.ln.png" - - spectra_asm_st_png: - type: file - description: "Stacked assembly spectra line plot in PNG format" - pattern: "*.spectra-asm.st.png" - - spectra_cn_fl_pdf: - type: file - description: "Unstacked copy number spectra filled plot in PDF format" - pattern: "*.spectra-cn.fl.pdf" - - spectra_cn_ln_pdf: - type: file - description: "Unstacked copy number spectra line plot in PDF format" - pattern: "*.spectra-cn.ln.pdf" - - spectra_cn_st_pdf: - type: file - description: "Stacked copy number spectra line plot in PDF format" - pattern: "*.spectra-cn.st.pdf" - - spectra_asm_fl_pdf: - type: file - description: "Unstacked assembly spectra filled plot in PDF format" - pattern: "*.spectra-asm.fl.pdf" - - spectra_asm_ln_pdf: - type: file - description: "Unstacked assembly spectra line plot in PDF format" - pattern: "*.spectra-asm.ln.pdf" - - spectra_asm_st_pdf: - type: file - description: "Stacked assembly spectra line plot in PDF format" - pattern: "*.spectra-asm.st.pdf" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.*_only.bed: + type: file + description: Assembly only kmer positions not supported by reads in bed format + pattern: "*_only.bed" - assembly_qv: - type: file - description: "error and qv table for each scaffold of the assembly" - pattern: "*.qv" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.*.qv: + type: file + description: "error and qv table for each scaffold of the assembly" + pattern: "*.qv" + - spectra_cn_fl: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.*.spectra-cn.fl.{png,pdf}: + type: file + description: "Unstacked copy number spectra filled plot in PNG or PDF format" + pattern: "*.spectra-cn.fl.{png,pdf}" + - spectra_cn_ln: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.*.spectra-cn.ln.{png,pdf}: + type: file + description: "Unstacked copy number spectra line plot in PNG or PDF format" + pattern: "*.spectra-cn.ln.{png,pdf}" + - spectra_cn_st: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.*.spectra-cn.st.{png,pdf}: + type: file + description: "Stacked copy number spectra line plot in PNG or PDF format" + pattern: "*.spectra-cn.st.{png,pdf}" - qv: - type: file - description: "error and qv of each assembly as a whole" - pattern: "*.qv" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.qv: + type: file + description: "error and qv of each assembly as a whole" + pattern: "*.qv" + - spectra_asm_fl: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.spectra-asm.fl.{png,pdf}: + type: file + description: "Unstacked assembly spectra filled plot in PNG or PDF format" + pattern: "*.spectra-asm.fl.{png,pdf}" + - spectra_asm_ln: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.spectra-asm.ln.{png,pdf}: + type: file + description: "Unstacked assembly spectra line plot in PNG or PDF format" + pattern: "*.spectra-asm.ln.{png,pdf}" + - spectra_asm_st: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.spectra-asm.st.{png,pdf}: + type: file + description: "Stacked assembly spectra line plot in PNG or PDF format" + pattern: "*.spectra-asm.st.{png,pdf}" + - phased_block_bed: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.phased_block.bed: + type: file + description: Assembly kmer positions seperated by block in bed format + pattern: "*.phased.block.bed" + - phased_block_stats: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.phased_block.stats: + type: file + description: phased assembly statistics file + pattern: "*.phased.block.stats" + - continuity_N: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.continuity.N.{pdf,png}: + type: file + description: "Stacked assembly N continuity plot in PNG or PDF format" + pattern: "*.continuity.N.{png,pdf}" + - block_N: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.block.N.{pdf,png}: + type: file + description: "Stacked assembly N continuity by block plot in PNG or PDF format" + pattern: "*.block.N.{png,pdf}" + - block_blob: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.block.blob.{pdf,png}: + type: file + description: "Stacked assembly block plot in PNG or PDF format" + pattern: "*.block.blob.{png,pdf}" + - hapmers_blob: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.hapmers.blob.{pdf,png}: + type: file + description: "Stacked assembly hapmers block plot in PNG or PDF format" + pattern: "*.hapmers.blob.{png,pdf}" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@mahesh-panchal" + - "@yumisims" maintainers: - "@mahesh-panchal" + - "@yumisims" diff --git a/modules/nf-core/merquryfk/merquryfk/tests/main.nf.test b/modules/nf-core/merquryfk/merquryfk/tests/main.nf.test new file mode 100644 index 00000000..c46843c6 --- /dev/null +++ b/modules/nf-core/merquryfk/merquryfk/tests/main.nf.test @@ -0,0 +1,170 @@ +nextflow_process { + + name "Test Process MERQURYFK" + script "../main.nf" + process "MERQURYFK_MERQURYFK" + + tag "modules" + tag "modules_nfcore" + tag "merquryfk" + tag "merquryfk/merquryfk" + tag "fastk" + tag "fastk/fastk" + + setup { + run("FASTK_FASTK") { + script "../../../fastk/fastk" + process { + """ + input[0] = [ + [ id:'test', single_end:false ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/fastq/test_1.fastq.gz', checkIfExists: true) + ] + """ + } + } + run("FASTK_FASTK", alias: "FASTK_MAT") { + script "../../../fastk/fastk" + process { + """ + input[0] = [ + [ id:'test', single_end:false ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/fastq/test_1.fastq.gz', checkIfExists: true), + ] + """ + } + } + + run("FASTK_FASTK", alias: "FASTK_PAT") { + script "../../../fastk/fastk" + process { + """ + input[0] = [ + [ id:'test', single_end:false ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/fastq/test_2.fastq.gz', checkIfExists: true), + ] + """ + } + } + } + + test("homo_sapiens - Illumina - png") { + config "./nextflow.png.config" + when { + process { + """ + assembly = [ + [ id:'test', single_end:true ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ] + + haplotigs = [ + [ id:'test', single_end:true ], [] + ] + input[0] = FASTK_FASTK.out.hist.join(FASTK_FASTK.out.ktab).join(Channel.from(assembly)).join(Channel.from(haplotigs)) + input[1] = [] + input[2] = [] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("homo_sapiens - Illumina - pdf") { + config "./nextflow.pdf.config" + when { + process { + """ + assembly = [ + [ id:'test', single_end:true ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ] + + haplotigs = [ + [ id:'test', single_end:true ], [] + ] + input[0] = FASTK_FASTK.out.hist.join(FASTK_FASTK.out.ktab).join(Channel.from(assembly)).join(Channel.from(haplotigs)) + input[1] = [] + input[2] = [] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("homo_sapiens - Illumina - trio") { + config "./nextflow.trio.config" + when { + process { + """ + assembly = [ + [ id:'test', single_end:true ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ] + + haplotigs = [ + [ id:'test', single_end:true ], [] + ] + input[0] = FASTK_FASTK.out.hist.join(FASTK_FASTK.out.ktab).join(Channel.from(assembly)).join(Channel.from(haplotigs)) + input[1] = FASTK_MAT.out.ktab + input[2] = FASTK_PAT.out.ktab + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("homo_sapiens - Illumina - stub") { + options "-stub" + config "./nextflow.pdf.config" + when { + process { + """ + assembly = [ + [ id:'test', single_end:true ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ] + + haplotigs = [ + [ id:'test', single_end:true ], [] + ] + input[0] = FASTK_FASTK.out.hist.join(FASTK_FASTK.out.ktab).join(Channel.from(assembly)).join(Channel.from(haplotigs)) + input[1] = [] + input[2] = [] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + + +} \ No newline at end of file diff --git a/modules/nf-core/merquryfk/merquryfk/tests/main.nf.test.snap b/modules/nf-core/merquryfk/merquryfk/tests/main.nf.test.snap new file mode 100644 index 00000000..f7ce47f0 --- /dev/null +++ b/modules/nf-core/merquryfk/merquryfk/tests/main.nf.test.snap @@ -0,0 +1,454 @@ +{ + "homo_sapiens - Illumina - stub": { + "content": [ + { + "0": [ + + ], + "1": [ + + ], + "10": [ + + ], + "11": [ + + ], + "12": [ + + ], + "13": [ + + ], + "14": [ + + ], + "15": [ + + ], + "16": [ + + ], + "2": [ + + ], + "3": [ + + ], + "4": [ + + ], + "5": [ + + ], + "6": [ + + ], + "7": [ + + ], + "8": [ + + ], + "9": [ + + ], + "assembly_qv": [ + + ], + "bed": [ + + ], + "block_N": [ + + ], + "block_blob": [ + + ], + "continuity_N": [ + + ], + "hapmers_blob": [ + + ], + "phased_block_bed": [ + + ], + "phased_block_stats": [ + + ], + "qv": [ + + ], + "spectra_asm_fl": [ + + ], + "spectra_asm_ln": [ + + ], + "spectra_asm_st": [ + + ], + "spectra_cn_fl": [ + + ], + "spectra_cn_ln": [ + + ], + "spectra_cn_st": [ + + ], + "stats": [ + + ], + "versions": [ + + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-08-15T15:22:52.240373868" + }, + "homo_sapiens - Illumina - pdf": { + "content": [ + { + "0": [ + + ], + "1": [ + + ], + "10": [ + + ], + "11": [ + + ], + "12": [ + + ], + "13": [ + + ], + "14": [ + + ], + "15": [ + + ], + "16": [ + + ], + "2": [ + + ], + "3": [ + + ], + "4": [ + + ], + "5": [ + + ], + "6": [ + + ], + "7": [ + + ], + "8": [ + + ], + "9": [ + + ], + "assembly_qv": [ + + ], + "bed": [ + + ], + "block_N": [ + + ], + "block_blob": [ + + ], + "continuity_N": [ + + ], + "hapmers_blob": [ + + ], + "phased_block_bed": [ + + ], + "phased_block_stats": [ + + ], + "qv": [ + + ], + "spectra_asm_fl": [ + + ], + "spectra_asm_ln": [ + + ], + "spectra_asm_st": [ + + ], + "spectra_cn_fl": [ + + ], + "spectra_cn_ln": [ + + ], + "spectra_cn_st": [ + + ], + "stats": [ + + ], + "versions": [ + + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-08-15T15:22:19.530675341" + }, + "homo_sapiens - Illumina - png": { + "content": [ + { + "0": [ + + ], + "1": [ + + ], + "10": [ + + ], + "11": [ + + ], + "12": [ + + ], + "13": [ + + ], + "14": [ + + ], + "15": [ + + ], + "16": [ + + ], + "2": [ + + ], + "3": [ + + ], + "4": [ + + ], + "5": [ + + ], + "6": [ + + ], + "7": [ + + ], + "8": [ + + ], + "9": [ + + ], + "assembly_qv": [ + + ], + "bed": [ + + ], + "block_N": [ + + ], + "block_blob": [ + + ], + "continuity_N": [ + + ], + "hapmers_blob": [ + + ], + "phased_block_bed": [ + + ], + "phased_block_stats": [ + + ], + "qv": [ + + ], + "spectra_asm_fl": [ + + ], + "spectra_asm_ln": [ + + ], + "spectra_asm_st": [ + + ], + "spectra_cn_fl": [ + + ], + "spectra_cn_ln": [ + + ], + "spectra_cn_st": [ + + ], + "stats": [ + + ], + "versions": [ + + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-08-15T15:21:57.682723412" + }, + "homo_sapiens - Illumina - trio": { + "content": [ + { + "0": [ + + ], + "1": [ + + ], + "10": [ + + ], + "11": [ + + ], + "12": [ + + ], + "13": [ + + ], + "14": [ + + ], + "15": [ + + ], + "16": [ + + ], + "2": [ + + ], + "3": [ + + ], + "4": [ + + ], + "5": [ + + ], + "6": [ + + ], + "7": [ + + ], + "8": [ + + ], + "9": [ + + ], + "assembly_qv": [ + + ], + "bed": [ + + ], + "block_N": [ + + ], + "block_blob": [ + + ], + "continuity_N": [ + + ], + "hapmers_blob": [ + + ], + "phased_block_bed": [ + + ], + "phased_block_stats": [ + + ], + "qv": [ + + ], + "spectra_asm_fl": [ + + ], + "spectra_asm_ln": [ + + ], + "spectra_asm_st": [ + + ], + "spectra_cn_fl": [ + + ], + "spectra_cn_ln": [ + + ], + "spectra_cn_st": [ + + ], + "stats": [ + + ], + "versions": [ + + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-08-15T15:22:40.060937299" + } +} \ No newline at end of file diff --git a/modules/nf-core/merquryfk/merquryfk/tests/nextflow.config b/modules/nf-core/merquryfk/merquryfk/tests/nextflow.config new file mode 100644 index 00000000..ba1eebc9 --- /dev/null +++ b/modules/nf-core/merquryfk/merquryfk/tests/nextflow.config @@ -0,0 +1,8 @@ +process { + publishDir = { "${params.outdir}/${task.process.tokenize(':')[-1].tokenize('_')[0].toLowerCase()}" } + + withName: 'FASTK_.*' { + ext.args = '-t' + publishDir = [ enabled: false ] + } +} diff --git a/modules/nf-core/merquryfk/merquryfk/tests/nextflow.pdf.config b/modules/nf-core/merquryfk/merquryfk/tests/nextflow.pdf.config new file mode 100644 index 00000000..52beeaa3 --- /dev/null +++ b/modules/nf-core/merquryfk/merquryfk/tests/nextflow.pdf.config @@ -0,0 +1,9 @@ +process { + withName: 'FASTK_.*' { + ext.args = '-t' + publishDir = [ enabled: false ] + } + withName: 'MERQURYFK_MERQURYFK' { + ext.args = '-lfs -pdf' + } +} diff --git a/modules/nf-core/merquryfk/merquryfk/tests/nextflow.png.config b/modules/nf-core/merquryfk/merquryfk/tests/nextflow.png.config new file mode 100644 index 00000000..47c3d63e --- /dev/null +++ b/modules/nf-core/merquryfk/merquryfk/tests/nextflow.png.config @@ -0,0 +1,9 @@ +process { + withName: 'FASTK_.*' { + ext.args = '-t' + publishDir = [ enabled: false ] + } + withName: 'MERQURYFK_MERQURYFK' { + ext.args = '-lfs' + } +} diff --git a/modules/nf-core/merquryfk/merquryfk/tests/nextflow.trio.config b/modules/nf-core/merquryfk/merquryfk/tests/nextflow.trio.config new file mode 100644 index 00000000..47c3d63e --- /dev/null +++ b/modules/nf-core/merquryfk/merquryfk/tests/nextflow.trio.config @@ -0,0 +1,9 @@ +process { + withName: 'FASTK_.*' { + ext.args = '-t' + publishDir = [ enabled: false ] + } + withName: 'MERQURYFK_MERQURYFK' { + ext.args = '-lfs' + } +} diff --git a/modules/nf-core/merquryfk/merquryfk/tests/tags.yml b/modules/nf-core/merquryfk/merquryfk/tests/tags.yml new file mode 100644 index 00000000..7dcac99b --- /dev/null +++ b/modules/nf-core/merquryfk/merquryfk/tests/tags.yml @@ -0,0 +1,2 @@ +merquryfk/merquryfk: + - "modules/nf-core/merquryfk/merquryfk/**" diff --git a/modules/nf-core/minimap2/align/environment.yml b/modules/nf-core/minimap2/align/environment.yml new file mode 100644 index 00000000..dc6476b7 --- /dev/null +++ b/modules/nf-core/minimap2/align/environment.yml @@ -0,0 +1,8 @@ +channels: + - conda-forge + - bioconda + +dependencies: + - bioconda::htslib=1.20 + - bioconda::minimap2=2.28 + - bioconda::samtools=1.20 diff --git a/modules/nf-core/minimap2/align/main.nf b/modules/nf-core/minimap2/align/main.nf index d7057674..4faab880 100644 --- a/modules/nf-core/minimap2/align/main.nf +++ b/modules/nf-core/minimap2/align/main.nf @@ -1,43 +1,55 @@ process MINIMAP2_ALIGN { tag "$meta.id" - label 'process_medium' + label 'process_high' // Note: the versions here need to match the versions used in the mulled container below and minimap2/index - conda "bioconda::minimap2=2.24 bioconda::samtools=1.14" + conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/mulled-v2-66534bcbb7031a148b13e2ad42583020b9cd25c4:1679e915ddb9d6b4abda91880c4b48857d471bd8-0' : - 'biocontainers/mulled-v2-66534bcbb7031a148b13e2ad42583020b9cd25c4:1679e915ddb9d6b4abda91880c4b48857d471bd8-0' }" + 'https://depot.galaxyproject.org/singularity/mulled-v2-66534bcbb7031a148b13e2ad42583020b9cd25c4:3161f532a5ea6f1dec9be5667c9efc2afdac6104-0' : + 'biocontainers/mulled-v2-66534bcbb7031a148b13e2ad42583020b9cd25c4:3161f532a5ea6f1dec9be5667c9efc2afdac6104-0' }" input: tuple val(meta), path(reads) - path reference + tuple val(meta2), path(reference) val bam_format + val bam_index_extension val cigar_paf_format val cigar_bam - val bed_format + val bed_bool + output: - tuple val(meta), path("*.paf"), optional: true, emit: paf - tuple val(meta), path("*.bam"), optional: true, emit: bam - tuple val(meta), path("*.bed"), optional: true, emit: bed - path "versions.yml" , emit: versions + tuple val(meta), path("*.paf") , optional: true, emit: paf + tuple val(meta), path("*.bam") , optional: true, emit: bam + tuple val(meta), path("*.bam.${bam_index_extension}"), optional: true, emit: index + tuple val(meta), path("*.bed") , optional: true, emit: bed + path "versions.yml" , emit: versions when: task.ext.when == null || task.ext.when script: - def args = task.ext.args ?: '' + def args = task.ext.args ?: '' + def args2 = task.ext.args2 ?: '' + def args3 = task.ext.args3 ?: '' + def args4 = task.ext.args4 ?: '' def prefix = task.ext.prefix ?: "${meta.id}" - def bam_output = reference.size() > 2.5e9 && bam_format ? "-a | samtools view -b -T ${reference} - > ${prefix}.bam" : reference.size() < 2.5e9 && bam_format ? "-a | samtools view -@ ${task.cpus} -b -h -o ${prefix}.bam" : bed_format ? "| paftools.js splice2bed - > ${prefix}.bed " : "-o ${prefix}.paf" + def bam_index = bam_index_extension ? "${prefix}.bam##idx##${prefix}.bam.${bam_index_extension} --write-index" : "${prefix}.bam" + def bam_output = reference.size() > 2.5e9 && bam_format ? "-a | samtools view -b -T ${reference} - > ${prefix}.bam" : reference.size() < 2.5e9 && bam_format ? "-a | samtools view -@ ${task.cpus} -b -h -o ${prefix}.bam" : bed_bool ? "| paftools.js splice2bed - > ${prefix}.bed " : "-o ${prefix}.paf" def cigar_paf = cigar_paf_format && !bam_format ? "-c" : '' def set_cigar_bam = cigar_bam && bam_format ? "-L" : '' + def bam_input = "${reads.extension}".matches('sam|bam|cram') + def samtools_reset_fastq = bam_input ? "samtools reset --threads ${task.cpus-1} $args3 $reads | samtools fastq --threads ${task.cpus-1} $args4 |" : '' + def query = bam_input ? "-" : reads + def target = reference ?: (bam_input ? error("BAM input requires reference") : reads) """ + $samtools_reset_fastq \\ minimap2 \\ $args \\ -t $task.cpus \\ - "${reference ?: reads}" \\ - "$reads" \\ + $target \\ + $query \\ $cigar_paf \\ $set_cigar_bam \\ $bam_output @@ -46,23 +58,24 @@ process MINIMAP2_ALIGN { cat <<-END_VERSIONS > versions.yml "${task.process}": minimap2: \$(minimap2 --version 2>&1) + samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//') END_VERSIONS """ stub: def prefix = task.ext.prefix ?: "${meta.id}" - def bam_output = reference.size() > 2.5e9 && bam_format ? "-a | samtools view -b -T ${reference} - > ${prefix}.bam" : reference.size() < 2.5e9 && bam_format ? "-a | samtools view -@ ${task.cpus} -b -h -o ${prefix}.bam" : "-o ${prefix}.paf" - def cigar_paf = cigar_paf_format && !bam_format ? "-c" : '' - def set_cigar_bam = cigar_bam && bam_format ? "-L" : '' - def extension = bam_format ? "bam" : bed_format ? "bed" : "paf" + def output_file = bam_format ? "${prefix}.bam" : "${prefix}.paf" + def bam_index = bam_index_extension ? "touch ${prefix}.bam.${bam_index_extension}" : "" + def bam_input = "${reads.extension}".matches('sam|bam|cram') + def target = reference ?: (bam_input ? error("BAM input requires reference") : reads) + """ - touch ${prefix}.${extension} + touch $output_file + ${bam_index} cat <<-END_VERSIONS > versions.yml "${task.process}": minimap2: \$(minimap2 --version 2>&1) END_VERSIONS """ - } - diff --git a/modules/nf-core/minimap2/align/meta.yml b/modules/nf-core/minimap2/align/meta.yml index 991b39a0..a4cfc891 100644 --- a/modules/nf-core/minimap2/align/meta.yml +++ b/modules/nf-core/minimap2/align/meta.yml @@ -14,52 +14,86 @@ tools: homepage: https://github.com/lh3/minimap2 documentation: https://github.com/lh3/minimap2#uguide licence: ["MIT"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - reads: - type: file - description: | - List of input FASTA or FASTQ files of size 1 and 2 for single-end - and paired-end data, respectively. - - reference: - type: file - description: | - Reference database in FASTA format. - - bam_format: - type: boolean - description: Specify that output should be in BAM format - - cigar_paf_format: - type: boolean - description: Specify that output CIGAR should be in PAF format - - cigar_bam: - type: boolean - description: | - Write CIGAR with >65535 ops at the CG tag. This is recommended when - doing XYZ (https://github.com/lh3/minimap2#working-with-65535-cigar-operations) + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - reads: + type: file + description: | + List of input FASTA or FASTQ files of size 1 and 2 for single-end + and paired-end data, respectively. + - - meta2: + type: map + description: | + Groovy Map containing reference information + e.g. [ id:'test_ref'] + - reference: + type: file + description: | + Reference database in FASTA format. + - - bam_format: + type: boolean + description: Specify that output should be in BAM format + - - bam_index_extension: + type: string + description: BAM alignment index extension (e.g. "bai") + - - cigar_paf_format: + type: boolean + description: Specify that output CIGAR should be in PAF format + - - cigar_bam: + type: boolean + description: | + Write CIGAR with >65535 ops at the CG tag. This is recommended when + doing XYZ (https://github.com/lh3/minimap2#working-with-65535-cigar-operations) output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - paf: - type: file - description: Alignment in PAF format - pattern: "*.paf" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.paf": + type: file + description: Alignment in PAF format + pattern: "*.paf" - bam: - type: file - description: Alignment in BAM format - pattern: "*.bam" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bam": + type: file + description: Alignment in BAM format + pattern: "*.bam" + - index: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bam.${bam_index_extension}": + type: file + description: BAM alignment index + pattern: "*.bam.*" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@heuermh" - "@sofstam" - "@sateeshperi" - "@jfy133" + - "@fellen31" +maintainers: + - "@heuermh" + - "@sofstam" + - "@sateeshperi" + - "@jfy133" + - "@fellen31" diff --git a/modules/nf-core/minimap2/align/tests/main.nf.test b/modules/nf-core/minimap2/align/tests/main.nf.test new file mode 100644 index 00000000..4072c171 --- /dev/null +++ b/modules/nf-core/minimap2/align/tests/main.nf.test @@ -0,0 +1,441 @@ +nextflow_process { + + name "Test Process MINIMAP2_ALIGN" + script "../main.nf" + process "MINIMAP2_ALIGN" + + tag "modules" + tag "modules_nfcore" + tag "minimap2" + tag "minimap2/align" + + test("sarscov2 - fastq, fasta, true, [], false, false") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = true + input[3] = [] + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot( + bam(process.out.bam[0][1]).getHeader(), + bam(process.out.bam[0][1]).getReadsMD5(), + process.out.versions + ).match() } + ) + } + + } + + test("sarscov2 - fastq, fasta, true, 'bai', false, false") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = true + input[3] = 'bai' + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot( + bam(process.out.bam[0][1]).getHeader(), + bam(process.out.bam[0][1]).getReadsMD5(), + file(process.out.index[0][1]).name, + process.out.versions + ).match() } + ) + } + + } + + test("sarscov2 - [fastq1, fastq2], fasta, true, false, false") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:false ], // meta map + [ + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_2.fastq.gz', checkIfExists: true) + ] + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = true + input[3] = [] + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot( + bam(process.out.bam[0][1]).getHeader(), + bam(process.out.bam[0][1]).getReadsMD5(), + process.out.versions + ).match() } + ) + } + + } + + test("sarscov2 - fastq, [], true, false, false") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true), + ] + input[1] = [ + [ id:'test_ref' ], // meta map + [] + ] + input[2] = true + input[3] = [] + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot( + bam(process.out.bam[0][1]).getHeader(), + bam(process.out.bam[0][1]).getReadsMD5(), + process.out.versions + ).match() } + ) + } + + } + + test("sarscov2 - bam, fasta, true, [], false, false") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test3.single_end.markduplicates.sorted.bam', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = true + input[3] = [] + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot( + bam(process.out.bam[0][1]).getHeader(), + bam(process.out.bam[0][1]).getReadsMD5(), + process.out.versions + ).match() } + ) + } + + } + + test("sarscov2 - bam, fasta, true, 'bai', false, false") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test3.single_end.markduplicates.sorted.bam', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = true + input[3] = 'bai' + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot( + bam(process.out.bam[0][1]).getHeader(), + bam(process.out.bam[0][1]).getReadsMD5(), + file(process.out.index[0][1]).name, + process.out.versions + ).match() } + ) + } + + } + + test("sarscov2 - bam, [], true, false, false") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test3.single_end.markduplicates.sorted.bam', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + [] + ] + input[2] = true + input[3] = [] + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.failed } + ) + } + + } + + test("sarscov2 - fastq, fasta, true, [], false, false - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = true + input[3] = [] + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("sarscov2 - fastq, fasta, true, 'bai', false, false - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = true + input[3] = 'bai' + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("sarscov2 - fastq, fasta, false, [], false, false - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = false + input[3] = [] + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("sarscov2 - bam, fasta, true, [], false, false - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test3.single_end.markduplicates.sorted.bam', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = true + input[3] = [] + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("sarscov2 - bam, fasta, true, 'bai', false, false - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test3.single_end.markduplicates.sorted.bam', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + input[2] = true + input[3] = 'bai' + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("sarscov2 - bam, [], true, false, false - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test3.single_end.markduplicates.sorted.bam', checkIfExists: true) + ] + input[1] = [ + [ id:'test_ref' ], // meta map + [] + ] + input[2] = true + input[3] = [] + input[4] = false + input[5] = false + """ + } + } + + then { + assertAll( + { assert process.failed } + ) + } + + } + +} \ No newline at end of file diff --git a/modules/nf-core/minimap2/align/tests/main.nf.test.snap b/modules/nf-core/minimap2/align/tests/main.nf.test.snap new file mode 100644 index 00000000..12264a85 --- /dev/null +++ b/modules/nf-core/minimap2/align/tests/main.nf.test.snap @@ -0,0 +1,476 @@ +{ + "sarscov2 - bam, fasta, true, 'bai', false, false": { + "content": [ + [ + "@HD\tVN:1.6\tSO:coordinate", + "@SQ\tSN:MT192765.1\tLN:29829", + "@PG\tID:minimap2\tPN:minimap2\tVN:2.28-r1209\tCL:minimap2 -t 2 -a genome.fasta -", + "@PG\tID:samtools\tPN:samtools\tPP:minimap2\tVN:1.20\tCL:samtools sort -@ 1 -o test.bam##idx##test.bam.bai --write-index" + ], + "5d426b9a5f5b2c54f1d7f1e4c238ae94", + "test.bam.bai", + [ + "versions.yml:md5,3548eeba9066efbf8d78ea99f8d813fd" + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-25T09:03:00.827260362" + }, + "sarscov2 - bam, fasta, true, 'bai', false, false - stub": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "2": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam.bai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "3": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ], + "bam": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "index": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam.bai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "paf": [ + + ], + "versions": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-23T11:21:37.92353539" + }, + "sarscov2 - fastq, fasta, true, 'bai', false, false - stub": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "2": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam.bai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "3": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ], + "bam": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "index": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam.bai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "paf": [ + + ], + "versions": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-03T11:29:44.669021368" + }, + "sarscov2 - fastq, fasta, false, [], false, false - stub": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": true + }, + "test.paf:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ], + "bam": [ + + ], + "index": [ + + ], + "paf": [ + [ + { + "id": "test", + "single_end": true + }, + "test.paf:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-03T11:15:52.738781039" + }, + "sarscov2 - fastq, fasta, true, [], false, false - stub": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ], + "bam": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "index": [ + + ], + "paf": [ + + ], + "versions": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-06-03T11:15:23.033808223" + }, + "sarscov2 - [fastq1, fastq2], fasta, true, false, false": { + "content": [ + [ + "@HD\tVN:1.6\tSO:coordinate", + "@SQ\tSN:MT192765.1\tLN:29829", + "@PG\tID:minimap2\tPN:minimap2\tVN:2.28-r1209\tCL:minimap2 -t 2 -a genome.fasta test_1.fastq.gz test_2.fastq.gz", + "@PG\tID:samtools\tPN:samtools\tPP:minimap2\tVN:1.20\tCL:samtools sort -@ 1 -o test.bam" + ], + "1bc392244f228bf52cf0b5a8f6a654c9", + [ + "versions.yml:md5,3548eeba9066efbf8d78ea99f8d813fd" + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-23T11:18:18.964586894" + }, + "sarscov2 - fastq, fasta, true, [], false, false": { + "content": [ + [ + "@HD\tVN:1.6\tSO:coordinate", + "@SQ\tSN:MT192765.1\tLN:29829", + "@PG\tID:minimap2\tPN:minimap2\tVN:2.28-r1209\tCL:minimap2 -t 2 -a genome.fasta test_1.fastq.gz", + "@PG\tID:samtools\tPN:samtools\tPP:minimap2\tVN:1.20\tCL:samtools sort -@ 1 -o test.bam" + ], + "f194745c0ccfcb2a9c0aee094a08750", + [ + "versions.yml:md5,3548eeba9066efbf8d78ea99f8d813fd" + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-23T11:17:48.667488325" + }, + "sarscov2 - fastq, fasta, true, 'bai', false, false": { + "content": [ + [ + "@HD\tVN:1.6\tSO:coordinate", + "@SQ\tSN:MT192765.1\tLN:29829", + "@PG\tID:minimap2\tPN:minimap2\tVN:2.28-r1209\tCL:minimap2 -t 2 -a genome.fasta test_1.fastq.gz", + "@PG\tID:samtools\tPN:samtools\tPP:minimap2\tVN:1.20\tCL:samtools sort -@ 1 -o test.bam##idx##test.bam.bai --write-index" + ], + "f194745c0ccfcb2a9c0aee094a08750", + "test.bam.bai", + [ + "versions.yml:md5,3548eeba9066efbf8d78ea99f8d813fd" + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-23T11:18:02.517416733" + }, + "sarscov2 - bam, fasta, true, [], false, false": { + "content": [ + [ + "@HD\tVN:1.6\tSO:coordinate", + "@SQ\tSN:MT192765.1\tLN:29829", + "@PG\tID:minimap2\tPN:minimap2\tVN:2.28-r1209\tCL:minimap2 -t 2 -a genome.fasta -", + "@PG\tID:samtools\tPN:samtools\tPP:minimap2\tVN:1.20\tCL:samtools sort -@ 1 -o test.bam" + ], + "5d426b9a5f5b2c54f1d7f1e4c238ae94", + [ + "versions.yml:md5,3548eeba9066efbf8d78ea99f8d813fd" + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-25T09:02:49.64829488" + }, + "sarscov2 - bam, fasta, true, [], false, false - stub": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ], + "bam": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "index": [ + + ], + "paf": [ + + ], + "versions": [ + "versions.yml:md5,98b8f5f36aa54b82210094f0b0d11938" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-23T11:21:22.162291795" + }, + "sarscov2 - fastq, [], true, false, false": { + "content": [ + [ + "@HD\tVN:1.6\tSO:coordinate", + "@SQ\tSN:ERR5069949.2151832\tLN:150", + "@SQ\tSN:ERR5069949.576388\tLN:77", + "@SQ\tSN:ERR5069949.501486\tLN:146", + "@SQ\tSN:ERR5069949.1331889\tLN:132", + "@SQ\tSN:ERR5069949.2161340\tLN:80", + "@SQ\tSN:ERR5069949.973930\tLN:79", + "@SQ\tSN:ERR5069949.2417063\tLN:150", + "@SQ\tSN:ERR5069949.376959\tLN:151", + "@SQ\tSN:ERR5069949.1088785\tLN:149", + "@SQ\tSN:ERR5069949.1066259\tLN:147", + "@SQ\tSN:ERR5069949.2832676\tLN:139", + "@SQ\tSN:ERR5069949.2953930\tLN:151", + "@SQ\tSN:ERR5069949.324865\tLN:151", + "@SQ\tSN:ERR5069949.2185111\tLN:150", + "@SQ\tSN:ERR5069949.937422\tLN:151", + "@SQ\tSN:ERR5069949.2431709\tLN:150", + "@SQ\tSN:ERR5069949.1246538\tLN:148", + "@SQ\tSN:ERR5069949.1189252\tLN:98", + "@SQ\tSN:ERR5069949.2216307\tLN:147", + "@SQ\tSN:ERR5069949.3273002\tLN:148", + "@SQ\tSN:ERR5069949.3277445\tLN:151", + "@SQ\tSN:ERR5069949.3022231\tLN:147", + "@SQ\tSN:ERR5069949.184542\tLN:151", + "@SQ\tSN:ERR5069949.540529\tLN:149", + "@SQ\tSN:ERR5069949.686090\tLN:150", + "@SQ\tSN:ERR5069949.2787556\tLN:106", + "@SQ\tSN:ERR5069949.2650879\tLN:150", + "@SQ\tSN:ERR5069949.2064910\tLN:149", + "@SQ\tSN:ERR5069949.2328704\tLN:150", + "@SQ\tSN:ERR5069949.1067032\tLN:150", + "@SQ\tSN:ERR5069949.3338256\tLN:151", + "@SQ\tSN:ERR5069949.1412839\tLN:147", + "@SQ\tSN:ERR5069949.1538968\tLN:150", + "@SQ\tSN:ERR5069949.147998\tLN:94", + "@SQ\tSN:ERR5069949.366975\tLN:106", + "@SQ\tSN:ERR5069949.1372331\tLN:151", + "@SQ\tSN:ERR5069949.1709367\tLN:129", + "@SQ\tSN:ERR5069949.2388984\tLN:150", + "@SQ\tSN:ERR5069949.1132353\tLN:150", + "@SQ\tSN:ERR5069949.1151736\tLN:151", + "@SQ\tSN:ERR5069949.479807\tLN:150", + "@SQ\tSN:ERR5069949.2176303\tLN:151", + "@SQ\tSN:ERR5069949.2772897\tLN:151", + "@SQ\tSN:ERR5069949.1020777\tLN:122", + "@SQ\tSN:ERR5069949.465452\tLN:151", + "@SQ\tSN:ERR5069949.1704586\tLN:149", + "@SQ\tSN:ERR5069949.1258508\tLN:151", + "@SQ\tSN:ERR5069949.986441\tLN:119", + "@SQ\tSN:ERR5069949.2674295\tLN:148", + "@SQ\tSN:ERR5069949.885966\tLN:79", + "@SQ\tSN:ERR5069949.2342766\tLN:151", + "@SQ\tSN:ERR5069949.3122970\tLN:127", + "@SQ\tSN:ERR5069949.3279513\tLN:72", + "@SQ\tSN:ERR5069949.309410\tLN:151", + "@SQ\tSN:ERR5069949.532979\tLN:149", + "@SQ\tSN:ERR5069949.2888794\tLN:151", + "@SQ\tSN:ERR5069949.2205229\tLN:150", + "@SQ\tSN:ERR5069949.786562\tLN:151", + "@SQ\tSN:ERR5069949.919671\tLN:151", + "@SQ\tSN:ERR5069949.1328186\tLN:151", + "@SQ\tSN:ERR5069949.870926\tLN:149", + "@SQ\tSN:ERR5069949.2257580\tLN:151", + "@SQ\tSN:ERR5069949.3249622\tLN:77", + "@SQ\tSN:ERR5069949.611123\tLN:125", + "@SQ\tSN:ERR5069949.651338\tLN:142", + "@SQ\tSN:ERR5069949.169513\tLN:92", + "@SQ\tSN:ERR5069949.155944\tLN:150", + "@SQ\tSN:ERR5069949.2033605\tLN:150", + "@SQ\tSN:ERR5069949.2730382\tLN:142", + "@SQ\tSN:ERR5069949.2125592\tLN:150", + "@SQ\tSN:ERR5069949.1062611\tLN:151", + "@SQ\tSN:ERR5069949.1778133\tLN:151", + "@SQ\tSN:ERR5069949.3057020\tLN:95", + "@SQ\tSN:ERR5069949.2972968\tLN:141", + "@SQ\tSN:ERR5069949.2734474\tLN:149", + "@SQ\tSN:ERR5069949.856527\tLN:151", + "@SQ\tSN:ERR5069949.2098070\tLN:151", + "@SQ\tSN:ERR5069949.1552198\tLN:150", + "@SQ\tSN:ERR5069949.2385514\tLN:150", + "@SQ\tSN:ERR5069949.2270078\tLN:151", + "@SQ\tSN:ERR5069949.114870\tLN:150", + "@SQ\tSN:ERR5069949.2668880\tLN:147", + "@SQ\tSN:ERR5069949.257821\tLN:139", + "@SQ\tSN:ERR5069949.2243023\tLN:150", + "@SQ\tSN:ERR5069949.2605155\tLN:146", + "@SQ\tSN:ERR5069949.1340552\tLN:151", + "@SQ\tSN:ERR5069949.1561137\tLN:150", + "@SQ\tSN:ERR5069949.2361683\tLN:149", + "@SQ\tSN:ERR5069949.2521353\tLN:150", + "@SQ\tSN:ERR5069949.1261808\tLN:149", + "@SQ\tSN:ERR5069949.2734873\tLN:98", + "@SQ\tSN:ERR5069949.3017828\tLN:107", + "@SQ\tSN:ERR5069949.573706\tLN:150", + "@SQ\tSN:ERR5069949.1980512\tLN:151", + "@SQ\tSN:ERR5069949.1014693\tLN:150", + "@SQ\tSN:ERR5069949.3184655\tLN:150", + "@SQ\tSN:ERR5069949.29668\tLN:89", + "@SQ\tSN:ERR5069949.3258358\tLN:151", + "@SQ\tSN:ERR5069949.1476386\tLN:151", + "@SQ\tSN:ERR5069949.2415814\tLN:150", + "@PG\tID:minimap2\tPN:minimap2\tVN:2.28-r1209\tCL:minimap2 -t 2 -a test_1.fastq.gz test_1.fastq.gz", + "@PG\tID:samtools\tPN:samtools\tPP:minimap2\tVN:1.20\tCL:samtools sort -@ 1 -o test.bam" + ], + "16c1c651f8ec67383bcdee3c55aed94f", + [ + "versions.yml:md5,3548eeba9066efbf8d78ea99f8d813fd" + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-23T11:18:34.246998277" + } +} \ No newline at end of file diff --git a/modules/nf-core/minimap2/align/tests/tags.yml b/modules/nf-core/minimap2/align/tests/tags.yml new file mode 100644 index 00000000..39dba374 --- /dev/null +++ b/modules/nf-core/minimap2/align/tests/tags.yml @@ -0,0 +1,2 @@ +minimap2/align: + - "modules/nf-core/minimap2/align/**" diff --git a/modules/nf-core/minimap2/index/environment.yml b/modules/nf-core/minimap2/index/environment.yml index 2a66e410..d1c1b471 100644 --- a/modules/nf-core/minimap2/index/environment.yml +++ b/modules/nf-core/minimap2/index/environment.yml @@ -1,7 +1,5 @@ -name: minimap2_index channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::minimap2=2.24 + - bioconda::minimap2=2.28 diff --git a/modules/nf-core/minimap2/index/main.nf b/modules/nf-core/minimap2/index/main.nf index 45e1cec0..38320214 100644 --- a/modules/nf-core/minimap2/index/main.nf +++ b/modules/nf-core/minimap2/index/main.nf @@ -1,11 +1,11 @@ process MINIMAP2_INDEX { - label 'process_medium' + label 'process_low' // Note: the versions here need to match the versions used in minimap2/align conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/minimap2:2.24--h7132678_1' : - 'biocontainers/minimap2:2.24--h7132678_1' }" + 'https://depot.galaxyproject.org/singularity/minimap2:2.28--he4a0461_0' : + 'biocontainers/minimap2:2.28--he4a0461_0' }" input: tuple val(meta), path(fasta) @@ -31,4 +31,14 @@ process MINIMAP2_INDEX { minimap2: \$(minimap2 --version 2>&1) END_VERSIONS """ + + stub: + """ + touch ${fasta.baseName}.mmi + + cat <<-END_VERSIONS > versions.yml + "${task.process}": + minimap2: \$(minimap2 --version 2>&1) + END_VERSIONS + """ } diff --git a/modules/nf-core/minimap2/index/meta.yml b/modules/nf-core/minimap2/index/meta.yml index 1d29e3f2..57c80e29 100644 --- a/modules/nf-core/minimap2/index/meta.yml +++ b/modules/nf-core/minimap2/index/meta.yml @@ -11,30 +11,33 @@ tools: homepage: https://github.com/lh3/minimap2 documentation: https://github.com/lh3/minimap2#uguide licence: ["MIT"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - fasta: - type: file - description: | - Reference database in FASTA format. + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - fasta: + type: file + description: | + Reference database in FASTA format. output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - index: - type: file - description: Minimap2 fasta index. - pattern: "*.mmi" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.mmi": + type: file + description: Minimap2 fasta index. + pattern: "*.mmi" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@yuukiiwa" - "@drpatelh" diff --git a/modules/nf-core/minimap2/index/tests/main.nf.test b/modules/nf-core/minimap2/index/tests/main.nf.test new file mode 100644 index 00000000..97840ff7 --- /dev/null +++ b/modules/nf-core/minimap2/index/tests/main.nf.test @@ -0,0 +1,32 @@ +nextflow_process { + + name "Test Process MINIMAP2_INDEX" + script "../main.nf" + process "MINIMAP2_INDEX" + + tag "modules" + tag "modules_nfcore" + tag "minimap2" + tag "minimap2/index" + + test("minimap2 index") { + + when { + process { + """ + input[0] = [ + [ id:'test' ], + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ] + """ + } + } + + then { + assert process.success + assert snapshot(process.out).match() + } + + } + +} \ No newline at end of file diff --git a/modules/nf-core/minimap2/index/tests/main.nf.test.snap b/modules/nf-core/minimap2/index/tests/main.nf.test.snap new file mode 100644 index 00000000..0b098828 --- /dev/null +++ b/modules/nf-core/minimap2/index/tests/main.nf.test.snap @@ -0,0 +1,68 @@ +{ + "Should run without failures": { + "content": [ + { + "0": [ + [ + { + "id": "test_ref" + }, + "genome.mmi:md5,72e450f12dc691e763c697463bdb1571" + ] + ], + "1": [ + "versions.yml:md5,0fced0ee8015e7f50b82566e3db8f7b0" + ], + "index": [ + [ + { + "id": "test_ref" + }, + "genome.mmi:md5,72e450f12dc691e763c697463bdb1571" + ] + ], + "versions": [ + "versions.yml:md5,0fced0ee8015e7f50b82566e3db8f7b0" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-03-18T11:46:30.000058092" + }, + "minimap2 index": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "genome.mmi:md5,72e450f12dc691e763c697463bdb1571" + ] + ], + "1": [ + "versions.yml:md5,2f8340380c6741e9261a284262a90bde" + ], + "index": [ + [ + { + "id": "test" + }, + "genome.mmi:md5,72e450f12dc691e763c697463bdb1571" + ] + ], + "versions": [ + "versions.yml:md5,2f8340380c6741e9261a284262a90bde" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.10.1" + }, + "timestamp": "2024-04-05T10:58:29.828187662" + } +} \ No newline at end of file diff --git a/modules/nf-core/minimap2/index/tests/tags.yml b/modules/nf-core/minimap2/index/tests/tags.yml new file mode 100644 index 00000000..e5ef8e19 --- /dev/null +++ b/modules/nf-core/minimap2/index/tests/tags.yml @@ -0,0 +1,2 @@ +minimap2/index: + - modules/nf-core/minimap2/index/** diff --git a/modules/nf-core/miniprot/align/environment.yml b/modules/nf-core/miniprot/align/environment.yml index d437e07a..7e258a04 100644 --- a/modules/nf-core/miniprot/align/environment.yml +++ b/modules/nf-core/miniprot/align/environment.yml @@ -1,7 +1,5 @@ -name: miniprot_align channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::miniprot=0.11=he4a0461_2 diff --git a/modules/nf-core/miniprot/align/meta.yml b/modules/nf-core/miniprot/align/meta.yml index 8b4ffe9e..aadd6fdf 100644 --- a/modules/nf-core/miniprot/align/meta.yml +++ b/modules/nf-core/miniprot/align/meta.yml @@ -14,40 +14,49 @@ tools: homepage: https://github.com/lh3/miniprot documentation: https://github.com/lh3/miniprot licence: ["MIT"] + identifier: biotools:miniprot input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - pep: - type: file - description: a fasta file contains one or multiple protein sequences - - meta2: - type: map - description: | - Groovy Map containing reference information - - ref: - type: file - description: Reference database in FASTA format or miniprot index format. + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - pep: + type: file + description: a fasta file contains one or multiple protein sequences + - - meta2: + type: map + description: | + Groovy Map containing reference information + - ref: + type: file + description: Reference database in FASTA format or miniprot index format. output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - paf: - type: file - description: Alignment in PAF format - pattern: "*.paf" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.paf": + type: file + description: Alignment in PAF format + pattern: "*.paf" - gff: - type: file - description: Alignment in gff format - pattern: "*.gff" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.gff": + type: file + description: Alignment in gff format + pattern: "*.gff" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@yumisims" - "@muffato" diff --git a/modules/nf-core/miniprot/align/tests/main.nf.test b/modules/nf-core/miniprot/align/tests/main.nf.test new file mode 100644 index 00000000..12366934 --- /dev/null +++ b/modules/nf-core/miniprot/align/tests/main.nf.test @@ -0,0 +1,89 @@ + +nextflow_process { + + name "Test Process MINIPROT_ALIGN" + script "../main.nf" + process "MINIPROT_ALIGN" + + + tag "modules" + tag "modules_nfcore" + tag "miniprot" + tag "miniprot/align" + tag "miniprot/index" + + setup { + run("MINIPROT_INDEX") { + script "../../index/main.nf" + process { + """ + input[0] =[ [id:'test'], file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) ] + """ + } + } + } + + test("test-miniprot-align-gff") { + config "./nextflow.gff.config" + + when { + process { + """ + input[0] = [ [id:'test'], file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/proteome.fasta', checkIfExists: true)] + input[1] = MINIPROT_INDEX.out.index + + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("test-miniprot-align-paf") { + config "./nextflow.paf.config" + + when { + process { + """ + input[0] = [ [id:'test'], file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/proteome.fasta', checkIfExists: true)] + input[1] = MINIPROT_INDEX.out.index + + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("test-miniprot-align-paf-stub") { + config "./nextflow.paf.config" + options '-stub' + when { + process { + """ + input[0] = [ [id:'test'], file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/proteome.fasta', checkIfExists: true)] + input[1] = MINIPROT_INDEX.out.index + + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + +} diff --git a/modules/nf-core/miniprot/align/tests/main.nf.test.snap b/modules/nf-core/miniprot/align/tests/main.nf.test.snap new file mode 100644 index 00000000..63f41136 --- /dev/null +++ b/modules/nf-core/miniprot/align/tests/main.nf.test.snap @@ -0,0 +1,119 @@ +{ + "test-miniprot-align-paf-stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.paf:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + + ], + "2": [ + "versions.yml:md5,cd4a5baa0b71a4edc70e556289a35ee9" + ], + "gff": [ + + ], + "paf": [ + [ + { + "id": "test" + }, + "test.paf:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,cd4a5baa0b71a4edc70e556289a35ee9" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-08-30T13:45:20.16677" + }, + "test-miniprot-align-paf": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.paf:md5,f2b40da6a9605df44efc9df14b255b53" + ] + ], + "1": [ + + ], + "2": [ + "versions.yml:md5,cd4a5baa0b71a4edc70e556289a35ee9" + ], + "gff": [ + + ], + "paf": [ + [ + { + "id": "test" + }, + "test.paf:md5,f2b40da6a9605df44efc9df14b255b53" + ] + ], + "versions": [ + "versions.yml:md5,cd4a5baa0b71a4edc70e556289a35ee9" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-08-30T13:45:15.160286" + }, + "test-miniprot-align-gff": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test" + }, + "test.gff:md5,dc2d01ac840ee26416b5eb9cf252181d" + ] + ], + "2": [ + "versions.yml:md5,cd4a5baa0b71a4edc70e556289a35ee9" + ], + "gff": [ + [ + { + "id": "test" + }, + "test.gff:md5,dc2d01ac840ee26416b5eb9cf252181d" + ] + ], + "paf": [ + + ], + "versions": [ + "versions.yml:md5,cd4a5baa0b71a4edc70e556289a35ee9" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-08-30T13:45:09.941901" + } +} \ No newline at end of file diff --git a/modules/nf-core/miniprot/align/tests/nextflow.gff.config b/modules/nf-core/miniprot/align/tests/nextflow.gff.config new file mode 100644 index 00000000..f97d4fd2 --- /dev/null +++ b/modules/nf-core/miniprot/align/tests/nextflow.gff.config @@ -0,0 +1,5 @@ +process { + withName: 'MINIPROT_ALIGN' { + ext.args = "-u --gff" + } +} diff --git a/modules/nf-core/miniprot/align/tests/nextflow.paf.config b/modules/nf-core/miniprot/align/tests/nextflow.paf.config new file mode 100644 index 00000000..8d9c2550 --- /dev/null +++ b/modules/nf-core/miniprot/align/tests/nextflow.paf.config @@ -0,0 +1,5 @@ +process { + withName: 'MINIPROT_ALIGN' { + ext.args = "-u" + } +} diff --git a/modules/nf-core/miniprot/index/environment.yml b/modules/nf-core/miniprot/index/environment.yml new file mode 100644 index 00000000..7e258a04 --- /dev/null +++ b/modules/nf-core/miniprot/index/environment.yml @@ -0,0 +1,5 @@ +channels: + - conda-forge + - bioconda +dependencies: + - bioconda::miniprot=0.11=he4a0461_2 diff --git a/modules/nf-core/miniprot/index/main.nf b/modules/nf-core/miniprot/index/main.nf index ee3757b6..f2a036cb 100644 --- a/modules/nf-core/miniprot/index/main.nf +++ b/modules/nf-core/miniprot/index/main.nf @@ -2,7 +2,7 @@ process MINIPROT_INDEX { tag "$meta.id" label 'process_medium' - conda "bioconda::miniprot=0.11=he4a0461_2" + conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? 'https://depot.galaxyproject.org/singularity/miniprot:0.11--he4a0461_2': 'biocontainers/miniprot:0.11--he4a0461_2' }" diff --git a/modules/nf-core/miniprot/index/meta.yml b/modules/nf-core/miniprot/index/meta.yml index 3a098802..bd076ada 100644 --- a/modules/nf-core/miniprot/index/meta.yml +++ b/modules/nf-core/miniprot/index/meta.yml @@ -12,30 +12,36 @@ tools: homepage: https://github.com/lh3/miniprot documentation: https://github.com/lh3/miniprot licence: ["MIT"] + identifier: biotools:miniprot input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - fasta: - type: file - description: | - Reference database in FASTA format. + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - fasta: + type: file + description: | + Reference database in FASTA format. output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - index: - type: file - description: miniprot fasta index. - pattern: "*.mpi" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.mpi": + type: file + description: miniprot fasta index. + pattern: "*.mpi" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@yumisims" - "@muffato" +maintainers: + - "@yumisims" + - "@muffato" diff --git a/modules/nf-core/miniprot/index/tests/main.nf.test b/modules/nf-core/miniprot/index/tests/main.nf.test new file mode 100644 index 00000000..b6092df9 --- /dev/null +++ b/modules/nf-core/miniprot/index/tests/main.nf.test @@ -0,0 +1,51 @@ + +nextflow_process { + + name "Test Process MINIPROT_INDEX" + script "../main.nf" + process "MINIPROT_INDEX" + + tag "modules" + tag "modules_nfcore" + tag "miniprot" + tag "miniprot/index" + + test("test-miniprot-index") { + + when { + process { + """ + input[0] = [ [id:'test'], file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) ] + + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("test-miniprot-index-stub") { + options '-stub' + when { + process { + """ + input[0] = [ [id:'test'], file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) ] + + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + +} diff --git a/modules/nf-core/miniprot/index/tests/main.nf.test.snap b/modules/nf-core/miniprot/index/tests/main.nf.test.snap new file mode 100644 index 00000000..3bfc4b72 --- /dev/null +++ b/modules/nf-core/miniprot/index/tests/main.nf.test.snap @@ -0,0 +1,68 @@ +{ + "test-miniprot-index": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "genome.mpi:md5,6161e75cdeca970d844b6cfada14766b" + ] + ], + "1": [ + "versions.yml:md5,1e6ea592f24125bd879956ea78f6afeb" + ], + "index": [ + [ + { + "id": "test" + }, + "genome.mpi:md5,6161e75cdeca970d844b6cfada14766b" + ] + ], + "versions": [ + "versions.yml:md5,1e6ea592f24125bd879956ea78f6afeb" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-08-30T13:35:37.561483" + }, + "test-miniprot-index-stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "genome.mpi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,1e6ea592f24125bd879956ea78f6afeb" + ], + "index": [ + [ + { + "id": "test" + }, + "genome.mpi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,1e6ea592f24125bd879956ea78f6afeb" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-08-30T13:35:42.977101" + } +} \ No newline at end of file diff --git a/modules/nf-core/mummer/environment.yml b/modules/nf-core/mummer/environment.yml new file mode 100644 index 00000000..643eff0f --- /dev/null +++ b/modules/nf-core/mummer/environment.yml @@ -0,0 +1,5 @@ +channels: + - conda-forge + - bioconda +dependencies: + - bioconda::mummer=3.23 diff --git a/modules/nf-core/mummer/main.nf b/modules/nf-core/mummer/main.nf index 37d8e2b0..17b05dd5 100644 --- a/modules/nf-core/mummer/main.nf +++ b/modules/nf-core/mummer/main.nf @@ -3,13 +3,13 @@ process MUMMER { label 'process_low' // WARN: Version information not provided by tool on CLI. Please update version string below when bumping container versions. - conda "bioconda::mummer=3.23" + conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? 'https://depot.galaxyproject.org/singularity/mummer:3.23--pl5262h1b792b2_12' : 'biocontainers/mummer:3.23--pl5262h1b792b2_12' }" input: - tuple val(meta), path(ref, stageAs: 'ref.fasta'), path(query, stageAs: 'query.fasta') // Staging removes the ability to deal with zipped files here + tuple val(meta), path(ref), path(query) output: tuple val(meta), path("*.coords"), emit: coords @@ -50,7 +50,7 @@ process MUMMER { def prefix = task.ext.prefix ?: "${meta.id}" def VERSION = '3.23' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. """ - touch ${prefx}.coords + touch ${prefix}.coords cat <<-END_VERSIONS > versions.yml "${task.process}": diff --git a/modules/nf-core/mummer/meta.yml b/modules/nf-core/mummer/meta.yml index f03d483c..3a4be42f 100644 --- a/modules/nf-core/mummer/meta.yml +++ b/modules/nf-core/mummer/meta.yml @@ -12,37 +12,41 @@ tools: tool_dev_url: http://mummer.sourceforge.net/ doi: 10.1186/gb-2004-5-2-r12 licence: ["The Artistic License"] - + identifier: biotools:mummer input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - ref: - type: file - description: FASTA file of the reference sequence - pattern: "*.{fasta,fasta.gz,fa,fa.gz,fna,fna.gz}" - - query: - type: file - description: FASTA file of the query sequence - pattern: "*.{fasta,fasta.gz,fa,fa.gz,fna,fna.gz}" - + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ref: + type: file + description: FASTA file of the reference sequence + pattern: "*.{fasta,fasta.gz,fa,fa.gz,fna,fna.gz}" + - query: + type: file + description: FASTA file of the query sequence + pattern: "*.{fasta,fasta.gz,fa,fa.gz,fna,fna.gz}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - coords: - type: file - description: File containing coordinates of matches between reference and query sequence - pattern: "*.coords" - + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.coords": + type: file + description: File containing coordinates of matches between reference and query + sequence + pattern: "*.coords" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@mjcipriano" - "@sateeshperi" +maintainers: + - "@mjcipriano" + - "@sateeshperi" diff --git a/modules/nf-core/mummer/mummer.diff b/modules/nf-core/mummer/mummer.diff deleted file mode 100644 index a0391dd0..00000000 --- a/modules/nf-core/mummer/mummer.diff +++ /dev/null @@ -1,14 +0,0 @@ -Changes in module 'nf-core/mummer' ---- modules/nf-core/mummer/main.nf -+++ modules/nf-core/mummer/main.nf -@@ -9,7 +9,7 @@ - 'biocontainers/mummer:3.23--pl5262h1b792b2_12' }" - - input: -- tuple val(meta), path(ref), path(query) -+ tuple val(meta), path(ref, stageAs: 'ref.fasta'), path(query, stageAs: 'query.fasta') // Staging removes the ability to deal with zipped files here - - output: - tuple val(meta), path("*.coords"), emit: coords - -************************************************************ diff --git a/modules/nf-core/mummer/tests/main.nf.test b/modules/nf-core/mummer/tests/main.nf.test new file mode 100644 index 00000000..b079d209 --- /dev/null +++ b/modules/nf-core/mummer/tests/main.nf.test @@ -0,0 +1,58 @@ +nextflow_process { + + name "Test Process MUMMER" + script "../main.nf" + process "MUMMER" + + tag "modules" + tag "modules_nfcore" + tag "mummer" + + test("sarscov2 - bam") { + + when { + process { + """ + + input[0] = [ [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/transcriptome.fasta', checkIfExists: true) + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("sarscov2 - bam - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ [ id:'test' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/transcriptome.fasta', checkIfExists: true) + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + +} diff --git a/modules/nf-core/mummer/tests/main.nf.test.snap b/modules/nf-core/mummer/tests/main.nf.test.snap new file mode 100644 index 00000000..fea2282a --- /dev/null +++ b/modules/nf-core/mummer/tests/main.nf.test.snap @@ -0,0 +1,68 @@ +{ + "sarscov2 - bam - stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.coords:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,c36e386e2de805cf815ba20359fca81c" + ], + "coords": [ + [ + { + "id": "test" + }, + "test.coords:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,c36e386e2de805cf815ba20359fca81c" + ] + } + ], + "meta": { + "nf-test": "0.9.2", + "nextflow": "24.10.0" + }, + "timestamp": "2024-11-12T06:08:50.368788562" + }, + "sarscov2 - bam": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.coords:md5,6084fe43c7cb2eca8b96d674560bdefc" + ] + ], + "1": [ + "versions.yml:md5,c36e386e2de805cf815ba20359fca81c" + ], + "coords": [ + [ + { + "id": "test" + }, + "test.coords:md5,6084fe43c7cb2eca8b96d674560bdefc" + ] + ], + "versions": [ + "versions.yml:md5,c36e386e2de805cf815ba20359fca81c" + ] + } + ], + "meta": { + "nf-test": "0.9.2", + "nextflow": "24.10.0" + }, + "timestamp": "2024-11-12T06:08:40.932078101" + } +} \ No newline at end of file diff --git a/modules/nf-core/paftools/sam2paf/environment.yml b/modules/nf-core/paftools/sam2paf/environment.yml index a2f0c8e9..988733dd 100644 --- a/modules/nf-core/paftools/sam2paf/environment.yml +++ b/modules/nf-core/paftools/sam2paf/environment.yml @@ -1,8 +1,6 @@ -name: paftools_sam2paf channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::minimap2=2.24 - bioconda::samtools=1.14 diff --git a/modules/nf-core/paftools/sam2paf/main.nf b/modules/nf-core/paftools/sam2paf/main.nf index a9584000..914dd9ba 100644 --- a/modules/nf-core/paftools/sam2paf/main.nf +++ b/modules/nf-core/paftools/sam2paf/main.nf @@ -12,7 +12,7 @@ process PAFTOOLS_SAM2PAF { tuple val(meta), path(bam) output: - tuple val(meta), file("*.paf") , emit: paf + tuple val(meta), file("*.paf") , emit: paf path "versions.yml" , emit: versions when: diff --git a/modules/nf-core/paftools/sam2paf/meta.yml b/modules/nf-core/paftools/sam2paf/meta.yml index 438ab2dd..e5829740 100644 --- a/modules/nf-core/paftools/sam2paf/meta.yml +++ b/modules/nf-core/paftools/sam2paf/meta.yml @@ -11,30 +11,28 @@ tools: homepage: https://github.com/lh3/minimap2 documentation: https://github.com/lh3/minimap2/blob/master/README.md licence: ["MIT"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - bam: - type: file - description: An input bam file to be converted into paf. + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - bam: + type: file + description: An input bam file to be converted into paf. output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - paf: - type: file - description: | - An output paf containing detailed data about the sample - pattern: "${prefix}.paf" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@DLBPointon" maintainers: diff --git a/modules/nf-core/paftools/sam2paf/tests/main.nf.test b/modules/nf-core/paftools/sam2paf/tests/main.nf.test new file mode 100644 index 00000000..be31cc33 --- /dev/null +++ b/modules/nf-core/paftools/sam2paf/tests/main.nf.test @@ -0,0 +1,57 @@ + +nextflow_process { + + name "Test Process PAFTOOLS_SAM2PAF" + script "../main.nf" + process "PAFTOOLS_SAM2PAF" + + tag "modules" + tag "modules_nfcore" + tag "paftools" + tag "paftools/sam2paf" + + test("test-paftools-sam2paf") { + + when { + process { + """ + input[0] = [ + [id:'test'], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/scramble/test.bam', checkIfExists: true) + ] + + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("test-paftools-sam2paf-stub") { + options '-stub' + when { + process { + """ + input[0] = [ + [id:'test'], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/scramble/test.bam', checkIfExists: true) + ] + + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + +} diff --git a/modules/nf-core/paftools/sam2paf/tests/main.nf.test.snap b/modules/nf-core/paftools/sam2paf/tests/main.nf.test.snap new file mode 100644 index 00000000..3e7ab899 --- /dev/null +++ b/modules/nf-core/paftools/sam2paf/tests/main.nf.test.snap @@ -0,0 +1,68 @@ +{ + "test-paftools-sam2paf": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.paf:md5,bf981d964f3ffa6feac079199e157330" + ] + ], + "1": [ + "versions.yml:md5,ae0342a5ff0006c183531b23d9fb5d4f" + ], + "paf": [ + [ + { + "id": "test" + }, + "test.paf:md5,bf981d964f3ffa6feac079199e157330" + ] + ], + "versions": [ + "versions.yml:md5,ae0342a5ff0006c183531b23d9fb5d4f" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-08-30T12:37:30.82847" + }, + "test-paftools-sam2paf-stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.paf:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,88072120e3a09a8338ce7ae0c844067e" + ], + "paf": [ + [ + { + "id": "test" + }, + "test.paf:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,88072120e3a09a8338ce7ae0c844067e" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.4" + }, + "timestamp": "2024-08-30T12:37:35.457832" + } +} \ No newline at end of file diff --git a/modules/nf-core/pretextmap/environment.yml b/modules/nf-core/pretextmap/environment.yml new file mode 100644 index 00000000..bfbbee2b --- /dev/null +++ b/modules/nf-core/pretextmap/environment.yml @@ -0,0 +1,6 @@ +channels: + - conda-forge + - bioconda +dependencies: + - bioconda::pretextmap=0.1.9 + - bioconda::samtools=1.17 diff --git a/modules/nf-core/pretextmap/main.nf b/modules/nf-core/pretextmap/main.nf index 87427c40..cbcaf68c 100644 --- a/modules/nf-core/pretextmap/main.nf +++ b/modules/nf-core/pretextmap/main.nf @@ -3,24 +3,25 @@ process PRETEXTMAP { tag "$meta.id" label 'process_single' - container "quay.io/sanger-tol/pretext:0.0.2-yy5-c3" + conda "${moduleDir}/environment.yml" + container "quay.io/sanger-tol/pretext:0.0.3-yy5-c1" input: - tuple val(meta), path(input) - tuple val(meta2), path(fasta), path(genome) + tuple val(meta), path(input) + tuple val(meta2), path(fasta), path(fai) output: - tuple val(meta), path("*.pretext"), emit: pretext - path "versions.yml", emit: versions + tuple val(meta), path("*.pretext") , emit: pretext + path "versions.yml" , emit: versions when: task.ext.when == null || task.ext.when script: - def VERSION = "0.1.9" - def args = task.ext.args ?: '' - def prefix = task.ext.prefix ?: "${meta.id}" - def reference = fasta ? "--reference ${fasta}" : "" + def VERSION = "0.1.9" + def args = task.ext.args ?: '' + def prefix = task.ext.prefix ?: "${meta.id}" + def reference = fasta ? "--reference ${fasta}" : "" """ if [[ $input == *.pairs.gz ]]; then @@ -39,21 +40,21 @@ process PRETEXTMAP { cat <<-END_VERSIONS > versions.yml "${task.process}": - PretextMap: $VERSION + pretextmap: $VERSION samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//' ) END_VERSIONS """ stub: - def VERSION = "0.1.9" - def prefix = task.ext.prefix ?: "${meta.id}" + def VERSION = "0.1.9" + def prefix = task.ext.prefix ?: "${meta.id}" """ touch ${prefix}.pretext cat <<-END_VERSIONS > versions.yml "${task.process}": - PretextMap: $VERSION - samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//' )) + pretextmap: $VERSION + samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//') END_VERSIONS """ } diff --git a/modules/nf-core/pretextmap/meta.yml b/modules/nf-core/pretextmap/meta.yml index 47811974..d79295cd 100644 --- a/modules/nf-core/pretextmap/meta.yml +++ b/modules/nf-core/pretextmap/meta.yml @@ -6,38 +6,54 @@ keywords: - map tools: - "pretextmap": - description: "Paired REad TEXTure Mapper. Converts SAM formatted read pairs into genome contact maps." + description: "Paired REad TEXTure Mapper. Converts SAM formatted read pairs into + genome contact maps." homepage: "https://github.com/wtsi-hpag/PretextMap" documentation: "https://github.com/wtsi-hpag/PretextMap/blob/master/README.md" - - licence: "['MIT']" - + licence: ["MIT"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - input: - type: file - description: BAM/CRAM/SAM file or pairs formatted reads file - pattern: "*.{bam,cram,sam,pairs.gz}" - + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - input: + type: file + description: BAM/CRAM/SAM file or pairs formatted reads file + pattern: "*.{bam,cram,sam,pairs.gz}" + - - meta2: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - fasta: + type: file + description: Reference sequence file + pattern: "*.{fasta,fna,fa}" + - fai: + type: file + description: Reference sequence index file + pattern: "*.{fai}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - pretext: - type: file - description: pretext map - pattern: "*.pretext" - + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.pretext": + type: file + description: pretext map + pattern: "*.pretext" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@marrip" - "@getrudeln" +maintainers: + - "@marrip" + - "@getrudeln" diff --git a/modules/nf-core/pretextmap/pretextmap.diff b/modules/nf-core/pretextmap/pretextmap.diff index 2494d6ce..7343eff2 100644 --- a/modules/nf-core/pretextmap/pretextmap.diff +++ b/modules/nf-core/pretextmap/pretextmap.diff @@ -1,64 +1,53 @@ Changes in module 'nf-core/pretextmap' +'modules/nf-core/pretextmap/meta.yml' is unchanged +Changes in 'pretextmap/main.nf': --- modules/nf-core/pretextmap/main.nf +++ modules/nf-core/pretextmap/main.nf -@@ -3,26 +3,24 @@ - tag "$meta.id" +@@ -4,9 +4,7 @@ label 'process_single' -- conda "bioconda::pretextmap=0.1.9 bioconda::samtools=1.17" + conda "${moduleDir}/environment.yml" - container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/mulled-v2-f3591ce8609c7b3b33e5715333200aa5c163aa61%3A44321ab4d64f0b6d0c93abbd1406369d1b3da684-0': - 'biocontainers/mulled-v2-f3591ce8609c7b3b33e5715333200aa5c163aa61:44321ab4d64f0b6d0c93abbd1406369d1b3da684-0' }" -+ container "quay.io/sanger-tol/pretext:0.0.2-yy5-c3" ++ container "quay.io/sanger-tol/pretext:0.0.2-yy5-c4" input: -- tuple val(meta), path(input) -- path fasta -+ tuple val(meta), path(input) -+ tuple val(meta2), path(fasta), path(genome) - - output: - tuple val(meta), path("*.pretext"), emit: pretext -- path "versions.yml" , emit: versions -+ path "versions.yml", emit: versions - - when: + tuple val(meta), path(input) +@@ -20,6 +18,7 @@ task.ext.when == null || task.ext.when script: -- def args = task.ext.args ?: '' -- def prefix = task.ext.prefix ?: "${meta.id}" -- def reference = fasta ? "--reference ${fasta}" : "" -+ def VERSION = "0.1.9" -+ def args = task.ext.args ?: '' -+ def prefix = task.ext.prefix ?: "${meta.id}" -+ def reference = fasta ? "--reference ${fasta}" : "" - - """ - if [[ $input == *.pairs.gz ]]; then -@@ -41,19 +39,20 @@ ++ def VERSION = "0.1.9" + def args = task.ext.args ?: '' + def prefix = task.ext.prefix ?: "${meta.id}" + def reference = fasta ? "--reference ${fasta}" : "" +@@ -41,19 +40,20 @@ cat <<-END_VERSIONS > versions.yml "${task.process}": - pretextmap: \$(PretextMap | grep "Version" | sed 's/PretextMap Version //g') -+ PretextMap: $VERSION ++ pretextmap: $VERSION samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//' ) END_VERSIONS """ stub: -- def prefix = task.ext.prefix ?: "${meta.id}" -+ def VERSION = "0.1.9" -+ def prefix = task.ext.prefix ?: "${meta.id}" ++ def VERSION = "0.1.9" + def prefix = task.ext.prefix ?: "${meta.id}" """ touch ${prefix}.pretext cat <<-END_VERSIONS > versions.yml "${task.process}": - pretextmap: \$(PretextMap | grep "Version" | sed 's/PretextMap Version //g') -+ PretextMap: $VERSION - samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//' )) ++ pretextmap: $VERSION + samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//') END_VERSIONS """ +'modules/nf-core/pretextmap/environment.yml' is unchanged +'modules/nf-core/pretextmap/tests/main.nf.test' is unchanged +'modules/nf-core/pretextmap/tests/tags.yml' is unchanged +'modules/nf-core/pretextmap/tests/main.nf.test.snap' is unchanged ************************************************************ diff --git a/modules/nf-core/pretextmap/tests/main.nf.test b/modules/nf-core/pretextmap/tests/main.nf.test new file mode 100644 index 00000000..cd2c3919 --- /dev/null +++ b/modules/nf-core/pretextmap/tests/main.nf.test @@ -0,0 +1,148 @@ +nextflow_process { + + name "Test Process PRETEXTMAP" + script "../main.nf" + process "PRETEXTMAP" + + tag "modules" + tag "modules_nfcore" + tag "pretext" + tag "pretextmap" + + test("homo_cram_map") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true) + ] + + input[1] = [ + [ id: 'test2', single_end: false ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta.fai', checkIfExists: true) + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("homo_cram_map_nofai") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true) + ] + + input[1] = [ + [ id: 'test2', single_end: false ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true), + [] + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("homo_bam_map") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true) + ] + + input[1] = [ + [ id: 'test2', single_end: false ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta.fai', checkIfExists: true) + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("mapped_pairs_map") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:false ], // meta map + file("https://raw.githubusercontent.com/4dn-dcic/pairix/master/samples/test_4dn.pairs.gz", checkIfExists: true) + ] + + input[1] = [ + [], + [], + [] + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("homo_cram_map_stub") { + options "-stub" + when { + process { + """ + input[0] = [ + [ id:'test', single_end: false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true) + ] + + input[1] = [ + [ id: 'test2', single_end: false ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/chr21/sequence/genome.fasta', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/chr21/sequence/genome.fasta.fai', checkIfExists: true) + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + +} diff --git a/modules/nf-core/pretextmap/tests/main.nf.test.snap b/modules/nf-core/pretextmap/tests/main.nf.test.snap new file mode 100644 index 00000000..48697af8 --- /dev/null +++ b/modules/nf-core/pretextmap/tests/main.nf.test.snap @@ -0,0 +1,157 @@ +{ + "homo_cram_map": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,ae0f18b472e39de970fdc1aebc0f3f8d" + ] + ], + "1": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ], + "pretext": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,ae0f18b472e39de970fdc1aebc0f3f8d" + ] + ], + "versions": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ] + } + ], + "timestamp": "2024-03-21T10:53:43.322902" + }, + "homo_cram_map_stub": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ], + "pretext": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ] + } + ], + "timestamp": "2024-03-21T11:07:44.32508" + }, + "homo_cram_map_nofai": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,ae0f18b472e39de970fdc1aebc0f3f8d" + ] + ], + "1": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ], + "pretext": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,ae0f18b472e39de970fdc1aebc0f3f8d" + ] + ], + "versions": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ] + } + ], + "timestamp": "2024-03-21T10:58:16.804287" + }, + "mapped_pairs_map": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,b47be7fd285a2de68643f73f85ba84f1" + ] + ], + "1": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ], + "pretext": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,b47be7fd285a2de68643f73f85ba84f1" + ] + ], + "versions": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ] + } + ], + "timestamp": "2024-03-21T11:07:32.047007" + }, + "homo_bam_map": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,ae0f18b472e39de970fdc1aebc0f3f8d" + ] + ], + "1": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ], + "pretext": [ + [ + { + "id": "test", + "single_end": false + }, + "test.pretext:md5,ae0f18b472e39de970fdc1aebc0f3f8d" + ] + ], + "versions": [ + "versions.yml:md5,894afb81509c847f7d9a58885e086b5c" + ] + } + ], + "timestamp": "2024-03-21T11:02:50.120033" + } +} \ No newline at end of file diff --git a/modules/nf-core/pretextmap/tests/tags.yml b/modules/nf-core/pretextmap/tests/tags.yml new file mode 100644 index 00000000..70f7eefc --- /dev/null +++ b/modules/nf-core/pretextmap/tests/tags.yml @@ -0,0 +1,2 @@ +pretextmap: + - "modules/nf-core/pretextmap/**" diff --git a/modules/nf-core/pretextsnapshot/environment.yml b/modules/nf-core/pretextsnapshot/environment.yml new file mode 100644 index 00000000..ed721522 --- /dev/null +++ b/modules/nf-core/pretextsnapshot/environment.yml @@ -0,0 +1,5 @@ +channels: + - conda-forge + - bioconda +dependencies: + - bioconda::pretextsnapshot=0.0.4 diff --git a/modules/nf-core/pretextsnapshot/main.nf b/modules/nf-core/pretextsnapshot/main.nf index c896343c..5524785d 100644 --- a/modules/nf-core/pretextsnapshot/main.nf +++ b/modules/nf-core/pretextsnapshot/main.nf @@ -2,7 +2,10 @@ process PRETEXTSNAPSHOT { tag "$meta.id" label 'process_single' - container "quay.io/sanger-tol/pretext:0.0.2-yy5-c3" + conda "${moduleDir}/environment.yml" + container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? + 'https://depot.galaxyproject.org/singularity/pretextsnapshot:0.0.4--h7d875b9_0': + 'biocontainers/pretextsnapshot:0.0.4--h7d875b9_0' }" input: tuple val(meta), path(pretext_map) @@ -15,10 +18,9 @@ process PRETEXTSNAPSHOT { task.ext.when == null || task.ext.when script: - def VERSION = "0.0.4" - def args = task.ext.args ?: '' - def prefix = task.ext.prefix ?: "${meta.id}" - + def VERSION = "0.0.4" + def args = task.ext.args ?: '' + def prefix = task.ext.prefix ?: "${meta.id}." """ PretextSnapshot \\ $args \\ @@ -29,7 +31,7 @@ process PRETEXTSNAPSHOT { cat <<-END_VERSIONS > versions.yml "${task.process}": - PretextSnapshot: $VERSION + pretextsnapshot: $VERSION END_VERSIONS """ @@ -40,7 +42,7 @@ process PRETEXTSNAPSHOT { cat <<-END_VERSIONS > versions.yml "${task.process}": - PretextSnapshot: $VERSION + pretextsnapshot: $VERSION END_VERSIONS """ } diff --git a/modules/nf-core/pretextsnapshot/meta.yml b/modules/nf-core/pretextsnapshot/meta.yml index fe9cb17a..434da68e 100644 --- a/modules/nf-core/pretextsnapshot/meta.yml +++ b/modules/nf-core/pretextsnapshot/meta.yml @@ -13,33 +13,35 @@ tools: description: "Commandline image generator for Pretext Hi-C genome contact maps." homepage: "https://github.com/wtsi-hpag/PretextSnapshot" tool_dev_url: "https://github.com/wtsi-hpag/PretextSnapshot" - licence: "['https://github.com/wtsi-hpag/PretextSnapshot/blob/master/LICENSE']" - + licence: ["https://github.com/wtsi-hpag/PretextSnapshot/blob/master/LICENSE"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - pretext_map: - type: file - description: pretext hic map - pattern: "*.pretext" - + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - pretext_map: + type: file + description: pretext hic map + pattern: "*.pretext" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - image: - type: file - description: image of a hic contact map - pattern: "*.{png,jpg,bmp}" - + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.{jpeg,png,bmp}": + type: file + description: image of a hic contact map + pattern: "*.{png,jpg,bmp}" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@epaule" +maintainers: + - "@epaule" diff --git a/modules/nf-core/pretextsnapshot/pretextsnapshot.diff b/modules/nf-core/pretextsnapshot/pretextsnapshot.diff index caba07ac..4759c3ae 100644 --- a/modules/nf-core/pretextsnapshot/pretextsnapshot.diff +++ b/modules/nf-core/pretextsnapshot/pretextsnapshot.diff @@ -1,33 +1,19 @@ Changes in module 'nf-core/pretextsnapshot' +'modules/nf-core/pretextsnapshot/meta.yml' is unchanged +Changes in 'pretextsnapshot/main.nf': --- modules/nf-core/pretextsnapshot/main.nf +++ modules/nf-core/pretextsnapshot/main.nf -@@ -2,11 +2,9 @@ - tag "$meta.id" - label 'process_single' - -- conda "bioconda::pretextsnapshot=0.0.4" - container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? -- 'https://depot.galaxyproject.org/singularity/pretextsnapshot:0.0.4--h7d875b9_0': -- 'biocontainers/pretextsnapshot:0.0.4--h7d875b9_0' }" -- -+ 'https://depot.galaxyproject.org/singularity/ubuntu:20.04' : -+ 'docker.io/ubuntu:20.04' }" - input: - tuple val(meta), path(pretext_map) - -@@ -18,18 +16,32 @@ +@@ -18,18 +18,31 @@ task.ext.when == null || task.ext.when script: - def args = task.ext.args ?: '' -- def prefix = task.ext.prefix ?: "${meta.id}" -+ def VERSION = "0.0.4" -+ def args = task.ext.args ?: '' -+ def prefix = task.ext.prefix ?: "${meta.id}" -+ def pretext_path = "${projectDir}/bin/PretextSnapshot/bin/PretextSnapshot" +- def prefix = task.ext.prefix ?: "${meta.id}." ++ def VERSION = "0.0.4" ++ def args = task.ext.args ?: '' ++ def prefix = task.ext.prefix ?: "${meta.id}." """ -- PretextSnapshot \\ -+ ${pretext_path} \\ + PretextSnapshot \\ $args \\ + --memory $task.memory \\ --map $pretext_map \\ @@ -53,4 +39,5 @@ Changes in module 'nf-core/pretextsnapshot' """ } +'modules/nf-core/pretextsnapshot/environment.yml' is unchanged ************************************************************ diff --git a/modules/nf-core/samtools/faidx/environment.yml b/modules/nf-core/samtools/faidx/environment.yml index 4807ba5f..62054fc9 100644 --- a/modules/nf-core/samtools/faidx/environment.yml +++ b/modules/nf-core/samtools/faidx/environment.yml @@ -1,8 +1,8 @@ -name: samtools_faidx +--- +# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::samtools=1.18 - - bioconda::htslib=1.18 + - bioconda::htslib=1.21 + - bioconda::samtools=1.21 diff --git a/modules/nf-core/samtools/faidx/main.nf b/modules/nf-core/samtools/faidx/main.nf index d3461627..28c0a81c 100644 --- a/modules/nf-core/samtools/faidx/main.nf +++ b/modules/nf-core/samtools/faidx/main.nf @@ -4,8 +4,8 @@ process SAMTOOLS_FAIDX { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/samtools:1.18--h50ea8bc_1' : - 'biocontainers/samtools:1.18--h50ea8bc_1' }" + 'https://depot.galaxyproject.org/singularity/samtools:1.21--h50ea8bc_0' : + 'biocontainers/samtools:1.21--h50ea8bc_0' }" input: tuple val(meta), path(fasta) diff --git a/modules/nf-core/samtools/faidx/meta.yml b/modules/nf-core/samtools/faidx/meta.yml index e189af28..6721b2cb 100644 --- a/modules/nf-core/samtools/faidx/meta.yml +++ b/modules/nf-core/samtools/faidx/meta.yml @@ -14,43 +14,62 @@ tools: documentation: http://www.htslib.org/doc/samtools.html doi: 10.1093/bioinformatics/btp352 licence: ["MIT"] + identifier: biotools:samtools input: - - meta: - type: map - description: | - Groovy Map containing reference information - e.g. [ id:'test' ] - - fasta: - type: file - description: FASTA file - pattern: "*.{fa,fasta}" - - meta2: - type: map - description: | - Groovy Map containing reference information - e.g. [ id:'test' ] - - fai: - type: file - description: FASTA index file - pattern: "*.{fai}" + - - meta: + type: map + description: | + Groovy Map containing reference information + e.g. [ id:'test' ] + - fasta: + type: file + description: FASTA file + pattern: "*.{fa,fasta}" + - - meta2: + type: map + description: | + Groovy Map containing reference information + e.g. [ id:'test' ] + - fai: + type: file + description: FASTA index file + pattern: "*.{fai}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] + - fa: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.{fa,fasta}": + type: file + description: FASTA file + pattern: "*.{fa}" - fai: - type: file - description: FASTA index file - pattern: "*.{fai}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.fai": + type: file + description: FASTA index file + pattern: "*.{fai}" - gzi: - type: file - description: Optional gzip index file for compressed inputs - pattern: "*.gzi" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.gzi": + type: file + description: Optional gzip index file for compressed inputs + pattern: "*.gzi" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@drpatelh" - "@ewels" diff --git a/modules/nf-core/samtools/faidx/tests/main.nf.test b/modules/nf-core/samtools/faidx/tests/main.nf.test new file mode 100644 index 00000000..17244ef2 --- /dev/null +++ b/modules/nf-core/samtools/faidx/tests/main.nf.test @@ -0,0 +1,122 @@ +nextflow_process { + + name "Test Process SAMTOOLS_FAIDX" + script "../main.nf" + process "SAMTOOLS_FAIDX" + + tag "modules" + tag "modules_nfcore" + tag "samtools" + tag "samtools/faidx" + + test("test_samtools_faidx") { + + when { + process { + """ + input[0] = [ [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) ] + + input[1] = [[],[]] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("test_samtools_faidx_bgzip") { + + when { + process { + """ + input[0] = [ [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta.gz', checkIfExists: true)] + + input[1] = [[],[]] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("test_samtools_faidx_fasta") { + + config "./nextflow.config" + + when { + process { + """ + input[0] = [ [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) ] + + input[1] = [ [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta.fai', checkIfExists: true) ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("test_samtools_faidx_stub_fasta") { + + config "./nextflow2.config" + + when { + process { + """ + input[0] = [ [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) ] + + input[1] = [ [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta.fai', checkIfExists: true) ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("test_samtools_faidx_stub_fai") { + + when { + process { + """ + input[0] = [ [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) ] + + input[1] = [[],[]] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } +} \ No newline at end of file diff --git a/modules/nf-core/samtools/faidx/tests/main.nf.test.snap b/modules/nf-core/samtools/faidx/tests/main.nf.test.snap new file mode 100644 index 00000000..1bbb3ec2 --- /dev/null +++ b/modules/nf-core/samtools/faidx/tests/main.nf.test.snap @@ -0,0 +1,249 @@ +{ + "test_samtools_faidx": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test", + "single_end": false + }, + "genome.fasta.fai:md5,9da2a56e2853dc8c0b86a9e7229c9fe5" + ] + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ], + "fa": [ + + ], + "fai": [ + [ + { + "id": "test", + "single_end": false + }, + "genome.fasta.fai:md5,9da2a56e2853dc8c0b86a9e7229c9fe5" + ] + ], + "gzi": [ + + ], + "versions": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T07:57:47.450887871" + }, + "test_samtools_faidx_bgzip": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test", + "single_end": false + }, + "genome.fasta.gz.fai:md5,9da2a56e2853dc8c0b86a9e7229c9fe5" + ] + ], + "2": [ + [ + { + "id": "test", + "single_end": false + }, + "genome.fasta.gz.gzi:md5,7dea362b3fac8e00956a4952a3d4f474" + ] + ], + "3": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ], + "fa": [ + + ], + "fai": [ + [ + { + "id": "test", + "single_end": false + }, + "genome.fasta.gz.fai:md5,9da2a56e2853dc8c0b86a9e7229c9fe5" + ] + ], + "gzi": [ + [ + { + "id": "test", + "single_end": false + }, + "genome.fasta.gz.gzi:md5,7dea362b3fac8e00956a4952a3d4f474" + ] + ], + "versions": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T07:58:04.804905659" + }, + "test_samtools_faidx_fasta": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "extract.fa:md5,6a0774a0ad937ba0bfd2ac7457d90f36" + ] + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ], + "fa": [ + [ + { + "id": "test", + "single_end": false + }, + "extract.fa:md5,6a0774a0ad937ba0bfd2ac7457d90f36" + ] + ], + "fai": [ + + ], + "gzi": [ + + ], + "versions": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T07:58:23.831268154" + }, + "test_samtools_faidx_stub_fasta": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "extract.fa:md5,9da2a56e2853dc8c0b86a9e7229c9fe5" + ] + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ], + "fa": [ + [ + { + "id": "test", + "single_end": false + }, + "extract.fa:md5,9da2a56e2853dc8c0b86a9e7229c9fe5" + ] + ], + "fai": [ + + ], + "gzi": [ + + ], + "versions": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T07:58:35.600243706" + }, + "test_samtools_faidx_stub_fai": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test", + "single_end": false + }, + "genome.fasta.fai:md5,9da2a56e2853dc8c0b86a9e7229c9fe5" + ] + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ], + "fa": [ + + ], + "fai": [ + [ + { + "id": "test", + "single_end": false + }, + "genome.fasta.fai:md5,9da2a56e2853dc8c0b86a9e7229c9fe5" + ] + ], + "gzi": [ + + ], + "versions": [ + "versions.yml:md5,6bbe80a2e14bd61202ca63e12d66027f" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T07:58:54.705460167" + } +} \ No newline at end of file diff --git a/modules/nf-core/samtools/faidx/tests/nextflow.config b/modules/nf-core/samtools/faidx/tests/nextflow.config new file mode 100644 index 00000000..f76a3ba0 --- /dev/null +++ b/modules/nf-core/samtools/faidx/tests/nextflow.config @@ -0,0 +1,7 @@ +process { + + withName: SAMTOOLS_FAIDX { + ext.args = 'MT192765.1 -o extract.fa' + } + +} diff --git a/modules/nf-core/samtools/faidx/tests/nextflow2.config b/modules/nf-core/samtools/faidx/tests/nextflow2.config new file mode 100644 index 00000000..33ebbd5d --- /dev/null +++ b/modules/nf-core/samtools/faidx/tests/nextflow2.config @@ -0,0 +1,6 @@ +process { + + withName: SAMTOOLS_FAIDX { + ext.args = '-o extract.fa' + } +} diff --git a/modules/nf-core/samtools/faidx/tests/tags.yml b/modules/nf-core/samtools/faidx/tests/tags.yml new file mode 100644 index 00000000..e4a83948 --- /dev/null +++ b/modules/nf-core/samtools/faidx/tests/tags.yml @@ -0,0 +1,2 @@ +samtools/faidx: + - modules/nf-core/samtools/faidx/** diff --git a/modules/nf-core/samtools/index/environment.yml b/modules/nf-core/samtools/index/environment.yml index 296ed99e..62054fc9 100644 --- a/modules/nf-core/samtools/index/environment.yml +++ b/modules/nf-core/samtools/index/environment.yml @@ -1,7 +1,8 @@ -name: samtools_index +--- +# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::samtools=1.18 + - bioconda::htslib=1.21 + - bioconda::samtools=1.21 diff --git a/modules/nf-core/samtools/index/main.nf b/modules/nf-core/samtools/index/main.nf index 8ad18fdc..31175610 100644 --- a/modules/nf-core/samtools/index/main.nf +++ b/modules/nf-core/samtools/index/main.nf @@ -4,8 +4,8 @@ process SAMTOOLS_INDEX { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/samtools:1.18--h50ea8bc_1' : - 'biocontainers/samtools:1.18--h50ea8bc_1' }" + 'https://depot.galaxyproject.org/singularity/samtools:1.21--h50ea8bc_0' : + 'biocontainers/samtools:1.21--h50ea8bc_0' }" input: tuple val(meta), path(input) @@ -35,10 +35,11 @@ process SAMTOOLS_INDEX { """ stub: + def args = task.ext.args ?: '' + def extension = file(input).getExtension() == 'cram' ? + "crai" : args.contains("-c") ? "csi" : "bai" """ - touch ${input}.bai - touch ${input}.crai - touch ${input}.csi + touch ${input}.${extension} cat <<-END_VERSIONS > versions.yml "${task.process}": diff --git a/modules/nf-core/samtools/index/meta.yml b/modules/nf-core/samtools/index/meta.yml index 01a4ee03..db8df0d5 100644 --- a/modules/nf-core/samtools/index/meta.yml +++ b/modules/nf-core/samtools/index/meta.yml @@ -15,38 +15,52 @@ tools: documentation: http://www.htslib.org/doc/samtools.html doi: 10.1093/bioinformatics/btp352 licence: ["MIT"] + identifier: biotools:samtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - bam: - type: file - description: BAM/CRAM/SAM file - pattern: "*.{bam,cram,sam}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - input: + type: file + description: input file output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - bai: - type: file - description: BAM/CRAM/SAM index file - pattern: "*.{bai,crai,sai}" - - crai: - type: file - description: BAM/CRAM/SAM index file - pattern: "*.{bai,crai,sai}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bai": + type: file + description: BAM/CRAM/SAM index file + pattern: "*.{bai,crai,sai}" - csi: - type: file - description: CSI index file - pattern: "*.{csi}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.csi": + type: file + description: CSI index file + pattern: "*.{csi}" + - crai: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.crai": + type: file + description: BAM/CRAM/SAM index file + pattern: "*.{bai,crai,sai}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@drpatelh" - "@ewels" diff --git a/modules/nf-core/samtools/index/tests/main.nf.test b/modules/nf-core/samtools/index/tests/main.nf.test index c76a9169..ca34fb5c 100644 --- a/modules/nf-core/samtools/index/tests/main.nf.test +++ b/modules/nf-core/samtools/index/tests/main.nf.test @@ -8,18 +8,34 @@ nextflow_process { tag "samtools" tag "samtools/index" - test("sarscov2 [BAI]") { - + test("bai") { when { - params { - outdir = "$outputDir" + process { + """ + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true) + ]) + """ } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("crai") { + when { process { """ - input[0] = [ - [ id:'test' ], // meta map - file(params.test_data['sarscov2']['illumina']['test_paired_end_sorted_bam'], checkIfExists: true) - ] + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.recalibrated.sorted.cram', checkIfExists: true) + ]) """ } } @@ -27,24 +43,45 @@ nextflow_process { then { assertAll ( { assert process.success }, - { assert snapshot(process.out.bai).match("bai") }, - { assert path(process.out.versions.get(0)).getText().contains("samtools") } + { assert snapshot(process.out).match() } ) } } - test("homo_sapiens [CRAI]") { + test("csi") { + config "./csi.nextflow.config" when { - params { - outdir = "$outputDir" + process { + """ + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true) + ]) + """ } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot( + file(process.out.csi[0][1]).name, + process.out.versions + ).match() } + ) + } + } + + test("bai - stub") { + options "-stub" + when { process { """ - input[0] = [ - [ id:'test' ], // meta map - file(params.test_data['homo_sapiens']['illumina']['test_paired_end_recalibrated_sorted_cram'], checkIfExists: true) - ] + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true) + ]) """ } } @@ -52,26 +89,43 @@ nextflow_process { then { assertAll ( { assert process.success }, - { assert snapshot(process.out.crai).match("crai") }, - { assert path(process.out.versions.get(0)).getText().contains("samtools") } + { assert snapshot(process.out).match() } ) } } - test("homo_sapiens [CSI]") { + test("crai - stub") { + options "-stub" + when { + process { + """ + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.recalibrated.sorted.cram', checkIfExists: true) + ]) + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + test("csi - stub") { + options "-stub" config "./csi.nextflow.config" when { - params { - outdir = "$outputDir" - } process { """ - input[0] = [ - [ id:'test' ], // meta map - file(params.test_data['sarscov2']['illumina']['test_paired_end_sorted_bam'], checkIfExists: true) - ] + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true) + ]) """ } } @@ -79,8 +133,7 @@ nextflow_process { then { assertAll ( { assert process.success }, - { assert path(process.out.csi.get(0).get(1)).exists() }, - { assert path(process.out.versions.get(0)).getText().contains("samtools") } + { assert snapshot(process.out).match() } ) } } diff --git a/modules/nf-core/samtools/index/tests/main.nf.test.snap b/modules/nf-core/samtools/index/tests/main.nf.test.snap index b3baee7f..72d65e81 100644 --- a/modules/nf-core/samtools/index/tests/main.nf.test.snap +++ b/modules/nf-core/samtools/index/tests/main.nf.test.snap @@ -1,28 +1,250 @@ { - "crai": { + "csi - stub": { "content": [ - [ - [ - { - "id": "test" - }, - "test.paired_end.recalibrated.sorted.cram.crai:md5,14bc3bd5c89cacc8f4541f9062429029" + { + "0": [ + + ], + "1": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.sorted.bam.csi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" + ], + "bai": [ + + ], + "crai": [ + + ], + "csi": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.sorted.bam.csi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:21:25.261127166" + }, + "crai - stub": { + "content": [ + { + "0": [ + + ], + "1": [ + + ], + "2": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.recalibrated.sorted.cram.crai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "3": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" + ], + "bai": [ + + ], + "crai": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.recalibrated.sorted.cram.crai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "csi": [ + + ], + "versions": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:21:12.653194876" + }, + "bai - stub": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.sorted.bam.bai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" + ], + "bai": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.sorted.bam.bai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "crai": [ + + ], + "csi": [ + + ], + "versions": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:21:01.854932651" + }, + "csi": { + "content": [ + "test.paired_end.sorted.bam.csi", + [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" ] ], - "timestamp": "2023-11-15T15:17:37.30801" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:20:51.485364222" + }, + "crai": { + "content": [ + { + "0": [ + + ], + "1": [ + + ], + "2": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.recalibrated.sorted.cram.crai:md5,14bc3bd5c89cacc8f4541f9062429029" + ] + ], + "3": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" + ], + "bai": [ + + ], + "crai": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.recalibrated.sorted.cram.crai:md5,14bc3bd5c89cacc8f4541f9062429029" + ] + ], + "csi": [ + + ], + "versions": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:20:40.518873972" }, "bai": { "content": [ - [ - [ - { - "id": "test" - }, - "test.paired_end.sorted.bam.bai:md5,704c10dd1326482448ca3073fdebc2f4" + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.sorted.bam.bai:md5,704c10dd1326482448ca3073fdebc2f4" + ] + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" + ], + "bai": [ + [ + { + "id": "test", + "single_end": false + }, + "test.paired_end.sorted.bam.bai:md5,704c10dd1326482448ca3073fdebc2f4" + ] + ], + "crai": [ + + ], + "csi": [ + + ], + "versions": [ + "versions.yml:md5,5e09a6fdf76de396728f877193d72315" ] - ] + } ], - "timestamp": "2023-11-15T15:17:30.869234" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:20:21.184050361" } } \ No newline at end of file diff --git a/modules/nf-core/samtools/markdup/environment.yml b/modules/nf-core/samtools/markdup/environment.yml index feb82f5f..62054fc9 100644 --- a/modules/nf-core/samtools/markdup/environment.yml +++ b/modules/nf-core/samtools/markdup/environment.yml @@ -1,8 +1,8 @@ -name: samtools_markdup +--- +# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::samtools=1.18 - - bioconda::htslib=1.18 + - bioconda::htslib=1.21 + - bioconda::samtools=1.21 diff --git a/modules/nf-core/samtools/markdup/main.nf b/modules/nf-core/samtools/markdup/main.nf index 803a6bfb..8453bf82 100644 --- a/modules/nf-core/samtools/markdup/main.nf +++ b/modules/nf-core/samtools/markdup/main.nf @@ -4,12 +4,12 @@ process SAMTOOLS_MARKDUP { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/samtools:1.18--h50ea8bc_1' : - 'biocontainers/samtools:1.18--h50ea8bc_1' }" + 'https://depot.galaxyproject.org/singularity/samtools:1.21--h50ea8bc_0' : + 'biocontainers/samtools:1.21--h50ea8bc_0' }" input: tuple val(meta), path(input) - path fasta + tuple val(meta2), path(fasta) output: tuple val(meta), path("*.bam"), emit: bam, optional: true diff --git a/modules/nf-core/samtools/markdup/meta.yml b/modules/nf-core/samtools/markdup/meta.yml index 5e8c9733..31cb0c63 100644 --- a/modules/nf-core/samtools/markdup/meta.yml +++ b/modules/nf-core/samtools/markdup/meta.yml @@ -13,30 +13,62 @@ tools: tool_dev_url: "https://github.com/samtools/samtools" doi: "10.1093/bioinformatics/btp352" licence: ["MIT"] + identifier: biotools:samtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - input: - type: file - description: BAM/CRAM/SAM file - pattern: "*.{bam,cram,sam}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - input: + type: file + description: BAM/CRAM/SAM file + pattern: "*.{bam,cram,sam}" + - - meta2: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - fasta: + type: file + description: A reference genome in FASTA format + pattern: "*.fasta" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] + - bam: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bam": + type: file + description: BAM file + pattern: "*{.bam}" + - cram: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.cram": + type: file + description: CRAM file + pattern: "*{.cram}" + - sam: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.sam": + type: file + description: SAM file + pattern: "*{.sam}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - - output: - type: file - description: Sorted BAM/CRAM/SAM file - pattern: "*.{bam,cram,sam}" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@priyanka-surana" maintainers: diff --git a/modules/nf-core/samtools/markdup/tests/main.nf.test b/modules/nf-core/samtools/markdup/tests/main.nf.test new file mode 100644 index 00000000..acfbd1c1 --- /dev/null +++ b/modules/nf-core/samtools/markdup/tests/main.nf.test @@ -0,0 +1,90 @@ +nextflow_process { + + name "Test Process SAMTOOLS_MARKDUP" + script "../main.nf" + process "SAMTOOLS_MARKDUP" + config "./nextflow.config" + + tag "modules" + tag "modules_nfcore" + tag "samtools" + tag "samtools/fixmate" + tag "samtools/sort" + tag "samtools/markdup" + + test("sarscov2 - bam") { + setup { + run("SAMTOOLS_FIXMATE") { + script "../../fixmate/main.nf" + process { + """ + input[0] = [ + [ id:'test', single_end:false ], + file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true), + ] + """ + } + } + run("SAMTOOLS_SORT") { + script "../../sort/main.nf" + process { + """ + input[0] = SAMTOOLS_FIXMATE.out.bam + input[1] = [ + [ id:'fasta' ], + file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true), + ] + """ + } + } + } + when { + process { + """ + input[0] = SAMTOOLS_SORT.out.bam + input[1] = [ + [ id:'fasta' ], + file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true), + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + + test("sarscov2 - bam - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:false ], + file(params.test_data['sarscov2']['illumina']['test_paired_end_sorted_bam'], checkIfExists: true), + ] + input[1] = [ + [ id:'fasta' ], + file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true), + ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + +} diff --git a/modules/nf-core/samtools/markdup/tests/main.nf.test.snap b/modules/nf-core/samtools/markdup/tests/main.nf.test.snap new file mode 100644 index 00000000..4e7cbf5e --- /dev/null +++ b/modules/nf-core/samtools/markdup/tests/main.nf.test.snap @@ -0,0 +1,96 @@ +{ + "sarscov2 - bam - stub": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.markdup.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,4ce5394244e549f97b8250b7dfad71ba" + ], + "bam": [ + [ + { + "id": "test", + "single_end": false + }, + "test.markdup.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "cram": [ + + ], + "sam": [ + + ], + "versions": [ + "versions.yml:md5,4ce5394244e549f97b8250b7dfad71ba" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-17T08:46:57.510270753" + }, + "sarscov2 - bam": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.markdup.bam:md5,e73ea77e3857e7c6b0e6db2ec660361f" + ] + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + "versions.yml:md5,4ce5394244e549f97b8250b7dfad71ba" + ], + "bam": [ + [ + { + "id": "test", + "single_end": false + }, + "test.markdup.bam:md5,e73ea77e3857e7c6b0e6db2ec660361f" + ] + ], + "cram": [ + + ], + "sam": [ + + ], + "versions": [ + "versions.yml:md5,4ce5394244e549f97b8250b7dfad71ba" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-17T08:51:17.068686604" + } +} \ No newline at end of file diff --git a/modules/nf-core/samtools/markdup/tests/nextflow.config b/modules/nf-core/samtools/markdup/tests/nextflow.config new file mode 100644 index 00000000..1a2b52b9 --- /dev/null +++ b/modules/nf-core/samtools/markdup/tests/nextflow.config @@ -0,0 +1,14 @@ +process { + withName: SAMTOOLS_COLLATE { + ext.prefix = { "${meta.id}.collate" } + } + withName: SAMTOOLS_FIXMATE { + ext.prefix = { "${meta.id}.fixmate" } + } + withName: SAMTOOLS_SORT { + ext.prefix = { "${meta.id}.sorted" } + } + withName: SAMTOOLS_MARKDUP { + ext.prefix = { "${meta.id}.markdup" } + } +} diff --git a/modules/nf-core/samtools/merge/environment.yml b/modules/nf-core/samtools/merge/environment.yml index 14f7e9aa..62054fc9 100644 --- a/modules/nf-core/samtools/merge/environment.yml +++ b/modules/nf-core/samtools/merge/environment.yml @@ -1,8 +1,8 @@ -name: samtools_merge +--- +# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::samtools=1.18 - - bioconda::htslib=1.18 + - bioconda::htslib=1.21 + - bioconda::samtools=1.21 diff --git a/modules/nf-core/samtools/merge/main.nf b/modules/nf-core/samtools/merge/main.nf index e104b90a..34da4c7c 100644 --- a/modules/nf-core/samtools/merge/main.nf +++ b/modules/nf-core/samtools/merge/main.nf @@ -4,8 +4,8 @@ process SAMTOOLS_MERGE { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/samtools:1.18--h50ea8bc_1' : - 'biocontainers/samtools:1.18--h50ea8bc_1' }" + 'https://depot.galaxyproject.org/singularity/samtools:1.21--h50ea8bc_0' : + 'biocontainers/samtools:1.21--h50ea8bc_0' }" input: tuple val(meta), path(input_files, stageAs: "?/*") diff --git a/modules/nf-core/samtools/merge/meta.yml b/modules/nf-core/samtools/merge/meta.yml index 2e8f3dbb..235aa219 100644 --- a/modules/nf-core/samtools/merge/meta.yml +++ b/modules/nf-core/samtools/merge/meta.yml @@ -15,60 +15,81 @@ tools: documentation: http://www.htslib.org/doc/samtools.html doi: 10.1093/bioinformatics/btp352 licence: ["MIT"] + identifier: biotools:samtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - input_files: - type: file - description: BAM/CRAM file - pattern: "*.{bam,cram,sam}" - - meta2: - type: map - description: | - Groovy Map containing reference information - e.g. [ id:'genome' ] - - fasta: - type: file - description: Reference file the CRAM was created with (optional) - pattern: "*.{fasta,fa}" - - meta3: - type: map - description: | - Groovy Map containing reference information - e.g. [ id:'genome' ] - - fai: - type: file - description: Index of the reference file the CRAM was created with (optional) - pattern: "*.fai" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - input_files: + type: file + description: BAM/CRAM file + pattern: "*.{bam,cram,sam}" + - - meta2: + type: map + description: | + Groovy Map containing reference information + e.g. [ id:'genome' ] + - fasta: + type: file + description: Reference file the CRAM was created with (optional) + pattern: "*.{fasta,fa}" + - - meta3: + type: map + description: | + Groovy Map containing reference information + e.g. [ id:'genome' ] + - fai: + type: file + description: Index of the reference file the CRAM was created with (optional) + pattern: "*.fai" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - bam: - type: file - description: BAM file - pattern: "*.{bam}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.bam: + type: file + description: BAM file + pattern: "*.{bam}" - cram: - type: file - description: CRAM file - pattern: "*.{cram}" - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.cram: + type: file + description: CRAM file + pattern: "*.{cram}" - csi: - type: file - description: BAM index file (optional) - pattern: "*.csi" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.csi": + type: file + description: BAM index file (optional) + pattern: "*.csi" - crai: - type: file - description: CRAM index file (optional) - pattern: "*.crai" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.crai": + type: file + description: CRAM index file (optional) + pattern: "*.crai" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@drpatelh" - "@yuukiiwa " diff --git a/modules/nf-core/samtools/merge/tests/main.nf.test b/modules/nf-core/samtools/merge/tests/main.nf.test index 024f9f70..40b36e82 100644 --- a/modules/nf-core/samtools/merge/tests/main.nf.test +++ b/modules/nf-core/samtools/merge/tests/main.nf.test @@ -9,21 +9,19 @@ nextflow_process { tag "samtools" tag "samtools/merge" - test("sarscov2 - [bam1, bam2, bam3], [], []") { + test("bams") { config "./index.config" when { process { """ - input[0] = [ - [ id: 'test' ], // meta map - [ - file(params.test_data['sarscov2']['illumina']['test_paired_end_methylated_sorted_bam'], checkIfExists: true), - file(params.test_data['sarscov2']['illumina']['test_paired_end_sorted_bam'], checkIfExists: true), - file(params.test_data['sarscov2']['illumina']['test_single_end_sorted_bam'], checkIfExists: true) - ] - ] + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.methylated.sorted.bam', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.single_end.sorted.bam', checkIfExists: true) ] + ]) input[1] = [[],[]] input[2] = [[],[]] """ @@ -33,40 +31,35 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.bam[0][1]).name, - process.out.cram, - file(process.out.csi[0][1]).name, - process.out.crai, - process.out.versions - ).match() } + { assert snapshot(file(process.out.bam[0][1]).name).match("bams_bam") }, + { assert snapshot(process.out.cram).match("bams_cram") }, + { assert snapshot(file(process.out.csi[0][1]).name).match("bams_csi") }, + { assert snapshot(process.out.crai).match("bams_crai") }, + { assert snapshot(process.out.versions).match("bams_versions") } ) } - } - test("homo_sapiens - [cram1, cram2], fasta, fai") { + test("crams") { config "./index.config" when { process { """ - input[0] = [ - [ id: 'test' ], // meta map - [ - file(params.test_data['homo_sapiens']['illumina']['test_paired_end_recalibrated_sorted_cram'], checkIfExists: true), - file(params.test_data['homo_sapiens']['illumina']['test2_paired_end_recalibrated_sorted_cram'], checkIfExists: true), - ] - ] - input[1] = [ - [id:'genome'], - file(params.test_data['homo_sapiens']['genome']['genome_fasta'], checkIfExists: true) - ] - input[2] = [ - [id:'genome'], - file(params.test_data['homo_sapiens']['genome']['genome_fasta_fai'], checkIfExists: true) - ] + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + [ file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.recalibrated.sorted.cram', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test2.paired_end.recalibrated.sorted.cram', checkIfExists: true) ] + ]) + input[1] = Channel.of([ + [ id:'genome' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) + input[2] = Channel.of([ + [ id:'genome' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta.fai', checkIfExists: true) + ]) """ } } @@ -74,27 +67,24 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.cram[0][1]).name, - process.out.bam, - file(process.out.crai[0][1]).name, - process.out.csi, - process.out.versions - ).match() } + { assert snapshot(file(process.out.cram[0][1]).name).match("crams_cram") }, + { assert snapshot(process.out.bam).match("crams_bam") }, + { assert snapshot(file(process.out.crai[0][1]).name).match("crams_crai") }, + { assert snapshot(process.out.csi).match("crams_csi") }, + { assert snapshot(process.out.versions).match("crams_versions") } ) } - } - test("sarscov2 - bam, [], []") { + test("bam") { when { process { """ - input[0] = [ - [ id: 'test' ], // meta map - file(params.test_data['sarscov2']['illumina']['test_paired_end_methylated_sorted_bam'], checkIfExists: true), - ] + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.methylated.sorted.bam', checkIfExists: true) ] + ]) input[1] = [[],[]] input[2] = [[],[]] """ @@ -104,19 +94,16 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.bam[0][1]).name, - process.out.cram, - process.out.crai, - process.out.csi, - process.out.versions - ).match() } + { assert snapshot(file(process.out.bam[0][1]).name).match("bam_bam") }, + { assert snapshot(process.out.cram).match("bam_cram") }, + { assert snapshot(process.out.crai).match("bam_crai") }, + { assert snapshot(process.out.csi).match("bam_csi") }, + { assert snapshot(process.out.versions).match("bam_versions") } ) } - } - test("sarscov2 - [bam1, bam2, bam3], [], [] - stub") { + test("bams_stub") { config "./index.config" options "-stub" @@ -124,14 +111,12 @@ nextflow_process { when { process { """ - input[0] = [ - [ id: 'test' ], // meta map - [ - file(params.test_data['sarscov2']['illumina']['test_paired_end_methylated_sorted_bam'], checkIfExists: true), - file(params.test_data['sarscov2']['illumina']['test_paired_end_sorted_bam'], checkIfExists: true), - file(params.test_data['sarscov2']['illumina']['test_single_end_sorted_bam'], checkIfExists: true) - ] - ] + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.methylated.sorted.bam', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.single_end.sorted.bam', checkIfExists: true) ] + ]) input[1] = [[],[]] input[2] = [[],[]] """ @@ -141,16 +126,12 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.bam[0][1]).name, - process.out.cram, - file(process.out.csi[0][1]).name, - process.out.crai, - process.out.versions - ).match() } + { assert snapshot(file(process.out.bam[0][1]).name).match("bams_stub_bam") }, + { assert snapshot(process.out.cram).match("bams_stub_cram") }, + { assert snapshot(file(process.out.csi[0][1]).name).match("bams_stub_csi") }, + { assert snapshot(process.out.crai).match("bams_stub_crai") }, + { assert snapshot(process.out.versions).match("bams_stub_versions") } ) } - } - } diff --git a/modules/nf-core/samtools/merge/tests/main.nf.test.snap b/modules/nf-core/samtools/merge/tests/main.nf.test.snap index 3ab57d81..0a41e01a 100644 --- a/modules/nf-core/samtools/merge/tests/main.nf.test.snap +++ b/modules/nf-core/samtools/merge/tests/main.nf.test.snap @@ -1,68 +1,228 @@ { - "sarscov2 - bam, [], []": { + "crams_cram": { + "content": [ + "test.cram" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:00.647389" + }, + "bams_stub_cram": { "content": [ - "test.bam", [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:19.937013" + }, + "bams_crai": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:49:24.928616" + }, + "bams_bam": { + "content": [ + "test.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:49:24.923289" + }, + "bams_cram": { + "content": [ [ - ], - [ - "versions.yml:md5,71986103374bdddb2e3093d20e7d06cb" ] ], - "timestamp": "2023-12-04T17:13:30.244841621" + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:49:24.925716" }, - "sarscov2 - [bam1, bam2, bam3], [], [] - stub": { + "crams_csi": { "content": [ - "test.bam", [ - ], - "test.csi", + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:00.655959" + }, + "bam_bam": { + "content": [ + "test.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:10.319539" + }, + "bam_versions": { + "content": [ + [ + "versions.yml:md5,d51d18a97513e370e43f0c891c51dfc4" + ] + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T09:16:30.476887194" + }, + "bams_csi": { + "content": [ + "test.bam.csi" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:49:24.92719" + }, + "bams_stub_csi": { + "content": [ + "test.csi" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:19.940498" + }, + "bam_crai": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:10.328852" + }, + "bams_stub_versions": { + "content": [ [ - "versions.yml:md5,71986103374bdddb2e3093d20e7d06cb" + "versions.yml:md5,d51d18a97513e370e43f0c891c51dfc4" ] ], - "timestamp": "2023-12-04T17:10:14.861445721" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T09:16:52.203823961" }, - "homo_sapiens - [cram1, cram2], fasta, fai": { + "bam_cram": { "content": [ - "test.cram", [ - ], - "test.cram.crai", + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:10.324219" + }, + "bams_stub_bam": { + "content": [ + "test.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:19.933153" + }, + "bams_versions": { + "content": [ + [ + "versions.yml:md5,d51d18a97513e370e43f0c891c51dfc4" + ] + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:29:57.524363148" + }, + "crams_bam": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:00.650652" + }, + "crams_versions": { + "content": [ [ - "versions.yml:md5,71986103374bdddb2e3093d20e7d06cb" + "versions.yml:md5,d51d18a97513e370e43f0c891c51dfc4" ] ], - "timestamp": "2023-12-04T17:09:29.716002618" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T09:16:06.977096207" }, - "sarscov2 - [bam1, bam2, bam3], [], []": { + "bam_csi": { "content": [ - "test.bam", [ - ], - "test.bam.csi", + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:10.33292" + }, + "crams_crai": { + "content": [ + "test.cram.crai" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:00.653512" + }, + "bams_stub_crai": { + "content": [ [ - ], - [ - "versions.yml:md5,71986103374bdddb2e3093d20e7d06cb" ] ], - "timestamp": "2023-12-04T17:08:42.329973045" + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T18:50:19.943839" } } \ No newline at end of file diff --git a/modules/nf-core/samtools/sort/environment.yml b/modules/nf-core/samtools/sort/environment.yml index f4064b72..62054fc9 100644 --- a/modules/nf-core/samtools/sort/environment.yml +++ b/modules/nf-core/samtools/sort/environment.yml @@ -1,8 +1,8 @@ -name: samtools_sort +--- +# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::samtools=1.18 - - bioconda::htslib=1.18 + - bioconda::htslib=1.21 + - bioconda::samtools=1.21 diff --git a/modules/nf-core/samtools/sort/main.nf b/modules/nf-core/samtools/sort/main.nf index 4a666d42..caf3c61a 100644 --- a/modules/nf-core/samtools/sort/main.nf +++ b/modules/nf-core/samtools/sort/main.nf @@ -4,16 +4,19 @@ process SAMTOOLS_SORT { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/samtools:1.18--h50ea8bc_1' : - 'biocontainers/samtools:1.18--h50ea8bc_1' }" + 'https://depot.galaxyproject.org/singularity/samtools:1.21--h50ea8bc_0' : + 'biocontainers/samtools:1.21--h50ea8bc_0' }" input: - tuple val(meta), path(bam) + tuple val(meta) , path(bam) + tuple val(meta2), path(fasta) output: - tuple val(meta), path("*.bam"), emit: bam - tuple val(meta), path("*.csi"), emit: csi, optional: true - path "versions.yml" , emit: versions + tuple val(meta), path("*.bam"), emit: bam, optional: true + tuple val(meta), path("*.cram"), emit: cram, optional: true + tuple val(meta), path("*.crai"), emit: crai, optional: true + tuple val(meta), path("*.csi"), emit: csi, optional: true + path "versions.yml", emit: versions when: task.ext.when == null || task.ext.when @@ -21,14 +24,23 @@ process SAMTOOLS_SORT { script: def args = task.ext.args ?: '' def prefix = task.ext.prefix ?: "${meta.id}" + def extension = args.contains("--output-fmt sam") ? "sam" : + args.contains("--output-fmt cram") ? "cram" : + "bam" + def reference = fasta ? "--reference ${fasta}" : "" if ("$bam" == "${prefix}.bam") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!" + """ + samtools cat \\ + ${bam} \\ + | \\ samtools sort \\ $args \\ - -@ $task.cpus \\ - -o ${prefix}.bam \\ - -T $prefix \\ - $bam + -T ${prefix} \\ + --threads $task.cpus \\ + ${reference} \\ + -o ${prefix}.${extension} \\ + - cat <<-END_VERSIONS > versions.yml "${task.process}": @@ -37,9 +49,20 @@ process SAMTOOLS_SORT { """ stub: + def args = task.ext.args ?: '' def prefix = task.ext.prefix ?: "${meta.id}" + def extension = args.contains("--output-fmt sam") ? "sam" : + args.contains("--output-fmt cram") ? "cram" : + "bam" """ - touch ${prefix}.bam + touch ${prefix}.${extension} + if [ "${extension}" == "bam" ]; + then + touch ${prefix}.${extension}.csi + elif [ "${extension}" == "cram" ]; + then + touch ${prefix}.${extension}.crai + fi cat <<-END_VERSIONS > versions.yml "${task.process}": diff --git a/modules/nf-core/samtools/sort/meta.yml b/modules/nf-core/samtools/sort/meta.yml index 2200de72..a9dbec5a 100644 --- a/modules/nf-core/samtools/sort/meta.yml +++ b/modules/nf-core/samtools/sort/meta.yml @@ -15,37 +15,78 @@ tools: documentation: http://www.htslib.org/doc/samtools.html doi: 10.1093/bioinformatics/btp352 licence: ["MIT"] + identifier: biotools:samtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - bam: - type: file - description: BAM/CRAM/SAM file - pattern: "*.{bam,cram,sam}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - bam: + type: file + description: BAM/CRAM/SAM file(s) + pattern: "*.{bam,cram,sam}" + - - meta2: + type: map + description: | + Groovy Map containing reference information + e.g. [ id:'genome' ] + - fasta: + type: file + description: Reference genome FASTA file + pattern: "*.{fa,fasta,fna}" + optional: true output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - bam: - type: file - description: Sorted BAM/CRAM/SAM file - pattern: "*.{bam,cram,sam}" - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bam": + type: file + description: Sorted BAM file + pattern: "*.{bam}" + - cram: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.cram": + type: file + description: Sorted CRAM file + pattern: "*.{cram}" + - crai: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.crai": + type: file + description: CRAM index file (optional) + pattern: "*.crai" - csi: - type: file - description: BAM index file (optional) - pattern: "*.csi" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.csi": + type: file + description: BAM index file (optional) + pattern: "*.csi" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@drpatelh" - "@ewels" + - "@matthdsm" maintainers: - "@drpatelh" - "@ewels" + - "@matthdsm" diff --git a/modules/nf-core/samtools/sort/tests/main.nf.test b/modules/nf-core/samtools/sort/tests/main.nf.test index abb80978..b05e6691 100644 --- a/modules/nf-core/samtools/sort/tests/main.nf.test +++ b/modules/nf-core/samtools/sort/tests/main.nf.test @@ -8,22 +8,55 @@ nextflow_process { tag "samtools" tag "samtools/sort" - test("test_samtools_sort") { + test("bam") { config "./nextflow.config" when { - params { - outdir = "$outputDir" + process { + """ + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.bam', checkIfExists: true) + ]) + input[1] = Channel.of([ + [ id:'fasta' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ]) + """ } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot( + process.out.bam, + process.out.csi.collect { it.collect { it instanceof Map ? it : file(it).name } }, + process.out.versions + ).match()} + ) + } + } + + test("multiple bam") { + + config "./nextflow.config" + + when { process { """ - input[0] = [ - [ id:'test', single_end:false ], + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map [ - file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test2.paired_end.sorted.bam', checkIfExists: true) ] - ] + ]) + input[1] = Channel.of([ + [ id:'fasta' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) """ } } @@ -31,29 +64,92 @@ nextflow_process { then { assertAll ( { assert process.success }, - { assert snapshot(process.out).match() } + { assert snapshot( + process.out.bam, + process.out.csi.collect { it.collect { it instanceof Map ? it : file(it).name } }, + process.out.versions + ).match()} ) } + } + + test("cram") { + + config "./nextflow_cram.config" + + when { + process { + """ + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true) + ]) + input[1] = Channel.of([ + [ id:'fasta' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) + """ + } + } + then { + assertAll ( + { assert process.success }, + { assert snapshot( + process.out.cram.collect { it.collect { it instanceof Map ? it : file(it).name } }, + process.out.crai.collect { it.collect { it instanceof Map ? it : file(it).name } }, + process.out.versions + ).match()} + ) + } } - test("test_samtools_sort_stub") { + test("bam - stub") { + options "-stub" config "./nextflow.config" - options "-stub-run" when { - params { - outdir = "$outputDir" + process { + """ + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.bam', checkIfExists: true) + ]) + input[1] = Channel.of([ + [ id:'fasta' ], // meta map + file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true) + ]) + """ } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + + test("multiple bam - stub") { + + config "./nextflow.config" + + when { process { """ - input[0] = [ - [ id:'test', single_end:false ], + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map [ - file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true) + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/test2.paired_end.sorted.bam', checkIfExists: true) ] - ] + ]) + input[1] = Channel.of([ + [ id:'fasta' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) """ } } @@ -61,13 +157,36 @@ nextflow_process { then { assertAll ( { assert process.success }, - { assert snapshot( - file(process.out.bam[0][1]).name, - process.out.versions - ).match() } + { assert snapshot(process.out).match() } ) } - } + test("cram - stub") { + + options "-stub" + config "./nextflow_cram.config" + + when { + process { + """ + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true) + ]) + input[1] = Channel.of([ + [ id:'fasta' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } } diff --git a/modules/nf-core/samtools/sort/tests/main.nf.test.snap b/modules/nf-core/samtools/sort/tests/main.nf.test.snap index ff722259..469891fe 100644 --- a/modules/nf-core/samtools/sort/tests/main.nf.test.snap +++ b/modules/nf-core/samtools/sort/tests/main.nf.test.snap @@ -1,5 +1,35 @@ { - "test_samtools_sort": { + "cram": { + "content": [ + [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.cram" + ] + ], + [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.cram.crai" + ] + ], + [ + "versions.yml:md5,2659b187d681241451539d4c53500b9f" + ] + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:49:58.207549273" + }, + "bam - stub": { "content": [ { "0": [ @@ -8,14 +38,26 @@ "id": "test", "single_end": false }, - "test.sorted.bam:md5,ea6a0fef94eb534e901f107a05a33a06" + "test.sorted.bam:md5,d41d8cd98f00b204e9800998ecf8427e" ] ], "1": [ ], "2": [ - "versions.yml:md5,33b6a403dc19a0d28e4219ccab0a1d80" + + ], + "3": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam.csi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "4": [ + "versions.yml:md5,2659b187d681241451539d4c53500b9f" ], "bam": [ [ @@ -23,26 +65,223 @@ "id": "test", "single_end": false }, - "test.sorted.bam:md5,ea6a0fef94eb534e901f107a05a33a06" + "test.sorted.bam:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "crai": [ + + ], + "cram": [ + + ], + "csi": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam.csi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,2659b187d681241451539d4c53500b9f" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:50:08.630951018" + }, + "cram - stub": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.cram:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "2": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.cram.crai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "3": [ + + ], + "4": [ + "versions.yml:md5,2659b187d681241451539d4c53500b9f" + ], + "bam": [ + + ], + "crai": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.cram.crai:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "cram": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.cram:md5,d41d8cd98f00b204e9800998ecf8427e" ] ], "csi": [ ], "versions": [ - "versions.yml:md5,33b6a403dc19a0d28e4219ccab0a1d80" + "versions.yml:md5,2659b187d681241451539d4c53500b9f" ] } ], - "timestamp": "2023-12-04T11:11:22.005628301" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T08:50:19.061912443" + }, + "multiple bam": { + "content": [ + [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam:md5,8a16ba90c7d294cbb4c33ac0f7127a12" + ] + ], + [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam.csi" + ] + ], + [ + "versions.yml:md5,2659b187d681241451539d4c53500b9f" + ] + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.09.0" + }, + "timestamp": "2024-10-08T11:59:55.479443" }, - "test_samtools_sort_stub": { + "multiple bam - stub": { "content": [ - "test.sorted.bam", + { + "0": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam:md5,8a16ba90c7d294cbb4c33ac0f7127a12" + ] + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam.csi:md5,d185916eaff9afeb4d0aeab3310371f9" + ] + ], + "4": [ + "versions.yml:md5,2659b187d681241451539d4c53500b9f" + ], + "bam": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam:md5,8a16ba90c7d294cbb4c33ac0f7127a12" + ] + ], + "crai": [ + + ], + "cram": [ + + ], + "csi": [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam.csi:md5,d185916eaff9afeb4d0aeab3310371f9" + ] + ], + "versions": [ + "versions.yml:md5,2659b187d681241451539d4c53500b9f" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.09.0" + }, + "timestamp": "2024-10-08T11:36:13.781404" + }, + "bam": { + "content": [ + [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam:md5,34aa85e86abefe637f7a4a9887f016fc" + ] + ], + [ + [ + { + "id": "test", + "single_end": false + }, + "test.sorted.bam.csi" + ] + ], [ - "versions.yml:md5,33b6a403dc19a0d28e4219ccab0a1d80" + "versions.yml:md5,2659b187d681241451539d4c53500b9f" ] ], - "timestamp": "2023-12-04T17:47:22.314445935" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.09.0" + }, + "timestamp": "2024-10-08T11:59:46.372244" } } \ No newline at end of file diff --git a/modules/nf-core/samtools/sort/tests/nextflow.config b/modules/nf-core/samtools/sort/tests/nextflow.config index d0f35086..f642771f 100644 --- a/modules/nf-core/samtools/sort/tests/nextflow.config +++ b/modules/nf-core/samtools/sort/tests/nextflow.config @@ -1,7 +1,8 @@ process { withName: SAMTOOLS_SORT { - ext.prefix = { "${meta.id}.sorted" } + ext.prefix = { "${meta.id}.sorted" } + ext.args = "--write-index" } } diff --git a/modules/nf-core/samtools/sort/tests/nextflow_cram.config b/modules/nf-core/samtools/sort/tests/nextflow_cram.config new file mode 100644 index 00000000..3a8c0188 --- /dev/null +++ b/modules/nf-core/samtools/sort/tests/nextflow_cram.config @@ -0,0 +1,8 @@ +process { + + withName: SAMTOOLS_SORT { + ext.prefix = { "${meta.id}.sorted" } + ext.args = "--write-index --output-fmt cram" + } + +} diff --git a/modules/nf-core/samtools/view/environment.yml b/modules/nf-core/samtools/view/environment.yml index 73ce7999..02cda6e6 100644 --- a/modules/nf-core/samtools/view/environment.yml +++ b/modules/nf-core/samtools/view/environment.yml @@ -1,8 +1,10 @@ -name: samtools_view +--- +# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::samtools=1.18 - - bioconda::htslib=1.18 + # renovate: datasource=conda depName=bioconda/htslib + - bioconda::htslib=1.21 + # renovate: datasource=conda depName=bioconda/samtools + - bioconda::samtools=1.21 diff --git a/modules/nf-core/samtools/view/main.nf b/modules/nf-core/samtools/view/main.nf index 0b5a2912..41fa3d6a 100644 --- a/modules/nf-core/samtools/view/main.nf +++ b/modules/nf-core/samtools/view/main.nf @@ -4,8 +4,8 @@ process SAMTOOLS_VIEW { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/samtools:1.18--h50ea8bc_1' : - 'biocontainers/samtools:1.18--h50ea8bc_1' }" + 'https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/9e/9edc2564215d5cd137a8b25ca8a311600987186d406b092022444adf3c4447f7/data' : + 'community​.wave​.seqera​.io/library/htslib_samtools:1​.21--6cb89bfd40cbaabf' }" input: tuple val(meta), path(input), path(index) @@ -13,13 +13,15 @@ process SAMTOOLS_VIEW { path qname output: - tuple val(meta), path("*.bam"), emit: bam, optional: true - tuple val(meta), path("*.cram"), emit: cram, optional: true - tuple val(meta), path("*.sam"), emit: sam, optional: true - tuple val(meta), path("*.bai"), emit: bai, optional: true - tuple val(meta), path("*.csi"), emit: csi, optional: true - tuple val(meta), path("*.crai"), emit: crai, optional: true - path "versions.yml", emit: versions + tuple val(meta), path("${prefix}.bam"), emit: bam, optional: true + tuple val(meta), path("${prefix}.cram"), emit: cram, optional: true + tuple val(meta), path("${prefix}.sam"), emit: sam, optional: true + tuple val(meta), path("${prefix}.${file_type}.bai"), emit: bai, optional: true + tuple val(meta), path("${prefix}.${file_type}.csi"), emit: csi, optional: true + tuple val(meta), path("${prefix}.${file_type}.crai"), emit: crai, optional: true + tuple val(meta), path("${prefix}.unselected.${file_type}"), emit: unselected, optional: true + tuple val(meta), path("${prefix}.unselected.${file_type}.{bai,csi,crsi}"), emit: unselected_index, optional: true + path "versions.yml", emit: versions when: task.ext.when == null || task.ext.when @@ -27,13 +29,13 @@ process SAMTOOLS_VIEW { script: def args = task.ext.args ?: '' def args2 = task.ext.args2 ?: '' - def prefix = task.ext.prefix ?: "${meta.id}" + prefix = task.ext.prefix ?: "${meta.id}" def reference = fasta ? "--reference ${fasta}" : "" - def readnames = qname ? "--qname-file ${qname}": "" - def file_type = args.contains("--output-fmt sam") ? "sam" : - args.contains("--output-fmt bam") ? "bam" : - args.contains("--output-fmt cram") ? "cram" : - input.getExtension() + file_type = args.contains("--output-fmt sam") ? "sam" : + args.contains("--output-fmt bam") ? "bam" : + args.contains("--output-fmt cram") ? "cram" : + input.getExtension() + readnames = qname ? "--qname-file ${qname} --output-unselected ${prefix}.unselected.${file_type}": "" if ("$input" == "${prefix}.${file_type}") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!" """ samtools \\ @@ -54,14 +56,14 @@ process SAMTOOLS_VIEW { stub: def args = task.ext.args ?: '' - def prefix = task.ext.prefix ?: "${meta.id}" - def file_type = args.contains("--output-fmt sam") ? "sam" : - args.contains("--output-fmt bam") ? "bam" : - args.contains("--output-fmt cram") ? "cram" : - input.getExtension() + prefix = task.ext.prefix ?: "${meta.id}" + file_type = args.contains("--output-fmt sam") ? "sam" : + args.contains("--output-fmt bam") ? "bam" : + args.contains("--output-fmt cram") ? "cram" : + input.getExtension() if ("$input" == "${prefix}.${file_type}") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!" - def index = args.contains("--write-index") ? "touch ${prefix}.csi" : "" + def index = args.contains("--write-index") ? "touch ${prefix}.${file_type}.csi" : "" """ touch ${prefix}.${file_type} diff --git a/modules/nf-core/samtools/view/meta.yml b/modules/nf-core/samtools/view/meta.yml index 3dadafae..caa7b015 100644 --- a/modules/nf-core/samtools/view/meta.yml +++ b/modules/nf-core/samtools/view/meta.yml @@ -15,68 +15,120 @@ tools: documentation: http://www.htslib.org/doc/samtools.html doi: 10.1093/bioinformatics/btp352 licence: ["MIT"] + identifier: biotools:samtools input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - input: - type: file - description: BAM/CRAM/SAM file - pattern: "*.{bam,cram,sam}" - - index: - type: file - description: BAM.BAI/BAM.CSI/CRAM.CRAI file (optional) - pattern: "*.{.bai,.csi,.crai}" - - meta2: - type: map - description: | - Groovy Map containing reference information - e.g. [ id:'test' ] - - fasta: - type: file - description: Reference file the CRAM was created with (optional) - pattern: "*.{fasta,fa}" - - qname: - type: file - description: Optional file with read names to output only select alignments - pattern: "*.{txt,list}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - input: + type: file + description: BAM/CRAM/SAM file + pattern: "*.{bam,cram,sam}" + - index: + type: file + description: BAM.BAI/BAM.CSI/CRAM.CRAI file (optional) + pattern: "*.{.bai,.csi,.crai}" + - - meta2: + type: map + description: | + Groovy Map containing reference information + e.g. [ id:'test' ] + - fasta: + type: file + description: Reference file the CRAM was created with (optional) + pattern: "*.{fasta,fa}" + - - qname: + type: file + description: Optional file with read names to output only select alignments + pattern: "*.{txt,list}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - bam: - type: file - description: optional filtered/converted BAM file - pattern: "*.{bam}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.bam: + type: file + description: optional filtered/converted BAM file + pattern: "*.{bam}" - cram: - type: file - description: optional filtered/converted CRAM file - pattern: "*.{cram}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.cram: + type: file + description: optional filtered/converted CRAM file + pattern: "*.{cram}" - sam: - type: file - description: optional filtered/converted SAM file - pattern: "*.{sam}" - # bai, csi, and crai are created with `--write-index` + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.sam: + type: file + description: optional filtered/converted SAM file + pattern: "*.{sam}" - bai: - type: file - description: optional BAM file index - pattern: "*.{bai}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.${file_type}.bai: + type: file + description: optional BAM file index + pattern: "*.{bai}" - csi: - type: file - description: optional tabix BAM file index - pattern: "*.{csi}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.${file_type}.csi: + type: file + description: optional tabix BAM file index + pattern: "*.{csi}" - crai: - type: file - description: optional CRAM file index - pattern: "*.{crai}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.${file_type}.crai: + type: file + description: optional CRAM file index + pattern: "*.{crai}" + - unselected: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.unselected.${file_type}: + type: file + description: optional file with unselected alignments + pattern: "*.unselected.{bam,cram,sam}" + - unselected_index: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${prefix}.unselected.${file_type}.{bai,csi,crsi}: + type: file + description: index for the "unselected" file + pattern: "*.unselected.{bai,csi,crai}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@drpatelh" - "@joseespinosa" diff --git a/modules/nf-core/samtools/view/tests/main.nf.test b/modules/nf-core/samtools/view/tests/main.nf.test index 89ed3555..37b81a91 100644 --- a/modules/nf-core/samtools/view/tests/main.nf.test +++ b/modules/nf-core/samtools/view/tests/main.nf.test @@ -9,16 +9,16 @@ nextflow_process { tag "samtools" tag "samtools/view" - test("sarscov2 - [bam, []], [], []") { + test("bam") { when { process { """ - input[0] = [ + input[0] = Channel.of([ [ id:'test', single_end:false ], // meta map - file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.bam', checkIfExists: true), [] - ] + ]) input[1] = [[],[]] input[2] = [] """ @@ -28,34 +28,31 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.bam[0][1]).name, - process.out.cram, - process.out.sam, - process.out.bai, - process.out.crai, - process.out.csi, - process.out.versions - ).match() } + { assert snapshot(file(process.out.bam[0][1]).name).match("bam_bam") }, + { assert snapshot(process.out.bai).match("bam_bai") }, + { assert snapshot(process.out.crai).match("bam_crai") }, + { assert snapshot(process.out.cram).match("bam_cram") }, + { assert snapshot(process.out.csi).match("bam_csi") }, + { assert snapshot(process.out.sam).match("bam_sam") }, + { assert snapshot(process.out.versions).match("bam_versions") } ) } - } - test("homo_sapiens - [cram, crai], fasta, []") { + test("cram") { when { process { """ - input[0] = [ - [ id: 'test' ], // meta map - file(params.test_data['homo_sapiens']['illumina']['test_paired_end_sorted_cram'], checkIfExists: true), - file(params.test_data['homo_sapiens']['illumina']['test_paired_end_sorted_cram_crai'], checkIfExists: true) - ] - input[1] = [ - [ id:'genome' ], - file(params.test_data['homo_sapiens']['genome']['genome_fasta'], checkIfExists: true) - ] + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram.crai', checkIfExists: true) + ]) + input[1] = Channel.of([ + [ id:'genome' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) input[2] = [] """ } @@ -64,36 +61,33 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.cram[0][1]).name, - process.out.bam, - process.out.sam, - process.out.bai, - process.out.crai, - process.out.csi, - process.out.versions - ).match() } + { assert snapshot(file(process.out.cram[0][1]).name).match("cram_cram") }, + { assert snapshot(process.out.bai).match("cram_bai") }, + { assert snapshot(process.out.bam).match("cram_bam") }, + { assert snapshot(process.out.crai).match("cram_crai") }, + { assert snapshot(process.out.csi).match("cram_csi") }, + { assert snapshot(process.out.sam).match("cram_sam") }, + { assert snapshot(process.out.versions).match("cram_versions") } ) } - } - test("homo_sapiens - [cram, []], fasta, [] - bam output") { + test("cram_to_bam") { config "./bam.config" when { process { """ - input[0] = [ - [ id: 'test' ], // meta map - file(params.test_data['homo_sapiens']['illumina']['test_paired_end_sorted_cram'], checkIfExists: true), + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true), [] - ] - input[1] = [ - [ id:'genome' ], - file(params.test_data['homo_sapiens']['genome']['genome_fasta'], checkIfExists: true) - ] + ]) + input[1] = Channel.of([ + [ id:'genome' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) input[2] = [] """ } @@ -102,36 +96,33 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.bam[0][1]).name, - process.out.cram, - process.out.sam, - process.out.bai, - process.out.crai, - process.out.csi, - process.out.versions - ).match() } + { assert snapshot(file(process.out.bam[0][1]).name).match("cram_to_bam_bam") }, + { assert snapshot(process.out.bai).match("cram_to_bam_bai") }, + { assert snapshot(process.out.crai).match("cram_to_bam_crai") }, + { assert snapshot(process.out.cram).match("cram_to_bam_cram") }, + { assert snapshot(process.out.csi).match("cram_to_bam_csi") }, + { assert snapshot(process.out.sam).match("cram_to_bam_sam") }, + { assert snapshot(process.out.versions).match("cram_to_bam_versions") } ) } - } - test("homo_sapiens - [cram, []], fasta, [] - bam & index output") { + test("cram_to_bam_index") { config "./bam_index.config" when { process { """ - input[0] = [ - [ id: 'test' ], // meta map - file(params.test_data['homo_sapiens']['illumina']['test_paired_end_sorted_cram'], checkIfExists: true), + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true), [] - ] - input[1] = [ - [ id:'genome' ], - file(params.test_data['homo_sapiens']['genome']['genome_fasta'], checkIfExists: true) - ] + ]) + input[1] = Channel.of([ + [ id:'genome' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) input[2] = [] """ } @@ -140,36 +131,33 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.bam[0][1]).name, - process.out.cram, - process.out.sam, - file(process.out.csi[0][1]).name, - process.out.crai, - process.out.bai, - process.out.versions - ).match() } + { assert snapshot(file(process.out.bam[0][1]).name).match("cram_to_bam_index_bam") }, + { assert snapshot(file(process.out.csi[0][1]).name).match("cram_to_bam_index_csi") }, + { assert snapshot(process.out.bai).match("cram_to_bam_index_bai") }, + { assert snapshot(process.out.crai).match("cram_to_bam_index_crai") }, + { assert snapshot(process.out.cram).match("cram_to_bam_index_cram") }, + { assert snapshot(process.out.sam).match("cram_to_bam_index_sam") }, + { assert snapshot(process.out.versions).match("cram_to_bam_index_versions") } ) } - } - test("homo_sapiens - [cram, []], fasta, qname - bam & index output") { + test("cram_to_bam_index_qname") { config "./bam_index.config" when { process { """ - input[0] = [ - [ id: 'test' ], // meta map - file(params.test_data['homo_sapiens']['illumina']['test_paired_end_sorted_cram'], checkIfExists: true), + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true), [] - ] - input[1] = [ - [ id:'genome' ], - file(params.test_data['homo_sapiens']['genome']['genome_fasta'], checkIfExists: true) - ] + ]) + input[1] = Channel.of([ + [ id:'genome' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) input[2] = Channel.of("testN:2817", "testN:2814").collectFile(name: "readnames.list", newLine: true) """ } @@ -178,21 +166,20 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.bam[0][1]).name, - process.out.cram, - process.out.sam, - file(process.out.csi[0][1]).name, - process.out.crai, - process.out.bai, - process.out.versions - ).match() } + { assert snapshot(file(process.out.bam[0][1]).name).match("cram_to_bam_index_qname_bam") }, + { assert snapshot(file(process.out.csi[0][1]).name).match("cram_to_bam_index_qname_csi") }, + { assert snapshot(process.out.bai).match("cram_to_bam_index_qname_bai") }, + { assert snapshot(process.out.crai).match("cram_to_bam_index_qname_crai") }, + { assert snapshot(process.out.cram).match("cram_to_bam_index_qname_cram") }, + { assert snapshot(process.out.sam).match("cram_to_bam_index_qname_sam") }, + { assert snapshot(file(process.out.unselected[0][1]).name).match("cram_to_bam_index_qname_unselected") }, + { assert snapshot(file(process.out.unselected_index[0][1]).name).match("cram_to_bam_index_qname_unselected_csi") }, + { assert snapshot(process.out.versions).match("cram_to_bam_index_qname_versions") } ) } - } - test("sarscov2 - [bam, []], [], [] - stub") { + test("bam_stub") { options "-stub" config "./bam_index.config" @@ -200,11 +187,11 @@ nextflow_process { when { process { """ - input[0] = [ + input[0] = Channel.of([ [ id:'test', single_end:false ], // meta map - file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true), + file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.bam', checkIfExists: true), [] - ] + ]) input[1] = [[],[]] input[2] = [] """ @@ -214,18 +201,14 @@ nextflow_process { then { assertAll( { assert process.success }, - { assert snapshot( - file(process.out.bam[0][1]).name, - process.out.cram, - process.out.sam, - file(process.out.csi[0][1]).name, - process.out.crai, - process.out.bai, - process.out.versions - ).match() } + { assert snapshot(file(process.out.bam[0][1]).name).match("bam_stub_bam") }, + { assert snapshot(file(process.out.csi[0][1]).name).match("bam_stub_csi") }, + { assert snapshot(process.out.bai).match("bam_stub_bai") }, + { assert snapshot(process.out.crai).match("bam_stub_crai") }, + { assert snapshot(process.out.cram).match("bam_stub_cram") }, + { assert snapshot(process.out.sam).match("bam_stub_sam") }, + { assert snapshot(process.out.versions).match("bam_stub_versions") } ) } - } - } diff --git a/modules/nf-core/samtools/view/tests/main.nf.test.snap b/modules/nf-core/samtools/view/tests/main.nf.test.snap index 83427491..63849b03 100644 --- a/modules/nf-core/samtools/view/tests/main.nf.test.snap +++ b/modules/nf-core/samtools/view/tests/main.nf.test.snap @@ -1,140 +1,528 @@ { - "homo_sapiens - [cram, []], fasta, [] - bam output": { + "bam_bam": { + "content": [ + "test.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:51.256068" + }, + "cram_to_bam_index_csi": { + "content": [ + "test.bam.csi" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:12.958617" + }, + "bam_stub_bam": { + "content": [ + "test.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:32.065301" + }, + "bam_bai": { "content": [ - "test.bam", [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:51.258578" + }, + "bam_stub_bai": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:32.071284" + }, + "bam_stub_versions": { + "content": [ [ - - ], + "versions.yml:md5,176db5ec46b965219604bcdbb3ef9e07" + ] + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T09:26:24.461775464" + }, + "cram_to_bam_index_cram": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:12.972288" + }, + "cram_to_bam_sam": { + "content": [ [ - ], - [ - "versions.yml:md5,06b9049228b111e7bed5c52fe8a98d9b" ] ], - "timestamp": "2023-12-04T17:41:17.563069206" + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:04.999247" }, - "sarscov2 - [bam, []], [], []": { + "cram_to_bam_index_sam": { "content": [ - "test.bam", [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:12.976457" + }, + "cram_crai": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:56.497581" + }, + "cram_csi": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:56.50038" + }, + "cram_to_bam_cram": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:04.992239" + }, + "cram_to_bam_index_qname_csi": { + "content": [ + "test.bam.csi" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.325496" + }, + "bam_stub_sam": { + "content": [ [ - ], - [ - "versions.yml:md5,06b9049228b111e7bed5c52fe8a98d9b" ] ], - "timestamp": "2023-12-04T17:41:03.206994564" + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:32.079529" + }, + "cram_cram": { + "content": [ + "test.cram" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:56.490286" }, - "homo_sapiens - [cram, []], fasta, qname - bam & index output": { + "cram_to_bam_index_qname_unselected_csi": { + "content": [ + "test.unselected.bam.csi" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.328458" + }, + "bam_csi": { "content": [ - "test.bam", [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:51.262882" + }, + "cram_to_bam_crai": { + "content": [ [ - ], - "test.bam.csi", + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:04.989247" + }, + "cram_to_bam_index_crai": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:12.967681" + }, + "cram_to_bam_index_qname_versions": { + "content": [ [ - - ], + "versions.yml:md5,176db5ec46b965219604bcdbb3ef9e07" + ] + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T09:25:51.953436682" + }, + "cram_to_bam_bam": { + "content": [ + "test.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:04.982361" + }, + "cram_to_bam_index_bam": { + "content": [ + "test.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:12.95456" + }, + "cram_to_bam_index_versions": { + "content": [ [ - "versions.yml:md5,06b9049228b111e7bed5c52fe8a98d9b" + "versions.yml:md5,176db5ec46b965219604bcdbb3ef9e07" ] ], - "timestamp": "2023-12-04T17:44:39.165289759" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T09:25:14.475388399" }, - "homo_sapiens - [cram, []], fasta, [] - bam & index output": { + "cram_to_bam_bai": { "content": [ - "test.bam", [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:04.98601" + }, + "cram_to_bam_versions": { + "content": [ [ - - ], - "test.bam.csi", + "versions.yml:md5,176db5ec46b965219604bcdbb3ef9e07" + ] + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T09:24:49.673441798" + }, + "cram_bam": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:56.495512" + }, + "bam_stub_cram": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:32.076908" + }, + "cram_to_bam_index_qname_bai": { + "content": [ [ - "versions.yml:md5,06b9049228b111e7bed5c52fe8a98d9b" + ] ], - "timestamp": "2023-12-04T17:44:32.25731224" + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.328458" }, - "sarscov2 - [bam, []], [], [] - stub": { + "cram_to_bam_index_qname_crai": { "content": [ - "test.bam", [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.330789" + }, + "cram_bai": { + "content": [ [ - ], - "test.csi", + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:56.493129" + }, + "bam_stub_crai": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:32.074313" + }, + "cram_to_bam_index_qname_bam": { + "content": [ + "test.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.322874" + }, + "cram_to_bam_index_qname_unselected": { + "content": [ + "test.unselected.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.322874" + }, + "cram_to_bam_index_qname_unselected_csi": { + "content": [ + "test.unselected.bam.csi" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.328458" + }, + "bam_versions": { + "content": [ + [ + "versions.yml:md5,176db5ec46b965219604bcdbb3ef9e07" + ] + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T09:23:27.151650338" + }, + "cram_to_bam_index_qname_cram": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.333248" + }, + "bam_crai": { + "content": [ [ - "versions.yml:md5,06b9049228b111e7bed5c52fe8a98d9b" + ] ], - "timestamp": "2023-12-04T17:44:45.81037195" + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:51.259774" }, - "homo_sapiens - [cram, crai], fasta, []": { + "bam_cram": { "content": [ - "test.cram", [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:51.261287" + }, + "cram_to_bam_csi": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:04.995454" + }, + "cram_sam": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:56.502625" + }, + "cram_versions": { + "content": [ + [ + "versions.yml:md5,176db5ec46b965219604bcdbb3ef9e07" + ] + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-16T09:24:12.95416913" + }, + "cram_to_bam_index_qname_unselected": { + "content": [ + "test.unselected.bam" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.322874" + }, + "bam_sam": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:37:51.264651" + }, + "cram_to_bam_index_bai": { + "content": [ [ - ], + ] + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:12.962863" + }, + "cram_to_bam_index_qname_sam": { + "content": [ [ - "versions.yml:md5,06b9049228b111e7bed5c52fe8a98d9b" + ] ], - "timestamp": "2023-12-04T17:41:10.730011823" + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:23.337634" + }, + "bam_stub_csi": { + "content": [ + "test.bam.csi" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "23.04.3" + }, + "timestamp": "2024-02-12T19:38:32.068596" } } \ No newline at end of file diff --git a/modules/nf-core/seqtk/cutn/environment.yml b/modules/nf-core/seqtk/cutn/environment.yml index a57afbb6..693aa5c1 100644 --- a/modules/nf-core/seqtk/cutn/environment.yml +++ b/modules/nf-core/seqtk/cutn/environment.yml @@ -1,7 +1,5 @@ -name: seqtk_cutn channels: - conda-forge - bioconda - - defaults dependencies: - bioconda::seqtk=1.4 diff --git a/modules/nf-core/seqtk/cutn/meta.yml b/modules/nf-core/seqtk/cutn/meta.yml index 10828679..2e39a477 100644 --- a/modules/nf-core/seqtk/cutn/meta.yml +++ b/modules/nf-core/seqtk/cutn/meta.yml @@ -6,35 +6,40 @@ keywords: - seqtk tools: - seqtk: - description: Seqtk is a fast and lightweight tool for processing sequences in the FASTA or FASTQ format. Seqtk mergepe command merges pair-end reads into one interleaved file. + description: Seqtk is a fast and lightweight tool for processing sequences in + the FASTA or FASTQ format. Seqtk mergepe command merges pair-end reads into + one interleaved file. homepage: https://github.com/lh3/seqtk documentation: https://docs.csc.fi/apps/seqtk/ tool_dev_url: https://github.com/lh3/seqtk licence: ["MIT"] + identifier: biotools:seqtk input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - fasta: - type: file - description: A single fasta file to be split. - pattern: "*.{fasta}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - fasta: + type: file + description: A single fasta file to be split. + pattern: "*.{fasta}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - bed: - type: file - description: The output bed which summarised locations of cuts - pattern: "*.{bed}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bed": + type: file + description: The output bed which summarised locations of cuts + pattern: "*.{bed}" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@DLBPointon" maintainers: diff --git a/modules/nf-core/seqtk/cutn/tests/main.nf.test b/modules/nf-core/seqtk/cutn/tests/main.nf.test new file mode 100644 index 00000000..46dfff1f --- /dev/null +++ b/modules/nf-core/seqtk/cutn/tests/main.nf.test @@ -0,0 +1,57 @@ +nextflow_process { + + name "Test Process SEQTK_CUTN" + script "../main.nf" + process "SEQTK_CUTN" + + tag "modules" + tag "modules_nfcore" + tag "seqtk" + tag "seqtk/cutn" + + test("homo_21_cut") { + + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/chr21/sequence/genome.fasta', checkIfExists: true) + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() }, + { assert snapshot( + file(process.out.bed[0][1]).name + ).match("genome_cut") + } + ) + } + } + + test("homo_21_cut_stub") { + options "-stub" + when { + process { + """ + input[0] = [ + [ id:'test', single_end:true ], + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/chr21/sequence/genome.fasta', checkIfExists: true) + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } +} diff --git a/modules/nf-core/seqtk/cutn/tests/main.nf.test.snap b/modules/nf-core/seqtk/cutn/tests/main.nf.test.snap new file mode 100644 index 00000000..998beda5 --- /dev/null +++ b/modules/nf-core/seqtk/cutn/tests/main.nf.test.snap @@ -0,0 +1,70 @@ +{ + "genome_cut": { + "content": [ + "test.bed" + ], + "timestamp": "2024-02-22T16:02:14.744148" + }, + "homo_21_cut_stub": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bed:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,3da8ed2738f3c093d1e62d796fd76428" + ], + "bed": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bed:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,3da8ed2738f3c093d1e62d796fd76428" + ] + } + ], + "timestamp": "2024-02-22T16:02:23.596389" + }, + "homo_21_cut": { + "content": [ + { + "0": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bed:md5,16cbba84e3a4bdbb52217afb5051f948" + ] + ], + "1": [ + "versions.yml:md5,3da8ed2738f3c093d1e62d796fd76428" + ], + "bed": [ + [ + { + "id": "test", + "single_end": true + }, + "test.bed:md5,16cbba84e3a4bdbb52217afb5051f948" + ] + ], + "versions": [ + "versions.yml:md5,3da8ed2738f3c093d1e62d796fd76428" + ] + } + ], + "timestamp": "2024-02-22T16:02:14.695205" + } +} \ No newline at end of file diff --git a/modules/nf-core/seqtk/cutn/tests/tags.yml b/modules/nf-core/seqtk/cutn/tests/tags.yml new file mode 100644 index 00000000..13c64cc3 --- /dev/null +++ b/modules/nf-core/seqtk/cutn/tests/tags.yml @@ -0,0 +1,2 @@ +seqtk/cutn: + - "modules/nf-core/seqtk/cutn/**" diff --git a/modules/nf-core/tabix/bgziptabix/environment.yml b/modules/nf-core/tabix/bgziptabix/environment.yml index 028461c9..017c259d 100644 --- a/modules/nf-core/tabix/bgziptabix/environment.yml +++ b/modules/nf-core/tabix/bgziptabix/environment.yml @@ -1,7 +1,7 @@ -name: tabix_bgziptabix channels: - conda-forge - bioconda - - defaults + dependencies: + - bioconda::htslib=1.20 - bioconda::tabix=1.11 diff --git a/modules/nf-core/tabix/bgziptabix/main.nf b/modules/nf-core/tabix/bgziptabix/main.nf index f9482690..22f37a77 100644 --- a/modules/nf-core/tabix/bgziptabix/main.nf +++ b/modules/nf-core/tabix/bgziptabix/main.nf @@ -4,8 +4,8 @@ process TABIX_BGZIPTABIX { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/tabix:1.11--hdfd78af_0' : - 'biocontainers/tabix:1.11--hdfd78af_0' }" + 'https://depot.galaxyproject.org/singularity/htslib:1.20--h5efdd21_2' : + 'biocontainers/htslib:1.20--h5efdd21_2' }" input: tuple val(meta), path(input) @@ -24,7 +24,7 @@ process TABIX_BGZIPTABIX { def prefix = task.ext.prefix ?: "${meta.id}" """ bgzip --threads ${task.cpus} -c $args $input > ${prefix}.${input.getExtension()}.gz - tabix $args2 ${prefix}.${input.getExtension()}.gz + tabix --threads ${task.cpus} $args2 ${prefix}.${input.getExtension()}.gz cat <<-END_VERSIONS > versions.yml "${task.process}": @@ -34,10 +34,11 @@ process TABIX_BGZIPTABIX { stub: def prefix = task.ext.prefix ?: "${meta.id}" + def args2 = task.ext.args2 ?: '' + def index = args2.contains("-C ") || args2.contains("--csi") ? "csi" : "tbi" """ - touch ${prefix}.${input.getExtension()}.gz - touch ${prefix}.${input.getExtension()}.gz.tbi - touch ${prefix}.${input.getExtension()}.gz.csi + echo "" | gzip > ${prefix}.${input.getExtension()}.gz + touch ${prefix}.${input.getExtension()}.gz.${index} cat <<-END_VERSIONS > versions.yml "${task.process}": diff --git a/modules/nf-core/tabix/bgziptabix/meta.yml b/modules/nf-core/tabix/bgziptabix/meta.yml index 438aba4d..806fbc12 100644 --- a/modules/nf-core/tabix/bgziptabix/meta.yml +++ b/modules/nf-core/tabix/bgziptabix/meta.yml @@ -13,38 +13,50 @@ tools: documentation: https://www.htslib.org/doc/tabix.1.html doi: 10.1093/bioinformatics/btq671 licence: ["MIT"] + identifier: biotools:tabix input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - tab: - type: file - description: TAB-delimited genome position file - pattern: "*.{bed,gff,sam,vcf}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - input: + type: file + description: Sorted tab-delimited genome file output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - gz: - type: file - description: Output compressed file - pattern: "*.{gz}" - - tbi: - type: file - description: tabix index file - pattern: "*.{gz.tbi}" - - csi: - type: file - description: tabix alternate index file - pattern: "*.{gz.csi}" + - gz_tbi: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.gz": + type: file + description: bgzipped tab-delimited genome file + pattern: "*.gz" + - "*.tbi": + type: file + description: tabix index file + pattern: "*.tbi" + - gz_csi: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.gz": + type: file + description: bgzipped tab-delimited genome file + pattern: "*.gz" + - "*.csi": + type: file + description: csi index file + pattern: "*.csi" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@maxulysse" - "@DLBPointon" diff --git a/modules/nf-core/tabix/bgziptabix/tests/main.nf.test b/modules/nf-core/tabix/bgziptabix/tests/main.nf.test new file mode 100644 index 00000000..4d4130dc --- /dev/null +++ b/modules/nf-core/tabix/bgziptabix/tests/main.nf.test @@ -0,0 +1,123 @@ +nextflow_process { + + name "Test Process TABIX_BGZIPTABIX" + script "modules/nf-core/tabix/bgziptabix/main.nf" + process "TABIX_BGZIPTABIX" + + tag "modules" + tag "modules_nfcore" + tag "tabix" + tag "tabix/bgziptabix" + + test("sarscov2_bed_tbi") { + config "./tabix_tbi.config" + + when { + process { + """ + input[0] = [ + [ id:'tbi_test' ], + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true) ] + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() }, + { assert snapshot( + file(process.out.gz_tbi[0][1]).name + ).match("tbi_test") + } + ) + } + } + + test("sarscov2_bed_csi") { + config "./tabix_csi.config" + + when { + process { + """ + input[0] = [ + [ id:'csi_test' ], + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true) ] + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() }, + { assert snapshot( + file(process.out.gz_csi[0][1]).name + ).match("csi_test") + } + ) + } + + } + + test("sarscov2_bed_csi_stub") { + config "./tabix_csi.config" + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test' ], + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true) ] + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() }, + { assert snapshot( + file(process.out.gz_csi[0][1]).name + ).match("csi_stub") + } + ) + } + + } + + test("sarscov2_bed_tbi_stub") { + config "./tabix_tbi.config" + + options "-stub" + + when { + process { + """ + input[0] = [ + [ id:'test' ], + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true) ] + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() }, + { assert snapshot( + file(process.out.gz_tbi[0][1]).name + ).match("tbi_stub") + } + ) + } + + } + +} diff --git a/modules/nf-core/tabix/bgziptabix/tests/main.nf.test.snap b/modules/nf-core/tabix/bgziptabix/tests/main.nf.test.snap new file mode 100644 index 00000000..fb87799b --- /dev/null +++ b/modules/nf-core/tabix/bgziptabix/tests/main.nf.test.snap @@ -0,0 +1,206 @@ +{ + "sarscov2_bed_tbi": { + "content": [ + { + "0": [ + [ + { + "id": "tbi_test" + }, + "tbi_test.bed.gz:md5,fe4053cf4de3aebbdfc3be2efb125a74", + "tbi_test.bed.gz.tbi:md5,ca06caf88b1e3c67d5fcba0a1460b52c" + ] + ], + "1": [ + + ], + "2": [ + "versions.yml:md5,736e7c3b16a3ac525253e5b5f5d8fdfa" + ], + "gz_csi": [ + + ], + "gz_tbi": [ + [ + { + "id": "tbi_test" + }, + "tbi_test.bed.gz:md5,fe4053cf4de3aebbdfc3be2efb125a74", + "tbi_test.bed.gz.tbi:md5,ca06caf88b1e3c67d5fcba0a1460b52c" + ] + ], + "versions": [ + "versions.yml:md5,736e7c3b16a3ac525253e5b5f5d8fdfa" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-19T11:29:16.053817543" + }, + "sarscov2_bed_csi": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "csi_test" + }, + "csi_test.bed.gz:md5,fe4053cf4de3aebbdfc3be2efb125a74", + "csi_test.bed.gz.csi:md5,c9c0377de58fdc89672bb3005a0d69f5" + ] + ], + "2": [ + "versions.yml:md5,736e7c3b16a3ac525253e5b5f5d8fdfa" + ], + "gz_csi": [ + [ + { + "id": "csi_test" + }, + "csi_test.bed.gz:md5,fe4053cf4de3aebbdfc3be2efb125a74", + "csi_test.bed.gz.csi:md5,c9c0377de58fdc89672bb3005a0d69f5" + ] + ], + "gz_tbi": [ + + ], + "versions": [ + "versions.yml:md5,736e7c3b16a3ac525253e5b5f5d8fdfa" + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-07-19T11:29:27.667745444" + }, + "csi_test": { + "content": [ + "csi_test.bed.gz" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-02-19T14:51:00.548801" + }, + "sarscov2_bed_tbi_stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.bed.gz:md5,68b329da9893e34099c7d8ad5cb9c940", + "test.bed.gz.tbi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + + ], + "2": [ + "versions.yml:md5,736e7c3b16a3ac525253e5b5f5d8fdfa" + ], + "gz_csi": [ + + ], + "gz_tbi": [ + [ + { + "id": "test" + }, + "test.bed.gz:md5,68b329da9893e34099c7d8ad5cb9c940", + "test.bed.gz.tbi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,736e7c3b16a3ac525253e5b5f5d8fdfa" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-25T14:45:18.533169949" + }, + "csi_stub": { + "content": [ + "test.bed.gz" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-02-19T14:51:09.218454" + }, + "tbi_stub": { + "content": [ + "test.bed.gz" + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-25T14:45:18.550930179" + }, + "tbi_test": { + "content": [ + "tbi_test.bed.gz" + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-02-19T14:50:51.579654" + }, + "sarscov2_bed_csi_stub": { + "content": [ + { + "0": [ + + ], + "1": [ + [ + { + "id": "test" + }, + "test.bed.gz:md5,68b329da9893e34099c7d8ad5cb9c940", + "test.bed.gz.csi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "2": [ + "versions.yml:md5,736e7c3b16a3ac525253e5b5f5d8fdfa" + ], + "gz_csi": [ + [ + { + "id": "test" + }, + "test.bed.gz:md5,68b329da9893e34099c7d8ad5cb9c940", + "test.bed.gz.csi:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "gz_tbi": [ + + ], + "versions": [ + "versions.yml:md5,736e7c3b16a3ac525253e5b5f5d8fdfa" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-09-25T14:44:19.786135972" + } +} \ No newline at end of file diff --git a/modules/nf-core/tabix/bgziptabix/tests/tabix_csi.config b/modules/nf-core/tabix/bgziptabix/tests/tabix_csi.config new file mode 100644 index 00000000..fb41a314 --- /dev/null +++ b/modules/nf-core/tabix/bgziptabix/tests/tabix_csi.config @@ -0,0 +1,5 @@ +process { + withName: TABIX_BGZIPTABIX { + ext.args2 = '-p vcf --csi' + } +} diff --git a/modules/nf-core/tabix/bgziptabix/tests/tabix_tbi.config b/modules/nf-core/tabix/bgziptabix/tests/tabix_tbi.config new file mode 100644 index 00000000..c1915dc4 --- /dev/null +++ b/modules/nf-core/tabix/bgziptabix/tests/tabix_tbi.config @@ -0,0 +1,5 @@ +process { + withName: TABIX_BGZIPTABIX { + ext.args2 = '-p vcf' + } +} \ No newline at end of file diff --git a/modules/nf-core/tabix/bgziptabix/tests/tags.yml b/modules/nf-core/tabix/bgziptabix/tests/tags.yml new file mode 100644 index 00000000..5052b4d7 --- /dev/null +++ b/modules/nf-core/tabix/bgziptabix/tests/tags.yml @@ -0,0 +1,2 @@ +tabix/bgziptabix: + - "modules/nf-core/tabix/bgziptabix/**" diff --git a/modules/nf-core/ucsc/bedgraphtobigwig/environment.yml b/modules/nf-core/ucsc/bedgraphtobigwig/environment.yml index f91e216f..27868a44 100644 --- a/modules/nf-core/ucsc/bedgraphtobigwig/environment.yml +++ b/modules/nf-core/ucsc/bedgraphtobigwig/environment.yml @@ -1,7 +1,5 @@ -name: ucsc_bedgraphtobigwig channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::ucsc-bedgraphtobigwig=445 + - bioconda::ucsc-bedgraphtobigwig=469 diff --git a/modules/nf-core/ucsc/bedgraphtobigwig/main.nf b/modules/nf-core/ucsc/bedgraphtobigwig/main.nf index bff0b00a..81cdee95 100644 --- a/modules/nf-core/ucsc/bedgraphtobigwig/main.nf +++ b/modules/nf-core/ucsc/bedgraphtobigwig/main.nf @@ -5,8 +5,8 @@ process UCSC_BEDGRAPHTOBIGWIG { // WARN: Version information not provided by tool on CLI. Please update version string below when bumping container versions. conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/ucsc-bedgraphtobigwig:445--h954228d_0' : - 'biocontainers/ucsc-bedgraphtobigwig:445--h954228d_0' }" + 'https://depot.galaxyproject.org/singularity/ucsc-bedgraphtobigwig:469--h9b8f530_0' : + 'biocontainers/ucsc-bedgraphtobigwig:469--h9b8f530_0' }" input: tuple val(meta), path(bedgraph) @@ -22,7 +22,7 @@ process UCSC_BEDGRAPHTOBIGWIG { script: def args = task.ext.args ?: '' def prefix = task.ext.prefix ?: "${meta.id}" - def VERSION = '445' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. + def VERSION = '469' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. """ bedGraphToBigWig \\ $bedgraph \\ @@ -37,7 +37,7 @@ process UCSC_BEDGRAPHTOBIGWIG { stub: def prefix = task.ext.prefix ?: "${meta.id}" - def VERSION = '445' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. + def VERSION = '469' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. """ touch ${prefix}.bigWig diff --git a/modules/nf-core/ucsc/bedgraphtobigwig/meta.yml b/modules/nf-core/ucsc/bedgraphtobigwig/meta.yml index a60118ab..5d42cbde 100755 --- a/modules/nf-core/ucsc/bedgraphtobigwig/meta.yml +++ b/modules/nf-core/ucsc/bedgraphtobigwig/meta.yml @@ -12,34 +12,37 @@ tools: homepage: http://hgdownload.cse.ucsc.edu/admin/exe/ documentation: https://genome.ucsc.edu/goldenPath/help/bigWig.html licence: ["varies; see http://genome.ucsc.edu/license"] + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - bedgraph: - type: file - description: bedGraph file - pattern: "*.{bedGraph}" - - sizes: - type: file - description: chromosome sizes file - pattern: "*.{sizes}" + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - bedgraph: + type: file + description: bedGraph file + pattern: "*.{bedGraph}" + - - sizes: + type: file + description: chromosome sizes file + pattern: "*.{sizes}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - bigwig: - type: file - description: bigWig file - pattern: "*.{bigWig}" + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bigWig": + type: file + description: bigWig file + pattern: "*.{bigWig}" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@drpatelh" maintainers: diff --git a/modules/nf-core/ucsc/bedgraphtobigwig/tests/main.nf.test b/modules/nf-core/ucsc/bedgraphtobigwig/tests/main.nf.test index 6209ddaf..94a799ef 100644 --- a/modules/nf-core/ucsc/bedgraphtobigwig/tests/main.nf.test +++ b/modules/nf-core/ucsc/bedgraphtobigwig/tests/main.nf.test @@ -9,17 +9,14 @@ nextflow_process { tag "ucsc/bedgraphtobigwig" test("Should run without failures") { - when { - params { - outdir = "$outputDir" - } process { """ - input[0] = [ [ id:'test' ], // meta map - [ file(params.test_data['sarscov2']['illumina']['test_bedgraph'], checkIfExists: true) ] - ] - input[1] = file(params.test_data['sarscov2']['genome']['genome_sizes'], checkIfExists: true) + input[0] = Channel.of([ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/bedgraph/test.bedgraph", checkIfExists: true) + ]) + input[1] = Channel.of(file(params.modules_testdata_base_path + "genomics/sarscov2/genome/genome.sizes", checkIfExists: true)) """ } } @@ -30,7 +27,27 @@ nextflow_process { { assert snapshot(process.out).match() } ) } - } + test("stub") { + options "-stub" + when { + process { + """ + input[0] = Channel.of([ + [ id:'test' ], // meta map + file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/bedgraph/test.bedgraph", checkIfExists: true) + ]) + input[1] = Channel.of(file(params.modules_testdata_base_path + "genomics/sarscov2/genome/genome.sizes", checkIfExists: true)) + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } } diff --git a/modules/nf-core/ucsc/bedgraphtobigwig/tests/main.nf.test.snap b/modules/nf-core/ucsc/bedgraphtobigwig/tests/main.nf.test.snap index 6b0de0ff..7c731f65 100644 --- a/modules/nf-core/ucsc/bedgraphtobigwig/tests/main.nf.test.snap +++ b/modules/nf-core/ucsc/bedgraphtobigwig/tests/main.nf.test.snap @@ -1,4 +1,37 @@ { + "stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.bigWig:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,db26514184acfdf220bb2f061382cf8c" + ], + "bigwig": [ + [ + { + "id": "test" + }, + "test.bigWig:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,db26514184acfdf220bb2f061382cf8c" + ] + } + ], + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-10-18T10:47:58.558813949" + }, "Should run without failures": { "content": [ { @@ -11,7 +44,7 @@ ] ], "1": [ - "versions.yml:md5,93b027527145a243903a3c687c3453b8" + "versions.yml:md5,db26514184acfdf220bb2f061382cf8c" ], "bigwig": [ [ @@ -22,10 +55,14 @@ ] ], "versions": [ - "versions.yml:md5,93b027527145a243903a3c687c3453b8" + "versions.yml:md5,db26514184acfdf220bb2f061382cf8c" ] } ], - "timestamp": "2023-10-18T04:06:47.826602" + "meta": { + "nf-test": "0.9.0", + "nextflow": "24.04.4" + }, + "timestamp": "2024-10-18T10:47:36.476844229" } } \ No newline at end of file diff --git a/modules/nf-core/ucsc/bedtobigbed/environment.yml b/modules/nf-core/ucsc/bedtobigbed/environment.yml new file mode 100644 index 00000000..62670c4e --- /dev/null +++ b/modules/nf-core/ucsc/bedtobigbed/environment.yml @@ -0,0 +1,5 @@ +channels: + - conda-forge + - bioconda +dependencies: + - bioconda::ucsc-bedtobigbed=447 diff --git a/modules/nf-core/ucsc/bedtobigbed/main.nf b/modules/nf-core/ucsc/bedtobigbed/main.nf index 1e40375d..9902560f 100644 --- a/modules/nf-core/ucsc/bedtobigbed/main.nf +++ b/modules/nf-core/ucsc/bedtobigbed/main.nf @@ -3,10 +3,10 @@ process UCSC_BEDTOBIGBED { label 'process_single' // WARN: Version information not provided by tool on CLI. Please update version string below when bumping container versions. - conda "bioconda::ucsc-bedtobigbed=377" + conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/ucsc-bedtobigbed:377--ha8a8165_3' : - 'biocontainers/ucsc-bedtobigbed:377--ha8a8165_3' }" + 'https://depot.galaxyproject.org/singularity/ucsc-bedtobigbed:447--h954228d_0' : + 'biocontainers/ucsc-bedtobigbed:447--h954228d_0' }" input: tuple val(meta), path(bed) @@ -24,7 +24,7 @@ process UCSC_BEDTOBIGBED { def args = task.ext.args ?: '' def prefix = task.ext.prefix ?: "${meta.id}" def as_option = autosql ? "-as=${autosql}" : "" - def VERSION = '377' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. + def VERSION = '447' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. """ bedToBigBed \\ $bed \\ @@ -41,7 +41,7 @@ process UCSC_BEDTOBIGBED { stub: def prefix = task.ext.prefix ?: "${meta.id}" - def VERSION = '377' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. + def VERSION = '447' // WARN: Version information not provided by tool on CLI. Please update this string when bumping container versions. """ touch ${prefix}.bigBed diff --git a/modules/nf-core/ucsc/bedtobigbed/meta.yml b/modules/nf-core/ucsc/bedtobigbed/meta.yml index 8e9e5291..6afc3148 100755 --- a/modules/nf-core/ucsc/bedtobigbed/meta.yml +++ b/modules/nf-core/ucsc/bedtobigbed/meta.yml @@ -12,40 +12,42 @@ tools: homepage: http://hgdownload.cse.ucsc.edu/admin/exe/ documentation: https://genome.ucsc.edu/goldenPath/help/bigBed.html licence: ["varies; see http://genome.ucsc.edu/license"] - + identifier: "" input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - bed: - type: file - description: bed file - pattern: "*.{bed}" - - sizes: - type: file - description: chromosome sizes file - pattern: "*.{sizes}" - - autosql: - type: file - description: autoSql file to describe the columns of the BED file - pattern: "*.{as}" - + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - bed: + type: file + description: bed file + pattern: "*.{bed}" + - - sizes: + type: file + description: chromosome sizes file + pattern: "*.{sizes}" + - - autosql: + type: file + description: autoSql file to describe the columns of the BED file + pattern: "*.{as}" output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" - bigbed: - type: file - description: bigBed file - pattern: "*.{bigBed}" - + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.bigBed": + type: file + description: bigBed file + pattern: "*.{bigBed}" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@drpatelh" +maintainers: + - "@drpatelh" diff --git a/modules/nf-core/windowmasker/mkcounts/environment.yml b/modules/nf-core/windowmasker/mkcounts/environment.yml index 15887425..777e097e 100644 --- a/modules/nf-core/windowmasker/mkcounts/environment.yml +++ b/modules/nf-core/windowmasker/mkcounts/environment.yml @@ -1,7 +1,5 @@ -name: windowmasker_mkcounts channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::blast=2.14.0 + - bioconda::blast=2.15.0 diff --git a/modules/nf-core/windowmasker/mkcounts/main.nf b/modules/nf-core/windowmasker/mkcounts/main.nf index 6bfd175e..406f7761 100644 --- a/modules/nf-core/windowmasker/mkcounts/main.nf +++ b/modules/nf-core/windowmasker/mkcounts/main.nf @@ -4,8 +4,8 @@ process WINDOWMASKER_MKCOUNTS { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/blast:2.14.0--h7d5a4b4_1': - 'biocontainers/blast:2.14.0--h7d5a4b4_1' }" + 'https://depot.galaxyproject.org/singularity/blast:2.15.0--pl5321h6f7f691_1': + 'biocontainers/blast:2.15.0--pl5321h6f7f691_1' }" input: tuple val(meta), path(ref) @@ -21,11 +21,11 @@ process WINDOWMASKER_MKCOUNTS { def args = task.ext.args ?: "" def prefix = task.ext.prefix ?: "${meta.id}" - def memory = 3072 + def memory = 3072 if (!task.memory) { log.info '[WINDOWMASKER: MK_COUNTS] Available memory not known - defaulting to 3GB. Specify process memory requirements to change this.' } else { - memory = (task.memory.toMega()).intValue() + memory = (task.memory.toMega()).intValue() } """ diff --git a/modules/nf-core/windowmasker/mkcounts/meta.yml b/modules/nf-core/windowmasker/mkcounts/meta.yml index 436ed7a5..825a0674 100644 --- a/modules/nf-core/windowmasker/mkcounts/meta.yml +++ b/modules/nf-core/windowmasker/mkcounts/meta.yml @@ -11,31 +11,32 @@ tools: homepage: https://github.com/ncbi/ncbi-cxx-toolkit-public documentation: https://ncbi.github.io/cxx-toolkit/ licence: ["MIT"] + identifier: biotools:windowmasker input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - ref: - type: file - description: An input nucleotide fasta file. + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ref: + type: file + description: An input nucleotide fasta file. output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - intervals: - type: file - description: | - An output file containing genomic locations of low - complexity and highly repetitive regions - pattern: "${prefix}.txt" + - counts: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - "*.txt": + type: file + description: A file containing frequency counts of repetitive units. + pattern: "*.txt" - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@DLBPointon" maintainers: diff --git a/modules/nf-core/windowmasker/mkcounts/tests/main.nf.test b/modules/nf-core/windowmasker/mkcounts/tests/main.nf.test new file mode 100644 index 00000000..bf53d7fa --- /dev/null +++ b/modules/nf-core/windowmasker/mkcounts/tests/main.nf.test @@ -0,0 +1,59 @@ +nextflow_process { + + name "Test Process WINDOWMASKER_MKCOUNTS" + script "../main.nf" + process "WINDOWMASKER_MKCOUNTS" + + tag "modules" + tag "modules_nfcore" + tag "windowmasker" + tag "windowmasker/mkcounts" + + test("sarscov2_fasta") { + + when { + params { + // define parameters here. Example: + // outdir = "tests/results" + } + process { + """ + input[0] = [ + [id: "test" ], + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)] + ] + """ + } + } + + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + test("sarscov2_fasta_stub") { + + options '-stub' + + when { + process { + """ + input[0] = [ + [id: "test" ], + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)] + ] + """ + } + } + then { + assertAll ( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + +} diff --git a/modules/nf-core/windowmasker/mkcounts/tests/main.nf.test.snap b/modules/nf-core/windowmasker/mkcounts/tests/main.nf.test.snap new file mode 100644 index 00000000..cae2d306 --- /dev/null +++ b/modules/nf-core/windowmasker/mkcounts/tests/main.nf.test.snap @@ -0,0 +1,60 @@ +{ + "sarscov2_fasta": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.txt:md5,5f5d7e926fdf13b0c57651f962cc1253" + ] + ], + "1": [ + "versions.yml:md5,57ae356f69298e25eb5d070900865cf2" + ], + "counts": [ + [ + { + "id": "test" + }, + "test.txt:md5,5f5d7e926fdf13b0c57651f962cc1253" + ] + ], + "versions": [ + "versions.yml:md5,57ae356f69298e25eb5d070900865cf2" + ] + } + ], + "timestamp": "2024-02-15T13:29:58.837482" + }, + "sarscov2_fasta_stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.txt:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + "versions.yml:md5,57ae356f69298e25eb5d070900865cf2" + ], + "counts": [ + [ + { + "id": "test" + }, + "test.txt:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions": [ + "versions.yml:md5,57ae356f69298e25eb5d070900865cf2" + ] + } + ], + "timestamp": "2024-02-15T13:30:07.618636" + } +} \ No newline at end of file diff --git a/modules/nf-core/windowmasker/mkcounts/tests/nextflow.config b/modules/nf-core/windowmasker/mkcounts/tests/nextflow.config new file mode 100644 index 00000000..65fc1910 --- /dev/null +++ b/modules/nf-core/windowmasker/mkcounts/tests/nextflow.config @@ -0,0 +1,5 @@ +process { + withName: WINDOWMASKER_MKCOUNTS { + ext.args = "" + } +} \ No newline at end of file diff --git a/modules/nf-core/windowmasker/mkcounts/tests/tags.yml b/modules/nf-core/windowmasker/mkcounts/tests/tags.yml new file mode 100644 index 00000000..95c67635 --- /dev/null +++ b/modules/nf-core/windowmasker/mkcounts/tests/tags.yml @@ -0,0 +1,2 @@ +windowmasker/mkcounts: + - "modules/nf-core/windowmasker/mkcounts/**" diff --git a/modules/nf-core/windowmasker/ustat/environment.yml b/modules/nf-core/windowmasker/ustat/environment.yml index a97fdd9d..777e097e 100644 --- a/modules/nf-core/windowmasker/ustat/environment.yml +++ b/modules/nf-core/windowmasker/ustat/environment.yml @@ -1,7 +1,5 @@ -name: windowmasker_ustat channels: - conda-forge - bioconda - - defaults dependencies: - - bioconda::blast=2.14.0 + - bioconda::blast=2.15.0 diff --git a/modules/nf-core/windowmasker/ustat/main.nf b/modules/nf-core/windowmasker/ustat/main.nf index 2cc3df63..7a7d29f6 100644 --- a/modules/nf-core/windowmasker/ustat/main.nf +++ b/modules/nf-core/windowmasker/ustat/main.nf @@ -4,8 +4,8 @@ process WINDOWMASKER_USTAT { conda "${moduleDir}/environment.yml" container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/blast:2.14.0--h7d5a4b4_1': - 'biocontainers/blast:2.14.0--h7d5a4b4_1' }" + 'https://depot.galaxyproject.org/singularity/blast:2.15.0--pl5321h6f7f691_1': + 'biocontainers/blast:2.15.0--pl5321h6f7f691_1' }" input: tuple val(meta) , path(counts) diff --git a/modules/nf-core/windowmasker/ustat/meta.yml b/modules/nf-core/windowmasker/ustat/meta.yml index 6a07c935..bc51a934 100644 --- a/modules/nf-core/windowmasker/ustat/meta.yml +++ b/modules/nf-core/windowmasker/ustat/meta.yml @@ -1,5 +1,6 @@ name: windowmasker_ustat -description: A program to take a counts file and creates a file of genomic co-ordinates to be masked. +description: A program to take a counts file and creates a file of genomic co-ordinates + to be masked. keywords: - fasta - interval @@ -11,39 +12,39 @@ tools: homepage: https://github.com/ncbi/ncbi-cxx-toolkit-public documentation: https://ncbi.github.io/cxx-toolkit/ licence: ["MIT"] + identifier: biotools:windowmasker input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test' ] - - counts: - type: file - description: Contains count data of repetitive regions. - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - ref: - type: file - description: An input nucleotide fasta file. + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test' ] + - counts: + type: file + description: Contains count data of repetitive regions. + - - meta2: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ref: + type: file + description: An input nucleotide fasta file. output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - wm_intervals: - type: file - description: | - An output file containing genomic locations of low - complexity and highly repetitive regions - pattern: "${output}" + - intervals: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test', single_end:false ] + - ${output}: + type: file + description: intervals - versions: - type: file - description: File containing software versions - pattern: "versions.yml" + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" authors: - "@DLBPointon" maintainers: diff --git a/modules/nf-core/windowmasker/ustat/tests/main.nf.test b/modules/nf-core/windowmasker/ustat/tests/main.nf.test new file mode 100644 index 00000000..6e02c9c1 --- /dev/null +++ b/modules/nf-core/windowmasker/ustat/tests/main.nf.test @@ -0,0 +1,66 @@ +nextflow_process { + + name "Test Process WINDOWMASKER_USTAT" + script "../main.nf" + process "WINDOWMASKER_USTAT" + config "./nextflow.config" + + tag "modules" + tag "modules_nfcore" + tag "windowmasker" + tag "windowmasker/ustat" + tag "windowmasker/mkcounts" + + + setup { + run("WINDOWMASKER_MKCOUNTS") { + script "../../mkcounts/main.nf" + process { + """ + input[0] = [ + [id: "test" ], + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)] + ] + """ + } + } + } + + test("sarscov2_fasta") { + when { + process { + """ + input[0] = WINDOWMASKER_MKCOUNTS.out.counts + input[1] = [ + [id: "test" ], + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)] + ] + """ + } + } + then { + assert process.success + assert snapshot(process.out).match() + } + } + + test("sarscov2_fasta_stub") { + when { + process { + """ + input[0] = WINDOWMASKER_MKCOUNTS.out.counts + input[1] = [ + [id: "test" ], + [ file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)] + ] + """ + } + } + then { + assert process.success + assert snapshot(process.out).match() + } + + } + +} diff --git a/modules/nf-core/windowmasker/ustat/tests/main.nf.test.snap b/modules/nf-core/windowmasker/ustat/tests/main.nf.test.snap new file mode 100644 index 00000000..79d3d82d --- /dev/null +++ b/modules/nf-core/windowmasker/ustat/tests/main.nf.test.snap @@ -0,0 +1,60 @@ +{ + "sarscov2_fasta": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.interval:md5,c91346601564ab88cbb0f913881d05e2" + ] + ], + "1": [ + "versions.yml:md5,d43f04bb181ac80da9ec79d9b49131cf" + ], + "intervals": [ + [ + { + "id": "test" + }, + "test.interval:md5,c91346601564ab88cbb0f913881d05e2" + ] + ], + "versions": [ + "versions.yml:md5,d43f04bb181ac80da9ec79d9b49131cf" + ] + } + ], + "timestamp": "2024-02-15T14:19:12.033774" + }, + "sarscov2_fasta_stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "test.interval:md5,c91346601564ab88cbb0f913881d05e2" + ] + ], + "1": [ + "versions.yml:md5,d43f04bb181ac80da9ec79d9b49131cf" + ], + "intervals": [ + [ + { + "id": "test" + }, + "test.interval:md5,c91346601564ab88cbb0f913881d05e2" + ] + ], + "versions": [ + "versions.yml:md5,d43f04bb181ac80da9ec79d9b49131cf" + ] + } + ], + "timestamp": "2024-02-15T14:19:21.850526" + } +} \ No newline at end of file diff --git a/modules/nf-core/windowmasker/ustat/tests/nextflow.config b/modules/nf-core/windowmasker/ustat/tests/nextflow.config new file mode 100644 index 00000000..00b63c45 --- /dev/null +++ b/modules/nf-core/windowmasker/ustat/tests/nextflow.config @@ -0,0 +1,5 @@ +process { + withName: 'test_windowmasker_ustat:WINDOWMASKER_USTAT' { + ext.args = "-dust true -outfmt interval" + } +} \ No newline at end of file diff --git a/modules/nf-core/windowmasker/ustat/tests/tags.yml b/modules/nf-core/windowmasker/ustat/tests/tags.yml new file mode 100644 index 00000000..28c74ca9 --- /dev/null +++ b/modules/nf-core/windowmasker/ustat/tests/tags.yml @@ -0,0 +1,2 @@ +windowmasker/ustat: + - "modules/nf-core/windowmasker/ustat/**" diff --git a/modules/nf-core/yahs/environment.yml b/modules/nf-core/yahs/environment.yml new file mode 100644 index 00000000..cb2acec8 --- /dev/null +++ b/modules/nf-core/yahs/environment.yml @@ -0,0 +1,5 @@ +channels: + - conda-forge + - bioconda +dependencies: + - bioconda::yahs=1.2a.2 diff --git a/modules/nf-core/yahs/main.nf b/modules/nf-core/yahs/main.nf new file mode 100644 index 00000000..6475fd91 --- /dev/null +++ b/modules/nf-core/yahs/main.nf @@ -0,0 +1,51 @@ +process YAHS { + tag "$meta.id" + label 'process_high' + + conda "${moduleDir}/environment.yml" + container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? + 'https://depot.galaxyproject.org/singularity/yahs:1.2--he4a0461_1': + 'biocontainers/yahs:1.2--he4a0461_1' }" + + input: + tuple val(meta), path(hic_map) + path fasta + path fai + + output: + tuple val(meta), path("*scaffolds_final.fa") , emit: scaffolds_fasta, optional: true + tuple val(meta), path("*scaffolds_final.agp"), emit: scaffolds_agp, optional: true + tuple val(meta), path("*bin") , emit: binary + path "versions.yml" , emit: versions + + when: + task.ext.when == null || task.ext.when + + script: + def args = task.ext.args ?: '' + def prefix = task.ext.prefix ?: "${meta.id}" + + """ + yahs $args \\ + -o $prefix \\ + $fasta \\ + $hic_map + + cat <<-END_VERSIONS > versions.yml + "${task.process}": + yahs: \$(yahs --version 2>&1) + END_VERSIONS + """ + + stub: + """ + touch ${prefix}_scaffold_final.fa + touch ${prefix}_scaffolds_final.agp + touch ${prefix}.bin + + cat <<-END_VERSIONS > versions.yml + "${task.process}": + yahs: \$(yahs --version 2>&1) + END_VERSIONS + """ +} diff --git a/modules/nf-core/yahs/meta.yml b/modules/nf-core/yahs/meta.yml new file mode 100644 index 00000000..38869045 --- /dev/null +++ b/modules/nf-core/yahs/meta.yml @@ -0,0 +1,78 @@ +name: "yahs" +description: Performs assembly scaffolding using YaHS +keywords: + - scaffolding + - assembly + - yahs + - hic +tools: + - "yahs": + description: "YaHS, yet another Hi-C scaffolding tool." + homepage: "https://github.com/c-zhou/yahs" + documentation: "https://github.com/c-zhou/yahs" + tool_dev_url: "https://github.com/c-zhou/yahs" + doi: "10.1093/bioinformatics/btac808" + licence: ["MIT"] + identifier: biotools:yahs +input: + # Only when we have meta + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test' ] + - hic_map: + type: file + description: BED file containing coordinates of read alignments + pattern: "*.{bed,bam,bin}" + - - fasta: + type: file + description: FASTA reference file + pattern: "*.{fasta,fa}" + - - fai: + type: file + description: index of the reference file + pattern: "*.{fai}" +output: + #Only when we have meta + - scaffolds_fasta: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test' ] + - "*scaffolds_final.fa": + type: file + description: FASTA file with resulting contigs + pattern: "*scaffolds_final.fa" + - scaffolds_agp: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test' ] + - "*scaffolds_final.agp": + type: file + description: AGP file containing contigs placing coordinates + pattern: "*scaffolds_final.agp" + - binary: + - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test' ] + - "*bin": + type: file + description: BIN file with alignment results of Hi-C reads to the contigs in + internal YaHS binary format + pattern: "*bin" + - versions: + - versions.yml: + type: file + description: File containing software versions + pattern: "versions.yml" +authors: + - "@ksenia-krasheninnikova" +maintainers: + - "@ksenia-krasheninnikova" + - "@yy5" diff --git a/modules/nf-core/yahs/tests/main.nf.test b/modules/nf-core/yahs/tests/main.nf.test new file mode 100644 index 00000000..ee53a110 --- /dev/null +++ b/modules/nf-core/yahs/tests/main.nf.test @@ -0,0 +1,54 @@ + +nextflow_process { + + name "Test Process YAHS" + script "../main.nf" + process "YAHS" + config "./nextflow.config" + + tag "modules" + tag "modules_nfcore" + tag "yahs" + tag "samtools/view" + + setup { + run("SAMTOOLS_VIEW") { + script "../../samtools/view" + process { + """ + input[0] = Channel.of([ + [ id:'test', single_end:false ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/cram/test.paired_end.sorted.cram', checkIfExists: true), + [] + ]) + input[1] = Channel.of([ + [ id:'genome' ], // meta map + file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + ]) + input[2] = [] + """ + } + } + } + + test("homo_sapiens-bam-fasta-fai") { + + when { + process { + """ + input[0] = SAMTOOLS_VIEW.out.bam + input[1] = file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta', checkIfExists: true) + input[2] = file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome.fasta.fai', checkIfExists: true) + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + } + +} diff --git a/modules/nf-core/yahs/tests/main.nf.test.snap b/modules/nf-core/yahs/tests/main.nf.test.snap new file mode 100644 index 00000000..c62caa91 --- /dev/null +++ b/modules/nf-core/yahs/tests/main.nf.test.snap @@ -0,0 +1,37 @@ +{ + "homo_sapiens-bam-fasta-fai": { + "content": [ + { + "0": [ + + ], + "1": [ + + ], + "2": [ + + ], + "3": [ + + ], + "binary": [ + + ], + "scaffolds_agp": [ + + ], + "scaffolds_fasta": [ + + ], + "versions": [ + + ] + } + ], + "meta": { + "nf-test": "0.8.4", + "nextflow": "24.04.2" + }, + "timestamp": "2024-09-26T22:35:24.103271222" + } +} \ No newline at end of file diff --git a/modules/nf-core/yahs/tests/nextflow.config b/modules/nf-core/yahs/tests/nextflow.config new file mode 100644 index 00000000..1691f241 --- /dev/null +++ b/modules/nf-core/yahs/tests/nextflow.config @@ -0,0 +1,6 @@ +process { + withName: 'test_yahs:YAHS' { + ext.prefix = { "${meta.id}" } + ext.args = '--no-contig-ec' + } +} diff --git a/nextflow.config b/nextflow.config index af9a14b9..2f1bcd14 100755 --- a/nextflow.config +++ b/nextflow.config @@ -26,6 +26,8 @@ params { validate_params = true show_hidden_params = false schema_ignore_params = 'genomes' + binfile = false + steps = 'ALL' // Config options custom_config_version = 'master' @@ -195,7 +197,7 @@ manifest { description = """A pipeline to generate supplemental data for genome curation""" mainScript = 'main.nf' nextflowVersion = '!>=22.10.1' - version = '1.1.1' + version = '1.2.0' doi = '10.5281/zenodo.10047653' } @@ -253,4 +255,4 @@ def round_up ( decimal_places, file, multiplier ){ } else { return dived * multiplier } -} \ No newline at end of file +} diff --git a/nextflow_schema.json b/nextflow_schema.json index 00e2ce44..af670eec 100755 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -35,6 +35,18 @@ "default": false, "fa_icon": "fas fa-check" }, + "binfile": { + "type": "boolean", + "format": "boolean", + "description": "If false then turn off bin file generation", + "default": false, + "fa_icon": "fas fa-check" + }, + "steps": { + "type": "string", + "description": "A csv list of steps to skip", + "fa_icon": "fas fa-folder-open" + }, "email": { "type": "string", "description": "Email address for completion summary.", diff --git a/subworkflows/local/busco_annotation.nf b/subworkflows/local/busco_annotation.nf index 974738f6..b3a4e972 100755 --- a/subworkflows/local/busco_annotation.nf +++ b/subworkflows/local/busco_annotation.nf @@ -8,7 +8,7 @@ // // MODULE IMPORT BLOCK // -include { BUSCO } from '../../modules/nf-core/busco/main' +include { BUSCO_BUSCO } from '../../modules/nf-core/busco/busco/main' include { UCSC_BEDTOBIGBED } from '../../modules/nf-core/ucsc/bedtobigbed/main' include { BEDTOOLS_SORT } from '../../modules/nf-core/bedtools/sort/main' include { EXTRACT_BUSCOGENE } from '../../modules/local/extract_buscogene' @@ -38,16 +38,15 @@ workflow BUSCO_ANNOTATION { // MODULE: RUN BUSCO TO OBTAIN FULL_TABLE.CSV // EMITS FULL_TABLE.CSV // - BUSCO ( + BUSCO_BUSCO ( reference_tuple, ch_busco_mode, lineageinfo, lineagespath, [] ) - ch_versions = ch_versions.mix( BUSCO.out.versions.first() ) - - ch_grab = GrabFiles( BUSCO.out.busco_dir ) + ch_versions = ch_versions.mix(BUSCO_BUSCO.out.versions.first()) + ch_grab = GrabFiles(BUSCO_BUSCO.out.busco_dir) // // MODULE: EXTRACT THE BUSCO GENES FOUND IN REFERENCE @@ -92,8 +91,8 @@ workflow BUSCO_ANNOTATION { // LOGIC: AGGREGATE DATA AND SORT BRANCH ON CLASS // lineageinfo - .combine( BUSCO.out.busco_dir ) - .combine( ancestral_table ) + .combine(BUSCO_BUSCO.out.busco_dir) + .combine(ancestral_table) .branch { lep: it[0].split('_')[0] == "lepidoptera" general: it[0].split('_')[0] != "lepidoptera" diff --git a/subworkflows/local/gene_alignment.nf b/subworkflows/local/gene_alignment.nf index 3269c201..1936bc99 100755 --- a/subworkflows/local/gene_alignment.nf +++ b/subworkflows/local/gene_alignment.nf @@ -18,9 +18,7 @@ workflow GENE_ALIGNMENT { dot_genome // Channel: [ val(meta), path(file) ] reference_tuple // Channel: [ val(meta), path(file) ] reference_index // Channel: [ val(meta), path(file) ] - alignment_datadir // Channel: val(geneset_dir) alignment_genesets // Channel: val(geneset_id) - alignment_common // Channel: val(common_name) // Not yet in use intron_size // Channel: val(50k) as_files // Channel: [ val(meta), path(file) ] @@ -50,11 +48,9 @@ workflow GENE_ALIGNMENT { // SUBWORKFLOW // ch_data - .combine( alignment_datadir ) - .combine( assembly_class ) .map { - ch_org, data_dir, classT -> - file("${data_dir}${classT}/csv_data/${ch_org}-data.csv") + geneset_path -> + file(geneset_path) } .splitCsv( header: true, sep:',') .map( row -> diff --git a/subworkflows/local/hic_bamtobed.nf b/subworkflows/local/hic_bamtobed.nf index 432ae1b7..d6e8eb34 100755 --- a/subworkflows/local/hic_bamtobed.nf +++ b/subworkflows/local/hic_bamtobed.nf @@ -1,6 +1,6 @@ #!/usr/bin/env nextflow -// This subworkflow takes converts .bam to .bed for the hic_mapping subworkflow. +// This subworkflow takes converts .bam to .bed for the hic_mapping subworkflow. // It runs markdup, sort and get paired contacts. // Input - Assembled genomic fasta file, .bam file // Output - sorted .bed and paired contact .bed @@ -21,23 +21,12 @@ workflow HIC_BAMTOBED { main: ch_versions = Channel.empty() - // - // LOGIC: PREPARE MARKDUP INPUT - // - bam_file - .combine( reference_tuple ) - .multiMap { meta_bam, bam_file, meta_ref, ref -> - bam : tuple(meta_bam, bam_file ) - reference : ref - } - .set { markdup_input } - // // MODULE: MERGE POSITION SORTED BAM FILES AND MARK DUPLICATES // SAMTOOLS_MARKDUP ( - markdup_input.bam, - markdup_input.reference + bam_file, + reference_tuple ) ch_versions = ch_versions.mix ( SAMTOOLS_MARKDUP.out.versions ) @@ -52,8 +41,8 @@ workflow HIC_BAMTOBED { // // MODULE: GENERATE CONTACT PAIRS // - GET_PAIRED_CONTACT_BED( - BAMTOBED_SORT.out.sorted_bed + GET_PAIRED_CONTACT_BED( + BAMTOBED_SORT.out.sorted_bed ) ch_versions = ch_versions.mix( GET_PAIRED_CONTACT_BED.out.versions ) diff --git a/subworkflows/local/hic_mapping.nf b/subworkflows/local/hic_mapping.nf index e379f49a..f6007993 100755 --- a/subworkflows/local/hic_mapping.nf +++ b/subworkflows/local/hic_mapping.nf @@ -22,6 +22,7 @@ include { HIC_BAMTOBED as HIC_BAMTOBED_COOLER } from '../../subworkf include { HIC_BAMTOBED as HIC_BAMTOBED_JUICER } from '../../subworkflows/local/hic_bamtobed' include { HIC_MINIMAP2 } from '../../subworkflows/local/hic_minimap2' include { HIC_BWAMEM2 } from '../../subworkflows/local/hic_bwamem2' +include { YAHS } from '../../modules/nf-core/yahs/main' workflow HIC_MAPPING { take: @@ -123,7 +124,25 @@ workflow HIC_MAPPING { genome_file ) } - .set { pretext_input } + .set {pretext_input} + + if ( params.binfile == true ) { + + // + // LOGIC: MAKE YAHS INPUT + // + ref_yahs.map { meta, ref -> ref }.set{ch_ref} + reference_index.map { meta, fai -> fai }.set{ch_fai} + + // + // MODULE: RUN YAHS TO GENERATE ALIGNMENT BIN FILE + // + YAHS ( + mergedbam, + ch_ref, + ch_fai + ) + } // // MODULE: GENERATE PRETEXT MAP FROM MAPPED BAM FOR LOW RES diff --git a/subworkflows/local/kmer.nf b/subworkflows/local/kmer.nf index 2c98f955..55c78725 100755 --- a/subworkflows/local/kmer.nf +++ b/subworkflows/local/kmer.nf @@ -66,7 +66,12 @@ workflow KMER { // // MODULE: USE KMER HISTOGRAM TO PRODUCE SPECTRA GRAPH // - MERQURYFK_MERQURYFK ( ch_merq ) + MERQURYFK_MERQURYFK ( + ch_merq, + [], + [] + ) + ch_versions = ch_versions.mix( MERQURYFK_MERQURYFK.out.versions.first() ) emit: diff --git a/subworkflows/local/nuc_alignments.nf b/subworkflows/local/nuc_alignments.nf index cff5235f..d83eb943 100755 --- a/subworkflows/local/nuc_alignments.nf +++ b/subworkflows/local/nuc_alignments.nf @@ -52,10 +52,11 @@ workflow NUC_ALIGNMENTS { false ) } - .multiMap { meta, nuc_file, reference, bool_1, bool_2, bool_3, bool_4 -> - nuc : tuple( meta, nuc_file) - ref : reference + .multiMap {meta, nuc_file, reference, bool_1, bool_2, bool_3, bool_4 -> + nuc : tuple(meta, nuc_file) + ref : tuple(meta, reference) bool_bam_output : bool_1 + val_bam_output : "bai" bool_cigar_paf : bool_2 bool_cigar_bam : bool_3 bool_bedfile : bool_4 @@ -70,6 +71,7 @@ workflow NUC_ALIGNMENTS { formatted_input.nuc, formatted_input.ref, formatted_input.bool_bam_output, + formatted_input.val_bam_output, formatted_input.bool_cigar_paf, formatted_input.bool_cigar_bam, formatted_input.bool_bedfile diff --git a/subworkflows/local/read_coverage.nf b/subworkflows/local/read_coverage.nf index 4da5b0d9..2eb61ec5 100755 --- a/subworkflows/local/read_coverage.nf +++ b/subworkflows/local/read_coverage.nf @@ -66,8 +66,9 @@ workflow READ_COVERAGE { pre_minimap_input .multiMap { meta, reads_path, ref, bam_output, cigar_paf, cigar_bam, bed_output, reads_type -> read_tuple : tuple( meta, reads_path) - ref : ref + ref : tuple( meta, ref) bool_bam_ouput : bam_output + val_bam_index : "bai" bool_cigar_paf : cigar_paf bool_cigar_bam : cigar_bam bool_bed_output : bed_output @@ -81,6 +82,7 @@ workflow READ_COVERAGE { minimap_input.read_tuple, minimap_input.ref, minimap_input.bool_bam_ouput, + minimap_input.val_bam_index, minimap_input.bool_cigar_paf, minimap_input.bool_cigar_bam, minimap_input.bool_bed_output @@ -109,6 +111,9 @@ workflow READ_COVERAGE { ) ch_versions = ch_versions.mix( CAT_CAT.out.versions ) + // + // MODULE: SORT THE MERGED BED FILE INTO CHROMOSOME-LOCATION ORDER + // GNU_SORT_BED( CAT_CAT.out.file_out ) @@ -132,7 +137,6 @@ workflow READ_COVERAGE { } .set { genomecov_input } - // // MODULE: Genome2Cov @@ -140,7 +144,8 @@ workflow READ_COVERAGE { BEDTOOLS_GENOMECOV( genomecov_input.input_tuple, genomecov_input.dot_genome, - genomecov_input.file_suffix + genomecov_input.file_suffix, + false ) ch_versions = ch_versions.mix(BEDTOOLS_GENOMECOV.out.versions) diff --git a/subworkflows/local/selfcomp.nf b/subworkflows/local/selfcomp.nf index 25022c77..bfb10848 100755 --- a/subworkflows/local/selfcomp.nf +++ b/subworkflows/local/selfcomp.nf @@ -12,7 +12,7 @@ include { BEDTOOLS_SORT } from '../../modules/nf-core/bedtools/ include { SELFCOMP_SPLITFASTA } from '../../modules/local/selfcomp_splitfasta' include { SELFCOMP_MUMMER2BED } from '../../modules/local/selfcomp_mummer2bed' include { SELFCOMP_MAPIDS } from '../../modules/local/selfcomp_mapids' -include { CHUNKFASTA } from '../../modules/local/chunkfasta' +include { SEQKIT_SPLIT } from '../../modules/local/seqkit/split/main' include { CAT_CAT } from '../../modules/nf-core/cat/cat/main' include { SELFCOMP_ALIGNMENTBLOCKS } from '../../modules/local/selfcomp_alignmentblocks' include { CONCATBLOCKS } from '../../modules/local/concatblocks' @@ -57,28 +57,28 @@ workflow SELFCOMP { // MODULE: SPLIT REFERENCE FILE INTO 1GB CHUNKS // THIS IS THE QUERY, AND REFERENCE IF GENOME.size() > 1GB // - CHUNKFASTA( + SEQKIT_SPLIT( SELFCOMP_SPLITFASTA.out.fa, chunk_number ) - ch_versions = ch_versions.mix( CHUNKFASTA.out.versions ) + ch_versions = ch_versions.mix(SEQKIT_SPLIT.out.versions) // // LOGIC: STRIP META FROM QUERY, AND COMBINE WITH REFERENCE FILE // THIS LEAVES US WITH n=( REFERENCE + QUERY) IF GENOME.SIZE() < 1GB // OR n=((REFERENCE / 1E9) * (REFENCE / 1E9)) IF GENOME.SIZE() > 1GB // - CHUNKFASTA.out.fasta - .map{ meta, query -> + SEQKIT_SPLIT.out.fasta + .map{meta, query -> query } - .collect() // Collect any output from CHUNKFASTA - .map { it -> + .collect() // Collect any output from SEQKIT_SPLIT + .map {it -> tuple( [ len: it.size() ], // Calc length of list it ) } - .set { len_ch } // tap out to preserve length of CHUNKFASTA list + .set {len_ch} // tap out to preserve length of SEQKIT_SPLIT list len_ch // tap swapped with set as tap stops pipeline completion .map { meta, files -> @@ -216,4 +216,3 @@ workflow SELFCOMP { ch_bigbed = UCSC_BEDTOBIGBED.out.bigbed versions = ch_versions.ifEmpty(null) } - diff --git a/subworkflows/local/synteny.nf b/subworkflows/local/synteny.nf index 33c0875b..8658d469 100755 --- a/subworkflows/local/synteny.nf +++ b/subworkflows/local/synteny.nf @@ -8,26 +8,33 @@ include { MINIMAP2_ALIGN } from '../../modules/nf-core/minimap2/align/mai workflow SYNTENY { take: reference_tuple // Channel: tuple [ val(meta), path(file) ] - synteny_path // Channel: val(meta) + synteny_paths // Channel: val(meta) main: ch_versions = Channel.empty() + ch_data = synteny_paths + .splitCsv() + .flatten() + // // LOGIC: PULL SYNTENIC GENOMES FROM DIRECTORY STRUCTURE // AND PARSE INTO CHANNEL PER GENOME // - reference_tuple - .combine( synteny_path ) - .map { meta, reference, dir_path -> - file("${dir_path}${meta.class}/*.fasta") + ch_data + .map{synteny_path -> + file(synteny_path) } - .flatten() - .combine( reference_tuple ) - .multiMap { syntenic_ref, meta, ref -> - syntenic_tuple : tuple( meta, syntenic_ref ) - reference_fa : ref + .combine(reference_tuple) + .multiMap{syntenic_ref, meta, ref -> + syntenic_tuple : tuple([ id: syntenic_ref.toString().split('/')[-1].split('.fasta')[0], + class: meta.class, + project_type: meta.project_type + ], + syntenic_ref) + reference_fa : tuple( meta, ref) bool_bam_output : false + val_bam_index : "bai" bool_cigar_paf : true bool_cigar_bam : false bool_bedfile : false @@ -42,6 +49,7 @@ workflow SYNTENY { mm_input.syntenic_tuple, mm_input.reference_fa, mm_input.bool_bam_output, + mm_input.val_bam_index, mm_input.bool_cigar_paf, mm_input.bool_cigar_bam, mm_input.bool_bedfile, diff --git a/subworkflows/local/yaml_input.nf b/subworkflows/local/yaml_input.nf index e3ad75da..0ce6ee6c 100755 --- a/subworkflows/local/yaml_input.nf +++ b/subworkflows/local/yaml_input.nf @@ -81,11 +81,9 @@ workflow YAML_INPUT { group .alignment - .combine( workflow_id ) - .multiMap { data, id -> - data_dir: (id == "FULL" ? data.data_dir : "") - common_name: (id == "FULL" ? data.common_name : "") - geneset_id: (id == "FULL" ? data.geneset_id : "") + .combine(workflow_id) + .multiMap{ data, id -> + genesets: (id == "FULL" || id == "JBROWSE" ? data.genesets : "") } .set{ alignment_data } @@ -98,14 +96,6 @@ workflow YAML_INPUT { } .set{ selfcomp_data } - group - .synteny - .combine( workflow_id ) - .multiMap { data, id -> - synteny_genome: (id == "FULL" ? data.synteny_genome_path: "") - } - .set{ synteny_data } - group .intron .combine( workflow_id ) @@ -225,14 +215,15 @@ workflow YAML_INPUT { hic_reads_ch = hic_ch supp_reads_ch = supplement_ch - align_data_dir = alignment_data.data_dir - align_geneset = alignment_data.geneset_id - align_common = alignment_data.common_name + // align_data_dir = alignment_data.data_dir + // align_geneset = alignment_data.geneset_id + // align_common = alignment_data.common_name + align_genesets = alignment_data.genesets motif_len = selfcomp_data.motif_len mummer_chunk = selfcomp_data.mummer_chunk - synteny_path = synteny_data.synteny_genome + synteny_paths = group.synteny intron_size = intron_size.size diff --git a/workflows/treeval.nf b/workflows/treeval.nf index 526075da..ab7b85db 100755 --- a/workflows/treeval.nf +++ b/workflows/treeval.nf @@ -135,17 +135,17 @@ workflow TREEVAL { // // SUBWORKFLOW: Takes input fasta to generate BB files containing alignment data // - GENE_ALIGNMENT ( - GENERATE_GENOME.out.dot_genome, - YAML_INPUT.out.reference_ch, - GENERATE_GENOME.out.ref_index, - YAML_INPUT.out.align_data_dir, - YAML_INPUT.out.align_geneset, - YAML_INPUT.out.align_common, - YAML_INPUT.out.intron_size, - gene_alignment_asfiles - ) - ch_versions = ch_versions.mix(GENE_ALIGNMENT.out.versions) + if ( !exclude_workflow_steps.contains("gene_alignment")) { + GENE_ALIGNMENT ( + GENERATE_GENOME.out.dot_genome, + YAML_INPUT.out.reference_ch, + GENERATE_GENOME.out.ref_index, + YAML_INPUT.out.align_genesets, + YAML_INPUT.out.intron_size, + gene_alignment_asfiles + ) + ch_versions = ch_versions.mix(GENE_ALIGNMENT.out.versions) + } // // SUBWORKFLOW: GENERATES A BIGWIG FOR A REPEAT DENSITY TRACK @@ -181,11 +181,15 @@ workflow TREEVAL { // SUBWORKFLOW: Takes reference, the directory of syntenic genomes and order/clade of sequence // and generated a file of syntenic blocks. // - SYNTENY ( - YAML_INPUT.out.reference_ch, - YAML_INPUT.out.synteny_path - ) - ch_versions = ch_versions.mix( SYNTENY.out.versions ) + if ( !exclude_workflow_steps.contains("synteny")) { + YAML_INPUT.out.synteny_paths.view {"SYNTENY_MAIN: $it"} + SYNTENY ( + YAML_INPUT.out.reference_ch, + YAML_INPUT.out.synteny_paths + ) + ch_versions = ch_versions.mix( SYNTENY.out.versions ) + } + // // SUBWORKFLOW: Takes reference, pacbio reads diff --git a/workflows/treeval_rapid.nf b/workflows/treeval_rapid.nf index 8a483b5e..b2135bad 100755 --- a/workflows/treeval_rapid.nf +++ b/workflows/treeval_rapid.nf @@ -55,6 +55,8 @@ workflow TREEVAL_RAPID { params.entry = 'RAPID' input_ch = Channel.fromPath(params.input, checkIfExists: true) + + // // SUBWORKFLOW: reads the yaml and pushing out into a channel per yaml field // @@ -63,6 +65,7 @@ workflow TREEVAL_RAPID { params.entry ) + // // SUBWORKFLOW: Takes input fasta file and sample ID to generate a my.genome file // @@ -72,6 +75,7 @@ workflow TREEVAL_RAPID { ) ch_versions = ch_versions.mix( GENERATE_GENOME.out.versions ) + // // SUBWORKFLOW: GENERATES A BIGWIG FOR A REPEAT DENSITY TRACK // @@ -81,6 +85,7 @@ workflow TREEVAL_RAPID { ) ch_versions = ch_versions.mix( REPEAT_DENSITY.out.versions ) + // // SUBWORKFLOW: GENERATES A GAP.BED FILE TO ID THE LOCATIONS OF GAPS // @@ -89,6 +94,7 @@ workflow TREEVAL_RAPID { ) ch_versions = ch_versions.mix( GAP_FINDER.out.versions ) + // // SUBWORKFLOW: GENERATE TELOMERE WINDOW FILES WITH PACBIO READS AND REFERENCE // @@ -97,33 +103,46 @@ workflow TREEVAL_RAPID { ) ch_versions = ch_versions.mix( TELO_FINDER.out.versions ) + // // SUBWORKFLOW: Takes reference, pacbio reads // - READ_COVERAGE ( - YAML_INPUT.out.reference_ch, - GENERATE_GENOME.out.dot_genome, - YAML_INPUT.out.read_ch - ) - ch_versions = ch_versions.mix( READ_COVERAGE.out.versions ) + if ( !exclude_workflow_steps.contains("read_coverage")) { + READ_COVERAGE ( + YAML_INPUT.out.reference_ch, + GENERATE_GENOME.out.dot_genome, + YAML_INPUT.out.read_ch + ) + coverage_report = READ_COVERAGE.out.ch_reporting + ch_versions = ch_versions.mix( READ_COVERAGE.out.versions ) + } else { + coverage_report = [] + } + // // SUBWORKFLOW: GENERATE HIC MAPPING TO GENERATE PRETEXT FILES AND JUICEBOX // - HIC_MAPPING ( - YAML_INPUT.out.reference_ch, - GENERATE_GENOME.out.ref_index, - GENERATE_GENOME.out.dot_genome, - YAML_INPUT.out.hic_reads_ch, - YAML_INPUT.out.assembly_id, - GAP_FINDER.out.gap_file, - READ_COVERAGE.out.ch_covbw_nor, - READ_COVERAGE.out.ch_covbw_avg, - TELO_FINDER.out.bedgraph_file, - REPEAT_DENSITY.out.repeat_density, - params.entry - ) - ch_versions = ch_versions.mix( HIC_MAPPING.out.versions ) + if ( !exclude_workflow_steps.contains("hic_mapping")) { + HIC_MAPPING ( + YAML_INPUT.out.reference_ch, + GENERATE_GENOME.out.ref_index, + GENERATE_GENOME.out.dot_genome, + YAML_INPUT.out.hic_reads_ch, + YAML_INPUT.out.assembly_id, + GAP_FINDER.out.gap_file, + READ_COVERAGE.out.ch_covbw_nor, + READ_COVERAGE.out.ch_covbw_avg, + TELO_FINDER.out.bedgraph_file, + REPEAT_DENSITY.out.repeat_density, + params.entry + ) + hic_report = HIC_MAPPING.out.ch_reporting + ch_versions = ch_versions.mix( HIC_MAPPING.out.versions ) + } else { + hic_report = [] + } + // // SUBWORKFLOW: Collates version data from prior subworflows @@ -132,6 +151,7 @@ workflow TREEVAL_RAPID { ch_versions.unique().collectFile(name: 'collated_versions.yml') ) + // // LOGIC: GENERATE SOME CHANNELS FOR REPORTING // diff --git a/workflows/treeval_rapid_tol.nf b/workflows/treeval_rapid_tol.nf index 5d651a1a..87d7b126 100755 --- a/workflows/treeval_rapid_tol.nf +++ b/workflows/treeval_rapid_tol.nf @@ -55,6 +55,8 @@ workflow TREEVAL_RAPID_TOL { params.entry = 'RAPID_TOL' input_ch = Channel.fromPath(params.input, checkIfExists: true) + + // // SUBWORKFLOW: reads the yaml and pushing out into a channel per yaml field // @@ -63,6 +65,7 @@ workflow TREEVAL_RAPID_TOL { params.entry ) + // // SUBWORKFLOW: Takes input fasta file and sample ID to generate a my.genome file // @@ -72,6 +75,7 @@ workflow TREEVAL_RAPID_TOL { ) ch_versions = ch_versions.mix( GENERATE_GENOME.out.versions ) + // // SUBWORKFLOW: GENERATES A BIGWIG FOR A REPEAT DENSITY TRACK // @@ -81,6 +85,7 @@ workflow TREEVAL_RAPID_TOL { ) ch_versions = ch_versions.mix( REPEAT_DENSITY.out.versions ) + // // SUBWORKFLOW: GENERATES A GAP.BED FILE TO ID THE LOCATIONS OF GAPS // @@ -89,6 +94,7 @@ workflow TREEVAL_RAPID_TOL { ) ch_versions = ch_versions.mix( GAP_FINDER.out.versions ) + // // SUBWORKFLOW: GENERATE TELOMERE WINDOW FILES WITH PACBIO READS AND REFERENCE // @@ -97,15 +103,22 @@ workflow TREEVAL_RAPID_TOL { ) ch_versions = ch_versions.mix( TELO_FINDER.out.versions ) + // // SUBWORKFLOW: Takes reference, pacbio reads // - READ_COVERAGE ( - YAML_INPUT.out.reference_ch, - GENERATE_GENOME.out.dot_genome, - YAML_INPUT.out.read_ch - ) - ch_versions = ch_versions.mix( READ_COVERAGE.out.versions ) + if ( !exclude_workflow_steps.contains("read_coverage")) { + READ_COVERAGE ( + YAML_INPUT.out.reference_ch, + GENERATE_GENOME.out.dot_genome, + YAML_INPUT.out.read_ch + ) + coverage_report = READ_COVERAGE.out.ch_reporting + ch_versions = ch_versions.mix( READ_COVERAGE.out.versions ) + } else { + coverage_report = [] + } + // // SUBWORKFLOW: Takes reads and assembly, produces kmer plot @@ -116,23 +129,30 @@ workflow TREEVAL_RAPID_TOL { ) ch_versions = ch_versions.mix( KMER.out.versions ) + // // SUBWORKFLOW: GENERATE HIC MAPPING TO GENERATE PRETEXT FILES AND JUICEBOX // - HIC_MAPPING ( - YAML_INPUT.out.reference_ch, - GENERATE_GENOME.out.ref_index, - GENERATE_GENOME.out.dot_genome, - YAML_INPUT.out.hic_reads_ch, - YAML_INPUT.out.assembly_id, - GAP_FINDER.out.gap_file, - READ_COVERAGE.out.ch_covbw_nor, - READ_COVERAGE.out.ch_covbw_avg, - TELO_FINDER.out.bedgraph_file, - REPEAT_DENSITY.out.repeat_density, - params.entry - ) - ch_versions = ch_versions.mix( HIC_MAPPING.out.versions ) + if ( !exclude_workflow_steps.contains("hic_mapping")) { + HIC_MAPPING ( + YAML_INPUT.out.reference_ch, + GENERATE_GENOME.out.ref_index, + GENERATE_GENOME.out.dot_genome, + YAML_INPUT.out.hic_reads_ch, + YAML_INPUT.out.assembly_id, + GAP_FINDER.out.gap_file, + READ_COVERAGE.out.ch_covbw_nor, + READ_COVERAGE.out.ch_covbw_avg, + TELO_FINDER.out.bedgraph_file, + REPEAT_DENSITY.out.repeat_density, + params.entry + ) + hic_report = HIC_MAPPING.out.ch_reporting + ch_versions = ch_versions.mix( HIC_MAPPING.out.versions ) + } else { + hic_report = [] + } + // // SUBWORKFLOW: Collates version data from prior subworflows @@ -141,6 +161,7 @@ workflow TREEVAL_RAPID_TOL { ch_versions.unique().collectFile(name: 'collated_versions.yml') ) + // // LOGIC: GENERATE SOME CHANNELS FOR REPORTING //