VCFTOOLs site_pi and het options #62

gq1 · 2023-11-07T21:45:04Z

PR checklist

This comment contains a description of changes (with reason).
If you've fixed a bug or added code that should be tested, add tests!
If you've added a new tool - have you followed the pipeline conventions in the contribution docs
Make sure your code lints (nf-core lint).
Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
Usage Documentation in docs/usage.md is updated.
Output Documentation in docs/output.md is updated.
CHANGELOG.md is updated.
README.md is updated (including new tool citations and authors/contributors).

…de diversity and heterozygosity

Add include_positions and exclude_positions for the whole workflow.

github-actions · 2023-11-07T21:45:18Z

This PR is against the `main` branch ❌

Do not close this PR
Click Edit and change the base to dev
This CI test will remain failed until you push a new commit

Hi @gq1,

It looks like this pull-request is has been made against the sanger-tol/variantcalling main branch.
The main branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to main are only allowed if they come from the sanger-tol/variantcalling dev branch.

You do not need to close this PR, you can change the target branch to dev by clicking the "Edit" button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.

Thanks again for your contribution!

github-actions · 2023-11-07T21:46:39Z

`nf-core lint` overall result: Passed ✅

Posted for pipeline commit 1882fef

+| ✅ 126 tests passed       |+
#| ❔  26 tests were ignored |#

❔ Tests ignored:

files_exist - File is ignored: CODE_OF_CONDUCT.md
files_exist - File is ignored: assets/nf-core-variantcalling_logo_light.png
files_exist - File is ignored: docs/images/nf-core-variantcalling_logo_light.png
files_exist - File is ignored: docs/images/nf-core-variantcalling_logo_dark.png
files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
files_exist - File is ignored: .github/workflows/awstest.yml
files_exist - File is ignored: .github/workflows/awsfulltest.yml
files_exist - File is ignored: assets/multiqc_config.yml
files_exist - File is ignored: conf/igenomes.config
nextflow_config - Config variable ignored: manifest.name
nextflow_config - Config variable ignored: manifest.homePage
files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
files_unchanged - File ignored due to lint config: assets/email_template.html
files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
files_unchanged - File does not exist: assets/nf-core-variantcalling_logo_light.png
files_unchanged - File does not exist: docs/images/nf-core-variantcalling_logo_light.png
files_unchanged - File does not exist: docs/images/nf-core-variantcalling_logo_dark.png
files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy
actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/variantcalling/variantcalling/.github/workflows/awstest.yml
multiqc_config - 'assets/multiqc_config.yml' not found

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: conf/base.config
files_exist - File found: lib/WorkflowVariantcalling.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-variantcalling_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: '1.1.0-dev'
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
readme - README Nextflow minimum version badge matched config. Badge: 22.10.1, Config: 22.10.1
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (88 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: sanger_test_full.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: sanger_test.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.8
Run at 2023-11-16 09:49:57

gq1 · 2023-11-08T11:21:07Z

Currently nf-core vcftools module has 3 inputs, I am not sure it is a good idea to to push the patch here to them.
It seems they have coped with all the output (big list) but not all the input yet.
https://nf-co.re/modules/vcftools

They did mention this in the module https://github.com/nf-core/modules/blob/18cd2206622dc606bbceea533c7823feb2a251db/modules/nf-core/vcftools/main.nf#L13

muffato

That's how I expected vcftools to be modified. I think you can propose that to nf-core.

The thing I'm wondering: does it make sense to run vcftools on both the VCF and gVCF files ? In the small test, the outputs are identical for the individual BAM and CRAM files but not the combined file.
I'll start the "full test" to check on it too

gq1 · 2023-11-10T15:53:05Z

nf-core/modules#4319

gq1 added 4 commits November 6, 2023 15:18

nf-core modules install vcftools

2c10d8d

nf-core modules update deepvariant

124c585

create a sub workflow to call vcftools to calculate per site nucleoti…

908e9f4

…de diversity and heterozygosity

Patch vcftools modules to allow an input for position files

27a04a8

Add include_positions and exclude_positions for the whole workflow.

gq1 changed the base branch from main to dev November 7, 2023 21:46

remove trailing white spaces

1feb4d8

gq1 requested review from muffato and priyanka-surana November 8, 2023 09:27

Set the positions to an empty list when no file provided.

3de0d8a

This was referenced Nov 8, 2023

Per site nucleotide diversity #60

Closed

Add calculation for heterozygosity #39

Closed

gq1 added 2 commits November 9, 2023 11:23

re-patch vcftools modules

a2128da

update vcftools module patch

c74f90e

muffato reviewed Nov 10, 2023

View reviewed changes

muffato assigned gq1 Nov 10, 2023

muffato added enhancement Improvement of the existing features feature Requests for new features user request Requests made by users and public labels Nov 10, 2023

muffato added this to the 1.1.0 milestone Nov 10, 2023

This was linked to issues Nov 10, 2023

Per site nucleotide diversity #60

Closed

Add calculation for heterozygosity #39

Closed

gq1 added 2 commits November 13, 2023 16:27

not run vcftools on gvcf files

e8a24cf

Update module vcftools to add one more keywords to minimum 3.

1882fef

muffato approved these changes Nov 16, 2023

View reviewed changes

gq1 merged commit 3b3a745 into dev Nov 16, 2023
6 checks passed

gq1 deleted the vcftools_site_pi_het branch November 16, 2023 22:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VCFTOOLs site_pi and het options #62

VCFTOOLs site_pi and het options #62

gq1 commented Nov 7, 2023

github-actions bot commented Nov 7, 2023

github-actions bot commented Nov 7, 2023 •

edited

Loading

❔ Tests ignored:

✅ Tests passed:

Run details

gq1 commented Nov 8, 2023 •

edited

Loading

muffato left a comment

gq1 commented Nov 10, 2023

VCFTOOLs site_pi and het options #62

VCFTOOLs site_pi and het options #62

Conversation

gq1 commented Nov 7, 2023

PR checklist

github-actions bot commented Nov 7, 2023

This PR is against the main branch ❌

github-actions bot commented Nov 7, 2023 • edited Loading

nf-core lint overall result: Passed ✅

❔ Tests ignored:

✅ Tests passed:

Run details

gq1 commented Nov 8, 2023 • edited Loading

muffato left a comment

Choose a reason for hiding this comment

gq1 commented Nov 10, 2023

This PR is against the `main` branch ❌

github-actions bot commented Nov 7, 2023 •

edited

Loading

`nf-core lint` overall result: Passed ✅

gq1 commented Nov 8, 2023 •

edited

Loading