-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refine Contamination Analysis Workflow for Extra Input Resources #443
base: main
Are you sure you want to change the base?
Commits on Nov 8, 2023
-
UPDATES TO THE Hifiasm pipeline:
* update Hifiasm to version 0.19.5 * update how Hifiasm outputs are compressed (bgz replacing gz), also * monitor hifiasm resources usage
Configuration menu - View commit details
-
Copy full SHA for e09dc65 - Browse repository at this point
Copy the full SHA e09dc65View commit details -
* update docker used in PBSV tasks to the version coming with official SMRTLink releases (2.9.0) * change how the 2-step PBSV process is done (following the recommended way now)
Configuration menu - View commit details
-
Copy full SHA for d3afc4b - Browse repository at this point
Copy the full SHA d3afc4bView commit details -
For both CCS/ONT, update Sniffles-2
* to version 2.0.7 * using TRF bed * conditionally phase sv (requires phased bam) * generates its own vcf.gz and tbi
Configuration menu - View commit details
-
Copy full SHA for 3a2ac5c - Browse repository at this point
Copy the full SHA 3a2ac5cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 29aa964 - Browse repository at this point
Copy the full SHA 29aa964View commit details -
MAJOR REFACTOR: UNIFY CCS/ONT WGS PIPELINE
Overhaul how small variants are called in the WG pipelines * default to use DV to call small variants, Clair3 analysis needs to be requested explicitly * retire the Pepper toolchain completely from the CCS pipeline, using DV directly * for R10.4+ ONT data, also use DV directly * older ONT data would still use the PEPPER-DV-Margin pipeline * offers GPU version (though based on, it's not worth it yet) * update how bam haplotagging is done Cleanup structural variants calling * experiment with SNF2 phasing SV calls (implicitly depends on small variants calling now) * tune PBSV calling - discover now supports --hifi - output vcf.gz and tbi - less verbose logging by default Misc.: * optimizations to BAM merging and metrics workflow * updates coverage collection step * new R script to visualize log from vm_monitoring_script.sh
Configuration menu - View commit details
-
Copy full SHA for 0262525 - Browse repository at this point
Copy the full SHA 0262525View commit details
Commits on Dec 1, 2023
-
MISC UPDATES TO SEVERAL UTILS TASKS
* organize dockstore.yml file a bit * make WDL validation shell script more usable * update pbmm2 and pbindex to versions in SMRTLink * update GeneralUtils.wdl - two bash-like new tasks [CoerceMapToArrayOfPairs, CoerceArrayOfPairsToMap] - cleanup task CollapseArrayOfStrings * update resource allocations to tasks - NanoplotFromBam (also changes docker) - MosDepthWGS
Configuration menu - View commit details
-
Copy full SHA for 7be9309 - Browse repository at this point
Copy the full SHA 7be9309View commit details -
New docker that's intended to replace lr-basic:
* incorporates gcloud cli (not just gsutil) * integrate libdeflate for more speedups
Configuration menu - View commit details
-
Copy full SHA for 1ff0912 - Browse repository at this point
Copy the full SHA 1ff0912View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4224ab4 - Browse repository at this point
Copy the full SHA 4224ab4View commit details -
Configuration menu - View commit details
-
Copy full SHA for a13a541 - Browse repository at this point
Copy the full SHA a13a541View commit details -
significantly boost capabilities of BAMutils.wdl
incorporate new tasks and optimize them * [CountMethylCallReads, GatherReadsWithoutMethylCalls] from sh_beans * [GetPileup, BamToRelevantPileup] from sh_more_atomic_qc * [GetReadGroupLines, GetSortOrder, SplitNameSortedUbam] from sh_ont_fc * [SamtoolsFlagStats, ParseFlagStatsJson] from sh_trvial_stats * [FilterBamByLen, InferSampleName] from sh_seqkit * [CountAlignmentRecords, StreamingBamErrored, CountAlignmentRecordsByFlag] from sh_maha_aln_metrics * [ResetSamplename] from sh_ingest_singlerg * [MergeBamsWithSamtools] from sh_ont_fc.Utils.wdl * [BamToFastq] from sh_more_bam_qcs and optimize it with sh_ingest_singlerg.Utils.wdl delete * GetSortOrder as that's now implemented in GatherBamMetadata * Drop2304Alignments as that's no longer used update dockers to the latest
Configuration menu - View commit details
-
Copy full SHA for 2905c69 - Browse repository at this point
Copy the full SHA 2905c69View commit details -
Configuration menu - View commit details
-
Copy full SHA for b16c619 - Browse repository at this point
Copy the full SHA b16c619View commit details
Commits on Dec 19, 2023
-
CHERRY-PICK FROM VARIOUS QC/METRICS BRANCHES: * collect information about ML/MM tags in a long-read BAM (sh_beans) * a heuristic way to find peaks in a distribution (using dyst) (sh_dyst_peaker) * filter reads by length in a BAM * collect some read quality stats from (length-filtered) FASTQ/BAM (sh_seq_kit) * VerifyBamID2 (for contamination estimation) * naive sex-concordance check (sh_more_atomic_qc) * check fingerprint of a single BAM file (sh_sample_fp) * collect SAM flag stats (sh_trivial_stats)
Configuration menu - View commit details
-
Copy full SHA for c96ea8e - Browse repository at this point
Copy the full SHA c96ea8eView commit details
Commits on Dec 27, 2023
-
Improve various existing codes
* make BeanCounter finalization optional (wdl/pipelines/TechAgnostic/Utility/CountTheBeans.wdl) * custom struct for sub-workflow config using a JSON (wdl/pipelines/TechAgnostic/Utility/LongReadsContaminationEstimation.wdl) * make fingerprint checking subworkflow control size filtering (wdl/tasks/QC/FPCheckAoU.wdl) (wdl/pipelines/TechAgnostic/Utility/VerifyBamFingerprint.wdl) * fix a warning by IDE/miniwdl complaining WDL stdlib function length only applies to Array (wdl/tasks/Utility/BAMutils.wdl) * various updates to Finalize (wdl/tasks/Utility/Finalize.wdl) New tasks in (wdl/tasks/Utility/GeneralUtils.wdl) to * correctly convert Map to TSV * concatenate files
Configuration menu - View commit details
-
Copy full SHA for 39d77d2 - Browse repository at this point
Copy the full SHA 39d77d2View commit details -
Configuration menu - View commit details
-
Copy full SHA for cbfed4e - Browse repository at this point
Copy the full SHA cbfed4eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8039c88 - Browse repository at this point
Copy the full SHA 8039c88View commit details -
Configuration menu - View commit details
-
Copy full SHA for b16155f - Browse repository at this point
Copy the full SHA b16155fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5bf0b2f - Browse repository at this point
Copy the full SHA 5bf0b2fView commit details
Commits on Dec 28, 2023
-
* AlignAndCheckFingerprintCCS.wdl * CollectPacBioAlignedMetrics.wdl * CollectSMRTCellUnalignedMetrics.wdl
Configuration menu - View commit details
-
Copy full SHA for 0d61935 - Browse repository at this point
Copy the full SHA 0d61935View commit details
Commits on Jan 2, 2024
-
Fix bug in deduplicating aligned ONT BAM
(CHRRY-PICK & follow up to PR 406)
Configuration menu - View commit details
-
Copy full SHA for 989ead1 - Browse repository at this point
Copy the full SHA 989ead1View commit details
Commits on Jan 3, 2024
-
* SampleLevelAlignedMetrics.wdl * PBCLRWholeGenome.wdl
Configuration menu - View commit details
-
Copy full SHA for b8e67ff - Browse repository at this point
Copy the full SHA b8e67ffView commit details -
* new struct in AlignedBamQCandMetrics.wdl to facilicate as-sub-workflow calling * change parameters name for fingerprint workflows
Configuration menu - View commit details
-
Copy full SHA for e5a79c2 - Browse repository at this point
Copy the full SHA e5a79c2View commit details
Commits on Jan 12, 2024
-
a few tweaks to to AlignedBamQCandMetrics:
* make saving of reads without methylation SAM tags optional * better parameter naming
Configuration menu - View commit details
-
Copy full SHA for b5dc978 - Browse repository at this point
Copy the full SHA b5dc978View commit details
Commits on Jan 18, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 1abcc7a - Browse repository at this point
Copy the full SHA 1abcc7aView commit details
Commits on Jan 22, 2024
-
Configuration menu - View commit details
-
Copy full SHA for da5bfb0 - Browse repository at this point
Copy the full SHA da5bfb0View commit details
Commits on Jan 23, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7975d6d - Browse repository at this point
Copy the full SHA 7975d6dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5620d74 - Browse repository at this point
Copy the full SHA 5620d74View commit details
Commits on Jan 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e910bac - Browse repository at this point
Copy the full SHA e910bacView commit details
Commits on Feb 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 04b82f8 - Browse repository at this point
Copy the full SHA 04b82f8View commit details
Commits on Feb 26, 2024
-
Safer and more efficient way to do targetted pileup conversion
(affects contamination estimation)
Configuration menu - View commit details
-
Copy full SHA for a03ff34 - Browse repository at this point
Copy the full SHA a03ff34View commit details
Commits on Feb 29, 2024
-
Refactor LongReadsContaminationEstimation and Contamination workflows…
… for efficiency - Remove unnecessary BED file input from LongReadsContaminationEstimation workflow as BED paths are now hardcoded in the Docker image. - Modify the inputs and commands in Contamination.wdl to align with new Docker setup and work with the .mu, .UD, and .bed files from the docker. - Adjust workflow parameters to better reflect current data processing requirements and practices.
Configuration menu - View commit details
-
Copy full SHA for 28a9e68 - Browse repository at this point
Copy the full SHA 28a9e68View commit details -
Removing SVDPrefix from the command line.
Configuration menu - View commit details
-
Copy full SHA for 7db20f6 - Browse repository at this point
Copy the full SHA 7db20f6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 784081a - Browse repository at this point
Copy the full SHA 784081aView commit details