You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Cannot get the test JSON input to run in the pipeline correctly, it fails at read_genome_tsv step and call-align. During pipeline installation I tried using the bash scripts/install_conda_env.sh within an interactive node, but our HPC is limited to interactive nodes of 12 hours duration and it would consistently stall at solving environment for the py2 and spp environments, so those were installed with mamba.
Conda env list lists all four environments,
encd-atac
encd-atac-macs2
encd-atac-py2
encd-atac-spp
When I use java/jdk-21.0.2, I get a build incompatibility with java so I downgrade to either java/jdk-17.0.7 or java/jdk-11.0.11; however, both fail with the same read genome error. Singularity is unfortunately not well supporting on our HPC.
I am running in the directory containing the atac-pipeline files and the atac.wdl file. Test pipeline runs on my own ATAC data and JSON file fails similarly, but where it is unable to find *.samstats.qc and other *.bam *.bai files in the directory, and the run fails.
Thank you for your help! Looking forward to using the pipeline.
Input JSON, from INPUT_JSON="https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ_subsampled.json"
Troubleshooting result
If you ran caper run without Caper server then Caper automatically runs a troubleshooter for failed workflows. Find troubleshooting result in the bottom of Caper's screen log.
If you ran caper submit with a running Caper server then first find your workflow ID (1st column) with caper list and run caper debug [WORKFLOW_ID].
Paste troubleshooting result.
2024-07-14 12:31:53,615|caper.cli|INFO| Cromwell stdout: /n/groups/neuroduo/Shon/venv/atac-seq-pipeline/cromwell.out.5
2024-07-14 12:31:53,649|caper.caper_base|INFO| Creating a timestamped temporary directory. /n/groups/neuroduo/Shon/exp/atac_tmp/atac/20240714_123153_643196
2024-07-14 12:31:53,649|caper.caper_runner|INFO| Localizing files on work_dir. /n/groups/neuroduo/Shon/exp/atac_tmp/atac/20240714_123153_643196
2024-07-14 12:31:53,777|autouri.autouri|INFO| cp: (bf282903) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/caf534ed3cf684406e731d19be272b4a/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.gz
2024-07-14 12:31:55,163|autouri.autouri|INFO| cp: (bf282903) skipped due to name_size_match. size=872949833, mt=1549739698.0
2024-07-14 12:31:55,167|autouri.autouri|INFO| cp: (0159d705) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only.fasta.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/f43b63a83784d3ec8055f1a22168ed89/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only.fasta.gz
2024-07-14 12:31:55,918|autouri.autouri|INFO| cp: (0159d705) skipped due to md5_match. md5=05297d96dd1f7cfb45a7b637d6dd7036
2024-07-14 12:31:55,923|autouri.autouri|INFO| cp: (216306a0) started. src=https://www.encodeproject.org/files/ENCFF356LFX/@@download/ENCFF356LFX.bed.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/f183dcba5d34f959d8b55ed438ee2e22/ENCFF356LFX.bed.gz
2024-07-14 12:31:56,664|autouri.autouri|INFO| cp: (216306a0) skipped due to md5_match. md5=393688b4f06c9ce26165d47433dd8c37
2024-07-14 12:31:57,735|autouri.autouri|INFO| cp: (285d073e) started. src=https://www.encodeproject.org/files/GRCh38_EBV.chrom.sizes/@@download/GRCh38_EBV.chrom.sizes.tsv, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/c52f52c7bfa357f55a39b1de7e4d0b0c/GRCh38_EBV.chrom.sizes.tsv
2024-07-14 12:31:58,440|autouri.autouri|INFO| cp: (285d073e) skipped due to md5_match. md5=c95303fb77cc3e11d50e3c3a4b93b3fb
2024-07-14 12:31:58,445|autouri.autouri|INFO| cp: (129d71d4) started. src=https://www.encodeproject.org/files/ENCFF110MCL/@@download/ENCFF110MCL.tar.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/3ff4ac4c3f59d096b1a3842a182072ae/ENCFF110MCL.tar.gz
2024-07-14 12:31:59,929|autouri.autouri|INFO| cp: (129d71d4) skipped due to name_size_match. size=3749246230, mt=1571469011.0
2024-07-14 12:31:59,944|autouri.autouri|INFO| cp: (c53da864) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/df5193e07055d13c48be59bacd0f56b8/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz
2024-07-14 12:32:00,698|autouri.autouri|INFO| cp: (c53da864) skipped due to md5_match. md5=80b263f6ea6ff65d547eef07102535db
2024-07-14 12:32:00,702|autouri.autouri|INFO| cp: (73fa6b27) started. src=https://www.encodeproject.org/files/ENCFF643CGH/@@download/ENCFF643CGH.tar.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/8c692fba4640609720272154ab0faa30/ENCFF643CGH.tar.gz
2024-07-14 12:32:02,221|autouri.autouri|INFO| cp: (73fa6b27) skipped due to name_size_match. size=4318261891, mt=1549723866.0
2024-07-14 12:32:02,225|autouri.autouri|INFO| cp: (28ab9180) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bwa_index/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bwa_index.tar.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/d3dff25534e93d893902540d81e4f475/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bwa_index.tar.gz
2024-07-14 12:32:02,964|autouri.autouri|INFO| cp: (28ab9180) skipped due to md5_match. md5=7e088c24a017a43b1db5e8f50060eec1
2024-07-14 12:32:02,967|autouri.autouri|INFO| cp: (6b4f724c) started. src=https://www.encodeproject.org/files/ENCFF766FGL/@@download/ENCFF766FGL.bed.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/f16cc7892540d7a9bc3207cc44eb8288/ENCFF766FGL.bed.gz
2024-07-14 12:32:03,709|autouri.autouri|INFO| cp: (6b4f724c) skipped due to md5_match. md5=d108aa2a392ce5dd207fb8769a6e8240
2024-07-14 12:32:03,713|autouri.autouri|INFO| cp: (51e2f481) started. src=https://www.encodeproject.org/files/ENCFF304XEX/@@download/ENCFF304XEX.bed.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/805e179275a9c0fb7a37def40c4312d1/ENCFF304XEX.bed.gz
2024-07-14 12:32:04,932|autouri.autouri|INFO| cp: (51e2f481) skipped due to name_size_match. size=14377496, mt=1592463730.0
2024-07-14 12:32:04,936|autouri.autouri|INFO| cp: (7c2a0b2a) started. src=https://www.encodeproject.org/files/ENCFF140XLU/@@download/ENCFF140XLU.bed.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/0cbd2c602ddad252bc39729fc8a29286/ENCFF140XLU.bed.gz
2024-07-14 12:32:05,690|autouri.autouri|INFO| cp: (7c2a0b2a) skipped due to md5_match. md5=91047588129069ff91ec1b0664179f8e
2024-07-14 12:32:05,693|autouri.autouri|INFO| cp: (2af5e75f) started. src=https://www.encodeproject.org/files/ENCFF212UAV/@@download/ENCFF212UAV.bed.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/1d3aa436b05f16a509edb94789c061d3/ENCFF212UAV.bed.gz
2024-07-14 12:32:06,909|autouri.autouri|INFO| cp: (2af5e75f) skipped due to name_size_match. size=18381891, mt=1592463727.0
2024-07-14 12:32:06,914|autouri.autouri|INFO| cp: (6d60822a) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_dnase_avg_fseq_signal_formatted.txt.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/3b39284516e676ea52238f0636c0bbbf/hg38_dnase_avg_fseq_signal_formatted.txt.gz
2024-07-14 12:32:07,149|autouri.autouri|INFO| cp: (6d60822a) skipped due to md5_match. md5=df624401f76fbd4d651e736068c43a1a
2024-07-14 12:32:07,154|autouri.autouri|INFO| cp: (722f9430) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_celltype_compare_subsample.bed.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/c73f434c3fa4f3f54bc2ecad09c065c2/hg38_celltype_compare_subsample.bed.gz
2024-07-14 12:32:07,243|autouri.autouri|INFO| cp: (722f9430) skipped due to md5_match. md5=ced0c653d28628654288f7a8ab052590
2024-07-14 12:32:07,247|autouri.autouri|INFO| cp: (53d2237e) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_dnase_avg_fseq_signal_metadata.txt, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/a9745b33b4ffdd83d7d2c5a7d3c8036a/hg38_dnase_avg_fseq_signal_metadata.txt
2024-07-14 12:32:07,328|autouri.autouri|INFO| cp: (53d2237e) skipped due to md5_match. md5=3f7fd85ab9a4c6274f28c3e82a79c10d
2024-07-14 12:32:07,344|autouri.autouri|INFO| cp: (5c64f2c0) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair1/ENCFF341MYG.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/7ded2935aace2939d00d8fb5b47867ae/ENCFF341MYG.subsampled.400.fastq.gz
2024-07-14 12:32:07,420|autouri.autouri|INFO| cp: (5c64f2c0) skipped due to md5_match. md5=285bde538821a9bf6ee8cd0abf04eff4
2024-07-14 12:32:07,423|autouri.autouri|INFO| cp: (e1ea4431) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair1/ENCFF106QGY.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/ca04bcf10cb8ac66068bc5f15a436f03/ENCFF106QGY.subsampled.400.fastq.gz
2024-07-14 12:32:07,666|autouri.autouri|INFO| cp: (e1ea4431) skipped due to md5_match. md5=90c790e8400d4f374b9544adcf2a8a28
2024-07-14 12:32:07,671|autouri.autouri|INFO| cp: (1f0909f1) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair2/ENCFF248EJF.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/27d48f46ef8e2ac6910ac87f3ccbe163/ENCFF248EJF.subsampled.400.fastq.gz
2024-07-14 12:32:07,764|autouri.autouri|INFO| cp: (1f0909f1) skipped due to md5_match. md5=9e815d0c9b4a411e7aec70642797bf17
2024-07-14 12:32:07,767|autouri.autouri|INFO| cp: (5421557f) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair2/ENCFF368TYI.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/a7ff3728754970409a3d2f093cf38b52/ENCFF368TYI.subsampled.400.fastq.gz
2024-07-14 12:32:07,984|autouri.autouri|INFO| cp: (5421557f) skipped due to md5_match. md5=9e86e4ce0cfb63cedef6b5c66ad0c820
2024-07-14 12:32:07,988|autouri.autouri|INFO| cp: (c4c48475) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF641SFZ.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/3c6d0cab14c16eaeaf22560de63ac804/ENCFF641SFZ.subsampled.400.fastq.gz
2024-07-14 12:32:08,074|autouri.autouri|INFO| cp: (c4c48475) skipped due to md5_match. md5=2976d268d837c81bc77bacdbd5b28cbf
2024-07-14 12:32:08,080|autouri.autouri|INFO| cp: (00b2e44d) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF751XTV.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/218fa2f732797e84fa153d479e7dcafb/ENCFF751XTV.subsampled.400.fastq.gz
2024-07-14 12:32:08,161|autouri.autouri|INFO| cp: (00b2e44d) skipped due to md5_match. md5=528e60ed63a10f0077a0c99a0c20b2e3
2024-07-14 12:32:08,164|autouri.autouri|INFO| cp: (ae738dd8) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF927LSG.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/dfea21b33ce8e0ab13fd158e7e812586/ENCFF927LSG.subsampled.400.fastq.gz
2024-07-14 12:32:08,354|autouri.autouri|INFO| cp: (ae738dd8) skipped due to md5_match. md5=82f155821d6088ebf5270beb1671d99a
2024-07-14 12:32:08,358|autouri.autouri|INFO| cp: (556f251c) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF859BDM.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/44b253b6db9e0ea8de62aea604878556/ENCFF859BDM.subsampled.400.fastq.gz
2024-07-14 12:32:08,647|autouri.autouri|INFO| cp: (556f251c) skipped due to md5_match. md5=def17d2720996446cba4fa45c479afc1
2024-07-14 12:32:08,651|autouri.autouri|INFO| cp: (386e276a) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF193RRC.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/db6282276790c12cb2e76b1ed84fb26f/ENCFF193RRC.subsampled.400.fastq.gz
2024-07-14 12:32:08,744|autouri.autouri|INFO| cp: (386e276a) skipped due to md5_match. md5=a443647c541ee8ce8353949825c63038
2024-07-14 12:32:08,746|autouri.autouri|INFO| cp: (a99d4a59) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF366DFI.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/88672b6fafd695007baf55f7f9757fed/ENCFF366DFI.subsampled.400.fastq.gz
2024-07-14 12:32:08,827|autouri.autouri|INFO| cp: (a99d4a59) skipped due to md5_match. md5=153c1446024dffaf36c321d0f8f9d904
2024-07-14 12:32:08,831|autouri.autouri|INFO| cp: (dbac0d88) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF031ARQ.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/713c163a0f65d7350956d72323fc0310/ENCFF031ARQ.subsampled.400.fastq.gz
2024-07-14 12:32:08,966|autouri.autouri|INFO| cp: (dbac0d88) skipped due to md5_match. md5=188ee4b5028bdd4f7def49a4a4ec829b
2024-07-14 12:32:08,969|autouri.autouri|INFO| cp: (e2922926) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF590SYZ.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/1654703543de9eb53b95469e4c26856c/ENCFF590SYZ.subsampled.400.fastq.gz
2024-07-14 12:32:09,136|autouri.autouri|INFO| cp: (e2922926) skipped due to md5_match. md5=9b5773cb34c7c639b826cdd767aede5d
2024-07-14 12:32:09,139|autouri.autouri|INFO| cp: (af77bda1) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF734PEQ.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/69b00bab505645c65eb5957a0b6b2392/ENCFF734PEQ.subsampled.400.fastq.gz
2024-07-14 12:32:09,334|autouri.autouri|INFO| cp: (af77bda1) skipped due to md5_match. md5=f2a15aad3828eb535056ec3bda008244
2024-07-14 12:32:09,337|autouri.autouri|INFO| cp: (68ea7863) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF007USV.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/3810785b7e92ca7d6c4612c20c3ca534/ENCFF007USV.subsampled.400.fastq.gz
2024-07-14 12:32:09,435|autouri.autouri|INFO| cp: (68ea7863) skipped due to md5_match. md5=d92c3cea6037efbd23ae6d2b60f4b4f6
2024-07-14 12:32:09,440|autouri.autouri|INFO| cp: (a07b5d77) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF886FSC.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/482c9ce8ed007c251adca385c7d4ce95/ENCFF886FSC.subsampled.400.fastq.gz
2024-07-14 12:32:09,523|autouri.autouri|INFO| cp: (a07b5d77) skipped due to md5_match. md5=63fcca97fd73a2eb624677c11c0ea435
2024-07-14 12:32:09,528|autouri.autouri|INFO| cp: (5a4302de) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF573UXK.subsampled.400.fastq.gz, dest=/n/groups/neuroduo/Shon/exp/atac_tmp/a3a22d635ec162098e31bcea6655fb34/ENCFF573UXK.subsampled.400.fastq.gz
2024-07-14 12:32:09,693|autouri.autouri|INFO| cp: (5a4302de) skipped due to md5_match. md5=95fc749df78710ab6dc48050e20f5c21
2024-07-14 12:32:10,598|caper.caper_workflow_opts|INFO| Conda environment name found in WDL metadata. wdl=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac.wdl, s=encd-atac
2024-07-14 12:32:10,623|caper.cromwell|INFO| Validating WDL/inputs/imports with Womtool...
2024-07-14 12:32:18,402|caper.nb_subproc_thread|INFO| Subprocess finished successfully.
2024-07-14 12:32:18,403|caper.cromwell|INFO| Passed Womtool validation.
2024-07-14 12:32:18,403|caper.caper_runner|INFO| launching run: wdl=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac.wdl, inputs=/n/groups/neuroduo/Shon/exp/atac_tmp/4b5ea65d4461c146c96a3239d93d05ce/ENCSR356KRQ_subsampled.local.json, backend_conf=/n/groups/neuroduo/Shon/exp/atac_tmp/atac/20240714_123153_643196/backend.conf
2024-07-14 12:32:31,639|caper.cromwell_workflow_monitor|INFO| Workflow: id=d631eff3-d373-4b1c-874e-ae83cef31d3f, status=Submitted
2024-07-14 12:32:31,726|caper.cromwell_workflow_monitor|INFO| Workflow: id=d631eff3-d373-4b1c-874e-ae83cef31d3f, status=Running
2024-07-14 12:32:46,216|caper.cromwell_workflow_monitor|INFO| Task: id=d631eff3-d373-4b1c-874e-ae83cef31d3f, task=atac.read_genome_tsv:-1, retry=0, status=CallCached
2024-07-14 12:34:32,242|caper.cromwell_workflow_monitor|INFO| Workflow: id=d631eff3-d373-4b1c-874e-ae83cef31d3f, status=Failed
2024-07-14 12:35:17,331|caper.cromwell_metadata|INFO| Wrote metadata file. /n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/metadata.json
2024-07-14 12:35:17,332|caper.cromwell|INFO| Workflow failed. Auto-troubleshooting...
2024-07-14 12:35:17,337|caper.nb_subproc_thread|ERROR| Cromwell failed. returncode=1
2024-07-14 12:35:17,337|caper.cli|ERROR| Check stdout in /n/groups/neuroduo/Shon/venv/atac-seq-pipeline/cromwell.out.5
* Started troubleshooting workflow: id=d631eff3-d373-4b1c-874e-ae83cef31d3f, status=Failed
* Found failures JSON object.
[
{
"causedBy": [
{
"message": "Unable to start job. Check the stderr file for possible errors: /n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align/shard-1/execution/stderr.submit",
"causedBy": []
},
{
"message": "Unable to start job. Check the stderr file for possible errors: /n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align/shard-0/execution/stderr.submit",
"causedBy": []
},
{
"message": "Unable to start job. Check the stderr file for possible errors: /n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align_mito/shard-1/execution/stderr.submit",
"causedBy": []
},
{
"message": "Unable to start job. Check the stderr file for possible errors: /n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align_mito/shard-0/execution/stderr.submit",
"causedBy": []
}
],
"message": "Workflow failed"
}
]
* Recursively finding failures in calls (tasks)...
==== NAME=atac.align_mito, STATUS=Failed, PARENT=
SHARD_IDX=0, RC=None, JOB_ID=None
START=2024-07-14T16:33:01.303Z, END=2024-07-14T16:34:31.739Z
STDOUT=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align_mito/shard-0/execution/stdout
STDERR=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align_mito/shard-0/execution/stderr
==== NAME=atac.align_mito, STATUS=Failed, PARENT=
SHARD_IDX=1, RC=None, JOB_ID=None
START=2024-07-14T16:32:59.308Z, END=2024-07-14T16:34:29.730Z
STDOUT=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align_mito/shard-1/execution/stdout
STDERR=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align_mito/shard-1/execution/stderr
==== NAME=atac.align, STATUS=Failed, PARENT=
SHARD_IDX=0, RC=None, JOB_ID=None
START=2024-07-14T16:32:57.316Z, END=2024-07-14T16:34:27.702Z
STDOUT=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align/shard-0/execution/stdout
STDERR=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align/shard-0/execution/stderr
==== NAME=atac.align, STATUS=Failed, PARENT=
SHARD_IDX=1, RC=None, JOB_ID=None
START=2024-07-14T16:32:55.460Z, END=2024-07-14T16:34:26.485Z
STDOUT=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align/shard-1/execution/stdout
STDERR=/n/groups/neuroduo/Shon/venv/atac-seq-pipeline/atac/d631eff3-d373-4b1c-874e-ae83cef31d3f/call-align/shard-1/execution/stderr
(encd-atac)
The text was updated successfully, but these errors were encountered:
Describe the bug
Cannot get the test JSON input to run in the pipeline correctly, it fails at read_genome_tsv step and call-align. During pipeline installation I tried using the bash scripts/install_conda_env.sh within an interactive node, but our HPC is limited to interactive nodes of 12 hours duration and it would consistently stall at solving environment for the py2 and spp environments, so those were installed with mamba.
Conda env list lists all four environments,
encd-atac
encd-atac-macs2
encd-atac-py2
encd-atac-spp
When I use java/jdk-21.0.2, I get a build incompatibility with java so I downgrade to either java/jdk-17.0.7 or java/jdk-11.0.11; however, both fail with the same read genome error. Singularity is unfortunately not well supporting on our HPC.
I am running in the directory containing the atac-pipeline files and the atac.wdl file. Test pipeline runs on my own ATAC data and JSON file fails similarly, but where it is unable to find *.samstats.qc and other *.bam *.bai files in the directory, and the run fails.
Thank you for your help! Looking forward to using the pipeline.
OS/Platform
Caper configuration file
Paste contents of
~/.caper/default.conf.Input JSON file
Paste contents of your input JSON file.
Input JSON, from INPUT_JSON="https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ_subsampled.json"Troubleshooting result
If you ran
caper runwithout Caper server then Caper automatically runs a troubleshooter for failed workflows. Find troubleshooting result in the bottom of Caper's screen log.If you ran
caper submitwith a running Caper server then first find your workflow ID (1st column) withcaper listand runcaper debug [WORKFLOW_ID].Paste troubleshooting result.
The text was updated successfully, but these errors were encountered: