CONSENT-polish fails at sort step

[nadegeguiglielmoni]

CONSENT v1.2.2

Hi,

Thank you for this tool.

I'm struggling to use CONSENT on a cluster, and the latest error I get is at the sort step :

Error log

[M::main] Version: 2.17-r941
[M::main] CMD: /home/nguiglie/Tools/minimap2-2.17_x64-linux/minimap2 --dual=yes -PD --no-long-join -w5 -g1000 -m30 -n1 -t30 -I1G raven_default.min100kb.assembly_1.fasta reads.pacbio.01.fasta
[M::main] Real time: 7419.892 sec; CPU: 179145.556 sec; Peak RSS: 18.149 GB
sort: erreur d'écriture: Erreur d'entrée/sortie

Output log

[sam. nov. 23 23:20:46 CET 2019] Aligning the long reads to the contigs (minimap2)
[dim. nov. 24 01:24:34 CET 2019] Sorting the overlaps

Sort version: sort (GNU coreutils) 8.22

Infos about output files:

124G 24 nov. 01:47 exploded_66171_2
121G 24 nov. 02:08 exploded_66171_3
115G 24 nov. 04:00 sorted_exploded_66171_1
19G 24 nov. 06:01 sorted_exploded_66171_2
359G 24 nov. 01:24 tmp_Alignments_66171.paf

Have you got any idea what could be the problem?
Also, it would be convenient to be able to restart the program.

[morispi]

Hi Nadège,

Sorry for answering so late, I'm quite busy with teaching lately.

I believe the problem might be that you used more memory than your cluster allows you to? I mean, I'm not sure how your cluster is set up, but judging from the error (sort not completing its job properly) I don't really know what else it could be. Googling the issue with sort didn't seem to help much...

Could you maybe try to run CONSENT-polish on a smaller dataset? That could help narrowing down the problem, and see if it's actually caused by an excessive usage of disk space.

Also, great suggestion, restarting the program from the last step would be super convenient, I'll be sure to include it in a further release.

Cheers,
Pierre

[sadikmu]

Hi,

CONSENT-polish failed at sort step. Any suggestions please?

CONSENT-polish --contigs contigs.fasta --reads epo.pacbio.fasta.gz --out polished.fasta

Last lines of err.out

[M::main] Version: 2.17-r941
[M::main] CMD: minimap2 --dual=yes -PD --no-long-join -w5 -g1000 -m30 -n1 -t88 -I1G contigs.fasta pacbio.fasta.gz
[M::main] Real time: 25939.646 sec; CPU: 404591.600 sec; Peak RSS: 37.921 GB
sort: close failed: ./sort2wwjKa: Input/output error

Thanks