Memory error when using metasv

[daxian248]

I'm using metasv to merge the output of pindel, cnvnator and breakdancer. But I get the error "OSError: [Errno 12] Cannot allocate memory". I want to know if the metasv read all the bam file into memory when merge the SVs.
Here is my command:
run_metasv.py --reference /public/home/ylma/genome/Sus_scrofa/Sus_scrofa.Sscrofa10.2.dna.toplevel.fa
--breakdancer_native rc.out
--cnvnator_native rc.cnv
--pindel_native rc_D rc_LI rc_SI rc_TD rc_INV
--sample BMX --bams SAMN02298127.02.bam SAMN02298128.02.bam SAMN02298129.02.bam SAMN02298130.02.bam SAMN02298131.02.bam SAMN02298132.02.bam
--spades /public/home/ylma/tools/SPAdes-3.10.1-Linux/bin/spades.py
--age /public/home/ylma/tools/AGE/age_align --num_threads 15 --workdir work --outdir out
--max_ins_intervals 500000 --isize_mean 500 --isize_sd 150

[msahraeian]

Thanks for using MetaSV @daxian248. Would you please share with us the log?

[daxian248]

Here is the log @msahraeian
metasv.txt

[msahraeian]

The error has occurred when it tries to do pool = multiprocessing.Pool(nthreads).
I suspect you don't have enough threads or memory on your system.
But in general you can use MetaSV to only merge the calls from different tools without assembly. For that setting you can use --disable_assembly flag. On the other hand, if you want to still use the assembly I suggest to use settings like --boost_sc --max_ins_intervals 20000 and you can specify which SV types to assemble using --svs_to_assemble (e.g. --svs_to_assemble INS DEL DUP INV).

Please let me know if it still doesn't work.

[daxian248]

Hi, Marghoob. I have tried many times and changed my command every time, but no one succeeded. Here is one the commands:
run_metasv.py --boost_sc --reference /public/home/ylma/genome/Sus_scrofa/Sus_scrofa.Sscrofa10.2.dna.toplevel.fa
--breakdancer_native rc.out
--cnvnator_native rc.cnv
--pindel_native rc_D rc_LI rc_SI rc_TD rc_INV
--sample BMX --bams SAMN02298127.02.bam SAMN02298128.02.bam SAMN02298129.02.bam SAMN02298130.02.bam SAMN02298131.02.bam SAMN02298132.02.bam
--spades /public/home/ylma/tools/SPAdes-3.10.1-Linux/bin/spades.py
--age /public/home/ylma/tools/AGE/age_align --num_threads 2 --min_support_ins 2 --workdir work --outdir out
--max_ins_intervals 20000 --mean_read_length 100 --isize_mean 304 --isize_sd 48.94 --svs_to_assemble INS --svs_to_softclip INS

I mainly changed the "--num_threads" and "--max_ins_intervals", all the log files is the same and here is one of them
metasv2.txt.

By the way, my metasv version is 0.5.2.

[marghoob]

Hi @daxian248 , could you please try the following if possible and if it makes sense: use MetaSV version 0.5.4, run only for the major contigs using --chromosomes chr1 chr2 ... .

[daxian248]

Thanks, @marghoob . I'll try version 0.5.4.

[daxian248]

Hi, marghoob. I tried "-chromosomes 1" with version 0.5.4, but still failed with the error "OSError: [Errno 12] Cannot allocate memory". Then I tried the smallest chromosome 18, this is the log.
metasv.log

[marghoob]

Thanks, @daxian248. Looks like your machine may not have enough memory. Can you share the memory and CPU specs of the machine you are trying to run on?

[daxian248]

[ylma@comput10 ~]$ grep 'physical id' /proc/cpuinfo | sort -u
physical id : 0
physical id : 1
[ylma@comput10 ~]$ grep 'core id' /proc/cpuinfo | sort -u | wc -l
12
[ylma@comput10 ~]$ free -m
total used free shared buffers cached
Mem: 64310 18395 45915 0 303 1463
-/+ buffers/cache: 16628 47681
Swap: 31999 453 31546

By the way, the average size of my six bam files is about 50G.