Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--master local[n] results in an error #57

Open
pdagosto opened this issue Feb 7, 2018 · 1 comment
Open

--master local[n] results in an error #57

pdagosto opened this issue Feb 7, 2018 · 1 comment

Comments

@pdagosto
Copy link

pdagosto commented Feb 7, 2018

If I specify --master local then things work as expected.

If, however, I specify --master local[*] or --master local[n] where n represents the number of cores to use then I get a failure.

Here's the command line:

spark-submit --class com.github.sparkbwa.SparkBWA --master local[32] ~/SparkBWA/target/SparkBWA-0.2.jar -n 32 -r -t sparkBwaTemp -i ~/gs/db/CriGri_1.0_gbff.fasta GStest_R1.fq GStest_R2.fq OutputSparkBWA.sam

I'm not sure exactly what goes wrong but I start seeing messages in the log such as Error saving stdout .

[Java_com_github_sparkbwa_BwaJni_bwa_1jni] option[0]: bwa.
[Java_com_github_sparkbwa_BwaJni_bwa_1jni] option[1]: mem.
[Java_com_github_sparkbwa_BwaJni_bwa_1jni] option[2]: /home/my_user_id/gs/db/CriGri_1.0_gbff.fasta.
[Java_com_github_sparkbwa_BwaJni_bwa_1jni] option[3]: sparkBwaTemp/local-1518020933380-RDD7_1.
[Java_com_github_sparkbwa_BwaJni_bwa_1jni] option[4]: sparkBwaTemp/local-1518020933380-RDD7_2.
[Java_com_github_sparkbwa_BwaJni_bwa_1jni] option[2]: /home/my_user_id/gs/db/CriGri_1.0_gbff.fasta.
[Java_com_github_sparkbwa_BwaJni_bwa_1jni] option[3]: sparkBwaTemp/local-1518020933380-RDD23_1.
[Java_com_github_sparkbwa_BwaJni_bwa_1jni] option[4]: sparkBwaTemp/local-1518020933380-RDD23_2.
[Java_com_github_sparkbwa_BwaJni_bwa_1jni] Error saving stdout.

There are other error messages at the end of the run:

[fclose] No such file or directory
[fclose] No such file or directory
======= Backtrace: =========
[fclose] No such file or directory
/lib64/libc.so.6(+0x7c619)[0x7f1f18783619]
/lib64/libc.so.6(fclose+0x155)[0x7f1f18771c15]
/tmp/libbwa2102141332556821063.so(err_fclose+0x9)[0x7f1cee9dc2b9]
/tmp/libbwa2102141332556821063.so(main+0x1a4)[0x7f1cee9953e4]
/tmp/libbwa2102141332556821063.so(Java_com_github_sparkbwa_BwaJni_bwa_1jni+0x2cc)[0x7f1cee99845c]
[0x7f1f01017774]
======= Memory map: ========
(nothing after this)

Has anyone else seen this?
Does anyone have an explanation?

@Maryom
Copy link

Maryom commented Mar 11, 2019

Has anyone else seen this?

Yes, I have.

Did you solve it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants