Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DIAMOND failing #22

Closed
beardymcjohnface opened this issue Jul 8, 2024 · 3 comments
Closed

DIAMOND failing #22

beardymcjohnface opened this issue Jul 8, 2024 · 3 comments
Labels
bug Something isn't working joss_review issue related to the joss review

Comments

@beardymcjohnface
Copy link

beardymcjohnface commented Jul 8, 2024 •
edited
Loading

For review openjournals/joss-reviews/issues/6782
Hi,
This is probably related to #13. I'm unable to run binette on the test dataset. I installed using the bioconda install instructions and ran the test command.

macOS Sonoma v14.5
conda 23.11.0
Python 3.8.18

$ binette -b binning_results/*.binning --contigs all_contigs.fna --checkm2_db checkm2_tiny_db/checkm2_tiny_db.dmnd -v -o test_results
[2024-07-08 17:48:56] INFO - Program started
[2024-07-08 17:48:56] INFO - command line: /Users/a1234202/miniconda3/envs/binette/bin/binette -b binning_results/A.binning binning_results/B.binning binning_results/C.binning --contigs all_contigs.fna --checkm2_db checkm2_tiny_db/checkm2_tiny_db.dmnd -v -o test_results
[2024-07-08 17:48:56] INFO - Parsing bin2contig files.
[2024-07-08 17:48:56] INFO - 3 bin sets processed:
[2024-07-08 17:48:56] INFO -  A - 6 bins
[2024-07-08 17:48:56] INFO -  B - 3 bins
[2024-07-08 17:48:56] INFO -  C - 4 bins
[2024-07-08 17:48:56] INFO - Parsing contig fasta file: all_contigs.fna
[2024-07-08 17:48:56] INFO - Predicting cds sequences with Pyrodigal using 1 threads.
[2024-07-08 17:48:58] INFO - Writing predicted protein sequences.
[2024-07-08 17:48:58] INFO - Running diamond
[2024-07-08 17:48:58] INFO - diamond blastp --outfmt 6 --max-target-seqs 1 --query test_results/temporary_files/assembly_proteins.faa -o test_results/temporary_files/diamond_result.tsv --threads 1 --db checkm2_tiny_db/checkm2_tiny_db.dmnd --query-cover 80 --subject-cover 80 --id 30 --evalue 1e-05 --block-size 2 2> test_results/temporary_files/diamond_result.log
/bin/sh: line 1: 20164 Illegal instruction: 4  diamond blastp --outfmt 6 --max-target-seqs 1 --query test_results/temporary_files/assembly_proteins.faa -o test_results/temporary_files/diamond_result.tsv --threads 1 --db checkm2_tiny_db/checkm2_tiny_db.dmnd --query-cover 80 --subject-cover 80 --id 30 --evalue 1e-05 --block-size 2 2> test_results/temporary_files/diamond_result.log
[2024-07-08 17:48:59] ERROR - An error occurred while running DIAMOND. Check log file: test_results/temporary_files/diamond_result.log
$ cat test_results/temporary_files/diamond_result.log
diamond v2.0.4.142 (C) Max Planck Society for the Advancement of Science
Documentation, support and updates available at http://www.diamondsearch.org

#CPU threads: 1
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory: test_results/temporary_files
Opening the database...  [0s]
#Target sequences to report alignments for: 1
Reference = checkm2_tiny_db/checkm2_tiny_db.dmnd
Sequences = 1775
Letters = 592231
Block size = 2000000000
Opening the input file...  [0s]
Opening the output file...  [0s]
Loading query sequences...  [0.002s]
Masking queries...  [0.041s]
Building query seed set...  [0.003s]
Algorithm: Double-indexed
Building query histograms...  [0.008s]
Allocating buffers...  [0s]
Loading reference sequences...  [0s]
Masking reference...  [0.029s]
Initializing temporary storage...  [0.002s]
Building reference histograms...  [0.005s]
Allocating buffers...  [0s]
Processing query block 1, reference block 1/1, shape 1/2, index chunk 1/4.
Building reference seed array...  [0.006s]
Building query seed array...  [0.008s]
Computing hash join...  [0.004s]
Building seed filter...  [0.001s]
Searching alignments...  [0.106s]
Processing query block 1, reference block 1/1, shape 1/2, index chunk 2/4.
Building reference seed array...  [0.008s]
Building query seed array...  [0.012s]
Computing hash join...  [0.004s]
Building seed filter...  [0.001s]
Searching alignments...  [0.099s]
Processing query block 1, reference block 1/1, shape 1/2, index chunk 3/4.
Building reference seed array...  [0.01s]
Building query seed array...  [0.014s]
Computing hash join...  [0.004s]
Building seed filter...  [0.001s]
Searching alignments...  [0.096s]
Processing query block 1, reference block 1/1, shape 1/2, index chunk 4/4.
Building reference seed array...  [0.006s]
Building query seed array...  [0.008s]
Computing hash join...  [0.004s]
Building seed filter...  [0.001s]
Searching alignments...  [0.095s]
Processing query block 1, reference block 1/1, shape 2/2, index chunk 1/4.
Building reference seed array...  [0.006s]
Building query seed array...  [0.008s]
Computing hash join...  [0.004s]
Building seed filter...  [0.001s]
Searching alignments...  [0.1s]
Processing query block 1, reference block 1/1, shape 2/2, index chunk 2/4.
Building reference seed array...  [0.008s]
Building query seed array...  [0.012s]
Computing hash join...  [0.004s]
Building seed filter...  [0.001s]
Searching alignments...  [0.096s]
Processing query block 1, reference block 1/1, shape 2/2, index chunk 3/4.
Building reference seed array...  [0.01s]
Building query seed array...  [0.014s]
Computing hash join...  [0.004s]
Building seed filter...  [0.001s]
Searching alignments...  [0.095s]
Processing query block 1, reference block 1/1, shape 2/2, index chunk 4/4.
Building reference seed array...  [0.006s]
Building query seed array...  [0.008s]
Computing hash join...  [0.004s]
Building seed filter...  [0.001s]
Searching alignments...  [0.093s]
Deallocating buffers...  [0s]
Clearing query masking...  [0s]
Computing alignments... 
$ conda list
# packages in environment at /Users/a1234202/miniconda3/envs/binette:
#
# Name                    Version                   Build  Channel
abseil-cpp                20200923.3           h23ab428_0  
absl-py                   2.1.0            py38hecd8cb5_0  
aiohttp                   3.9.5            py38h6c40b1e_0  
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
archspec                  0.2.3              pyhd3eb1b0_0  
astor                     0.8.1            py38hecd8cb5_0  
astunparse                1.6.3                      py_0  
async-timeout             4.0.3            py38hecd8cb5_0  
attrs                     23.2.0             pyh71513ae_0    conda-forge
binette                   1.0.1              pyh7e72e81_0    bioconda
blas                      2.122                  openblas    conda-forge
blas-devel                3.9.0           22_osx64_openblas    conda-forge
blinker                   1.8.2              pyhd8ed1ab_0    conda-forge
boost-cpp                 1.70.0               hef959ae_3    conda-forge
brotli-python             1.1.0            py38h940360d_1    conda-forge
bzip2                     1.0.8                h6c40b1e_6  
c-ares                    1.28.1               h10d778d_0    conda-forge
ca-certificates           2024.7.4             h8857fd0_0    conda-forge
cachetools                4.2.4              pyhd8ed1ab_0    conda-forge
certifi                   2024.6.2         py38hecd8cb5_0  
cffi                      1.16.0           py38h6c40b1e_1  
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
checkm2                   1.0.2              pyh7cba7a3_0    bioconda
click                     8.1.7            py38hecd8cb5_0  
coverage                  7.5.4            py38hc718529_0    conda-forge
cryptography              41.0.3           py38ha2381d6_0  
cython                    3.0.10           py38h6c40b1e_0  
diamond                   2.0.4                h31d8819_0    bioconda
frozenlist                1.4.1            py38hae2e43d_0    conda-forge
gast                      0.3.3                      py_0  
giflib                    5.2.2                h10d778d_0    conda-forge
google-auth               1.35.0             pyh6c4a22f_0    conda-forge
google-auth-oauthlib      0.4.6              pyhd8ed1ab_0    conda-forge
google-pasta              0.2.0              pyhd3eb1b0_0  
grpc-cpp                  1.36.4               h33525da_1    conda-forge
grpcio                    1.36.1           py38h97de6d8_1  
h5py                      2.10.0           py38h0601b69_1  
hdf5                      1.10.6               h10fe05b_1  
icu                       67.1                 hb1e8313_0    conda-forge
idna                      3.7              py38hecd8cb5_0  
importlib-metadata        8.0.0              pyha770c72_0    conda-forge
joblib                    1.4.2            py38hecd8cb5_0  
jpeg                      9e                   h6c40b1e_1  
keras-preprocessing       1.1.2              pyhd3eb1b0_0  
krb5                      1.20.1               hdba6334_1  
libblas                   3.9.0           22_osx64_openblas    conda-forge
libcblas                  3.9.0           22_osx64_openblas    conda-forge
libcurl                   8.2.1                ha585b31_0  
libcxx                    17.0.6               heb59cac_3    conda-forge
libedit                   3.1.20230828         h6c40b1e_0  
libev                     4.33                 h9ed2024_1  
libffi                    3.4.4                hecd8cb5_1  
libgfortran               5.0.0           13_2_0_h97931a8_3    conda-forge
libgfortran5              13.2.0               h2873a65_3    conda-forge
liblapack                 3.9.0           22_osx64_openblas    conda-forge
liblapacke                3.9.0           22_osx64_openblas    conda-forge
libnghttp2                1.52.0               h1c88b7d_1  
libopenblas               0.3.27          openmp_hfef2a42_0    conda-forge
libpng                    1.6.43               h92b6c6a_0    conda-forge
libprotobuf               3.15.8               hcf210ce_1    conda-forge
libsqlite                 3.46.0               h1b8f9f3_0    conda-forge
libssh2                   1.10.0               hdb2fb19_2  
libzlib                   1.2.13               h87427d6_6    conda-forge
lightgbm                  3.2.1            py38h23ab428_0  
llvm-openmp               18.1.8               h15ab845_0    conda-forge
lz4-c                     1.9.4                hcec6c5f_1  
markdown                  3.6                pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5            py38hae2e43d_0    conda-forge
multidict                 6.0.5            py38hef030d1_0    conda-forge
ncurses                   6.5                  h5846eda_0    conda-forge
networkx                  3.1              py38hecd8cb5_0  
numpy                     1.19.2           py38hb9de1e1_1  
numpy-base                1.19.2           py38hf048a4f_1  
oauthlib                  3.2.2            py38hecd8cb5_0  
openblas                  0.3.27          openmp_h6794695_0    conda-forge
openssl                   1.1.1w               hca72f7f_0  
opt_einsum                3.3.0              pyhd3eb1b0_1  
packaging                 24.1             py38hecd8cb5_0  
pandas                    1.4.0            py38ha53d530_0    conda-forge
pip                       24.0             py38hecd8cb5_0  
prodigal                  2.6.3                h8af04d4_9    bioconda
protobuf                  3.15.8           py38ha048514_0    conda-forge
pyasn1                    0.6.0              pyhd8ed1ab_0    conda-forge
pyasn1-modules            0.4.0              pyhd8ed1ab_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pyfastx                   2.1.0            py38h019ace8_2    bioconda
pyjwt                     2.8.0            py38hecd8cb5_0  
pyopenssl                 23.2.0           py38hecd8cb5_0  
pyrodigal                 3.4.1            py38h51c4a30_1    bioconda
pysocks                   1.7.1                    py38_1  
python                    3.8.18               h218abb5_0  
python-dateutil           2.9.0post0       py38hecd8cb5_2  
python-flatbuffers        1.12               pyhd3eb1b0_0  
python_abi                3.8                      2_cp38    conda-forge
pytz                      2024.1           py38hecd8cb5_0  
pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
re2                       2021.04.01           he49afe7_0    conda-forge
readline                  8.2                  hca72f7f_0  
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
requests-oauthlib         2.0.0              pyhd8ed1ab_0    conda-forge
rsa                       4.9                pyhd8ed1ab_0    conda-forge
scikit-learn              0.23.2           py38h959d312_0  
scipy                     1.9.3            py38h9034365_2  
setuptools                70.1.1             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyhd3eb1b0_1  
snappy                    1.1.10               hcec6c5f_1  
sqlite                    3.46.0               h28673e1_0    conda-forge
tensorboard               2.4.1              pyhd8ed1ab_1    conda-forge
tensorboard-plugin-wit    1.8.1              pyhd8ed1ab_0    conda-forge
tensorflow                2.4.0            py38h50d1736_0    conda-forge
tensorflow-base           2.4.0            py38h428766a_0    conda-forge
tensorflow-estimator      2.4.0              pyh9656e83_0    conda-forge
termcolor                 2.4.0              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.5.0            py38h20db666_0  
tk                        8.6.14               h4d00af3_0  
tomli                     2.0.1            py38hecd8cb5_0  
tqdm                      4.66.4           py38h20db666_0  
typing_extensions         4.12.2             pyha770c72_0    conda-forge
urllib3                   2.2.2            py38hecd8cb5_0  
werkzeug                  3.0.3            py38hecd8cb5_0  
wheel                     0.43.0           py38hecd8cb5_0  
wrapt                     1.16.0           py38hae2e43d_0    conda-forge
xz                        5.4.6                h6c40b1e_1  
yarl                      1.9.4            py38hae2e43d_0    conda-forge
zipp                      3.19.2             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h87427d6_6    conda-forge
zstd                      1.4.9                h322a384_0
@JeanMainguy
Copy link
Member

Hi,

Unfortunately, I'm not able to reproduce the error on my end.

I ran the CI on a macOS runner via GitHub to see if the problem might be OS-related. The CI passes without any issues (check it out here: CI Run). However, the GitHub macOS version is 13, which is lower than your version.

The error in your log seems to come from this line:

/bin/sh: line 1: 20164 Illegal instruction: 4  diamond blastp --outfmt 6 --max-target-seqs 1 --query test_results/temporary_files/assembly_proteins.faa -o test_results/temporary_files/diamond_result.tsv --threads 1 --db checkm2_tiny_db/checkm2_tiny_db.dmnd --query-cover 80 --subject-cover 80 --id 30 --evalue 1e-05 --block-size 2 2> test_results/temporary_files/diamond_result.log

I found this Stack Overflow question: What is the illegal instruction 4 error and why does it happen? and this GitHub issue: DIAMOND Issue #303.

It seems the issue might be related to the DIAMOND conda package not being compatible with your version of macOS.

@JeanMainguy
Copy link
Member

You might want to try installing a Binette environment with a more recent version of DIAMOND. Here's how you can do it using pip within a conda environment:

First, create a conda environment with Python 3.8 and DIAMOND 2.1.9:

conda create -n binette_with_diamond2.1.9 python=3.8 diamond=2.1.9
conda activate binette_with_diamond2.1.9

Then, install Binette using pip with the main_deps flag to ensure all dependencies are installed:

pip install binette[main_deps]

This will install all the necessary Python packages without checking the DIAMOND version, which is locked to 2.0.4 in the conda package as mentioned in #13.

Once you have everything set up, you can retry the Binette test as you did before:

git clone https://github.com/genotoul-bioinfo/Binette_TestData.git

cd Binette_TestData
binette -b binning_results/*binning --contigs all_contigs.fna --checkm2_db checkm2_tiny_db/checkm2_tiny_db.dmnd -v -o test_results

@JeanMainguy JeanMainguy added bug Something isn't working joss_review issue related to the joss review labels Jul 10, 2024
@beardymcjohnface
Copy link
Author

Hi Jean,
That makes sense. I have not tried the instructions for installing a newer version of diamond, but Binette works perfectly on my linux system!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working joss_review issue related to the joss review
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@beardymcjohnface @JeanMainguy