IGV Error

[barissalman]

Describe the bug
A clear and concise description of what the bug is, and what you instead expect to be happening

I have a CRAM file which I confirm opens when I use the igv web app at igv.org. However, IGV at seqr gives CRC block error.

Browser console looks like this:

Error: crc mismatch in block data: recorded CRC32 = 955321020, but calculated CRC32 = 102785351
    s http://10.10.10.26/static/19-ed35377a.js:1
    e http://10.10.10.26/static/19-ed35377a.js:1
    e http://10.10.10.26/static/19-ed35377a.js:1
    e http://10.10.10.26/static/19-ed35377a.js:1
    t http://10.10.10.26/static/19-ed35377a.js:1
    l http://10.10.10.26/static/19-ed35377a.js:1
    _invoke http://10.10.10.26/static/19-ed35377a.js:1
    e http://10.10.10.26/static/19-ed35377a.js:1
    i http://10.10.10.26/static/19-ed35377a.js:1
    a http://10.10.10.26/static/19-ed35377a.js:1
[19-ed35377a.js:1:230550](http://10.10.10.26/static/19-ed35377a.js)
Error: crc mismatch in block data: recorded CRC32 = 955321020, but calculated CRC32 = 102785351
    s http://10.10.10.26/static/19-ed35377a.js:1
    e http://10.10.10.26/static/19-ed35377a.js:1
    e http://10.10.10.26/static/19-ed35377a.js:1
    e http://10.10.10.26/static/19-ed35377a.js:1
    t http://10.10.10.26/static/19-ed35377a.js:1
    l http://10.10.10.26/static/19-ed35377a.js:1
    _invoke http://10.10.10.26/static/19-ed35377a.js:1
    e http://10.10.10.26/static/19-ed35377a.js:1
    i http://10.10.10.26/static/19-ed35377a.js:1
    a http://10.10.10.26/static/19-ed35377a.js:1

Link to page(s) where bug is occurring

I have a local installation. Happens in any IGV in seqr.

Scope of the bug
Are you experiencing this issue in all projects? Within specific families within one project?

I just have a single demo file.

Screenshots
If applicable, add screenshots to help explain your problem.

I tried the following:

1. I pulled the repo and build the image again from the main but error continued.
2. I tried to update the igv version to 2.15.11 in ui/package.json but it broke the ui :( .

[hanars]

I would recommend you delete the IGV file from you mounted volume a put in a fresh, uncorrupted copy and see if that fixes the issue. Its likely that something got malformed with the data itself and that is causing your issue

[barslmn]

You mean the cram file right?

[hanars]

yes, and also the index file if it exists and anything else associated with it

[barslmn]

I removed the old cram/crai files and copied again, also checked for md5sum.
Still the same error :/. I will do bit more investigating.

[hanars]

Some other questions that may help me understand your problem better: Are your other crams working? And also, did this cram ever work in seqr, or no?

[barslmn]

Trying bam in seqr

It's the first cram I tried in the seqr. I haven't try any other yet.
I converted the file to bam file with samtools and (edited the /admin/igvsample/ entry) was able to see the alignments without a problem.

trying igv.org/app

I can open the this particular cram file in igv.org/app.

trying igvjs v.2.10.5 in a dummy app

I also tested if cram works with igvjs v2.10.5 works in a standalone flask app which also opens.

[barslmn]

I stumbled upon some weird behaviour.

Cram file is sliced by dd in file_utils.py. However, dd's stderr is also in stdout, subprocess couldn't parse it maybe?
And also +1 added to length in igv_api.py. So responses from api /readviz/ have character "3" at the end because length of the bytes are 32768 and dd writes this number at the end. I think the first character of dd's stderr (char 3) is parsed into the CRAM data chunk for some reason.

>>> cmd = subprocess.Popen("dd skip=653197312 count=32768 bs=1 if=test.cram", stdout=subprocess.PIPE, stderr=subprocess.PIPE if False else subprocess.STDOUT, shell=True)

>>> [line for line in cmd.stdout][-3:]
[b'!\xfb^\x08\xfam\x18\\\x0c\xf1\xd8\x1c1\xeff\x83\x19)\xb2\x92b\x9fG\xc7\xa4\x85x\xdcJ\xceW\xab\xb9\x85x2\x8f\x1c\xfe\x90#K\xfb\xbb\x1cy\x1f?\x85:\xfa\x04\xd0\x1c\xb8w\xa7\xec\xc2\x80]\xe8\xbd\x8d\xb0\xd2R\x8cv\xc8\xf3n;\xfe\x0532768+0 records in\n', b'32768+0 records out\n', b'32768 bytes (33 kB, 32 KiB) copied, 4.83599 s, 6.8 kB/s\n']

with 2>/dev/null in the command.

>>> cmd = subprocess.Popen("dd skip=653197312 count=32768 bs=1 if=test.cram 2>/dev/null", stdout=subprocess.PIPE, stderr=subprocess.PIPE if False else subprocess.STDOUT, shell=True)

>>> [line for line in cmd.stdout][-1]
b'!\xfb^\x08\xfam\x18\\\x0c\xf1\xd8\x1c1\xeff\x83\x19)\xb2\x92b\x9fG\xc7\xa4\x85x\xdcJ\xceW\xab\xb9\x85x2\x8f\x1c\xfe\x90#K\xfb\xbb\x1cy\x1f?\x85:\xfa\x04\xd0\x1c\xb8w\xa7\xec\xc2\x80]\xe8\xbd\x8d\xb0\xd2R\x8cv\xc8\xf3n;\xfe\x05'

[hanars]

hmm okay looks like we should probably modify the dd command to include status='none' to surpress all the stats being wirtten to stderr. Feel free to submit a pull request to the seqr repo to make that change!

[barslmn]

I will test it on my seqr instance and let you know. I will be on vacation next week so it might wait the week after that. Also curious about why we are seeing this behaviour with cram and not bam :?

[hanars]

proabbaly BAM is loading with out a range header, so going down different code branch and executing a different command

[barslmn]

Alright, I think I got it. It is because this length here does not match this length here.

Off by one :)

Here is the PR:
#3762