bam to counts conversion

[cansavvy]

Description

This is adding the bam to counts conversion that's done by the custom scripts counter_efficient.Rand combine_counts.py in the original pipeline.

The only thing I find confusing is that the counts I'm getting (after calculating weights) do not turn out as whole numbers. And I'm not sure why they do in the original data. <\del>

Figured it out! See below. It's lining up with the expected counts from the original pipeline configuration. 🎉

Type of change

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

I haven't developed unit tests for this however the examples documented in the code are working examples and are how I have tested these functions.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[cansavvy]

I figured it out. Here's a plot to prove it. I'm putting proof of concept plots in the inst/extdata/rmd folder here so we have them for our records.

[cansavvy]

Also did some package docs updates

[marissafujimoto]

@cansavvy Looks great overall! Much cleaner compared to the original pipeline! Ship it.

My biggest thought when reading this (and I think it's out of scope for this PR) is whether we can do this count without loading the bams into memory. Instead maybe iteratively building up the counts and using O(# paired guides) memory by streaming the bam files.

Another thought I had was whether the bam / alignment step is necessary given that mageck argues that you should go directly from fastq and not count partial alignments, though I'm not sure if this should be different for paired guides or how much of our example data is perfectly aligned: https://sourceforge.net/p/mageck/wiki/advanced_tutorial/#tutorial-1-allow-mismatches-for-read-mapping

[cansavvy]

@cansavvy Looks great overall! Much cleaner compared to the original pipeline! Ship it.

I'll ship it after I make the tests pass. 👍

My biggest thought when reading this (and I think it's out of scope for this PR) is whether we can do this count without loading the bams into memory. Instead maybe iteratively building up the counts and using O(# paired guides) memory by streaming the bam files.

I ran this locally without a problem (took maybe 30 sec, 10 sec a sample) but certainly saving compute time and memory is good. And we should determine how big are the biggest files these assays might have to think more specifically about bench marking.

Another thought I had was whether the bam / alignment step is necessary given that mageck argues that you should go directly from fastq and not count partial alignments, though I'm not sure if this should be different for paired guides or how much of our example data is perfectly aligned: https://sourceforge.net/p/mageck/wiki/advanced_tutorial/#tutorial-1-allow-mismatches-for-read-mapping

Interesting thanks for sharing! I'll dig into this tomorrow!

[cansavvy]

Still some troubles with the Rsamtools dependency but I'll move on