Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor for parallelization #17

Open
kzuberi opened this issue Jan 8, 2015 · 0 comments
Open

refactor for parallelization #17

kzuberi opened this issue Jan 8, 2015 · 0 comments

Comments

@kzuberi
Copy link
Contributor

kzuberi commented Jan 8, 2015

Many of the data processing steps can be executed in parallel, however some steps particularly in the engine data portion of the pipeline are currently unnecessarily serial.

In particular, the cache-builder step creates one output at a time although those files are independent, reorganizing the snakefile and possibly revising corresponding engine program would speed this bottleneck.

Other steps in this part of the process may also lend themselves to similar reorganization (needs investigation).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant