[DO NOT MERGE] hpcSPAdes rebase, cleanup & implementation#1380

Open

asl wants to merge 109 commits intomainfrom mpi-ng

+7,389-772

Commits on Dec 18, 2024

Try to use concurrent pair info buffer in pair info improver
asl
committed

Commits on Dec 23, 2024

Commits on Dec 24, 2024

CMAKE: Hook in MPI

eodus
authored and
asl
committed
Add MPI runtime detection
asl
committed
CMAKE: Install spades-hpc

eodus
authored and
asl
committed
Add mpi_console_writer log_writer interface impl

eodus
authored and
asl
committed
Factor out mpi log writers as well
asl
committed
Add mpi partask core
asl
committed
Add MPI test
asl
committed
User-friendly rank reporting
asl
committed
Rudimentrary MPI stage manager & MPI stage
asl
committed
Add TestMPI stage

eodus
authored and
asl
committed
Separate out SPAdes and hpcSPAdes binaries
asl
committed
Even better SPAdes / hpcSPAdes separation
asl
committed
read_converter: changable chunk_num

olga24912
authored and
asl
committed
MPI: sequence mapper notifier

olga24912
authored and
asl
committed
MPI mismatch_corrector

olga24912
authored and
asl
committed
MPI: Gap closer

olga24912
authored and
asl
committed
MPI: pair_info_count

olga24912
authored and
asl
committed
Move implementation to .cpp. Small cleanup while there
asl
committed
Allow name ot be overriden
asl
committed
Factor out implementation into a separate class
asl
committed
Simplify parallel processing
asl
committed
Simplify
asl
committed
Ensure the streams are fresh
asl
committed
Add MPI construction stage
asl
committed
First real MPI construction step: collect k-mer coverage in parallel
asl
committed
add comment to DistanceEstimation

olga24912
authored and
asl
committed
DistanceEstimator MPI

olga24912
authored and
asl
committed
Distance Estimator MPI stage

olga24912
authored and
asl
committed
DistanceEstimator MPI wrapper

olga24912
authored and
asl
committed
MPI GraphCondensing

olga24912
authored and
asl
committed
remove Seq from extansion index

olga24912
authored and
asl
committed
remove friend from UnbranchingPathExtractor

olga24912
authored and
asl
committed
style fix

olga24912
authored and
asl
committed
MPI EarlyTipClipper

olga24912
authored and
asl
committed
MPI Build Extension Index

olga24912
authored and
asl
committed
comments for MergeKmerFileTask

olga24912
authored and
asl
committed
verification that after split the #buckets the same in all storages

olga24912
authored and
asl
committed
make kpostorage const

olga24912
authored and
asl
committed
comment for release_all

olga24912
authored and
asl
committed
add comment for Splitter

olga24912
authored and
asl
committed
resize one time in MergeKMers

olga24912
authored and
asl
committed
VERIFY kmers in kmer storage are sorted and unique

olga24912
authored and
asl
committed
fix warnings

olga24912
authored and
asl
committed
MPI KMerCounter

olga24912
authored and
asl
committed
close stream after use

olga24912
authored and
asl
committed
constat for contig output stage

olga24912
authored and
asl
committed
add sync for read conversion

olga24912
authored and
asl
committed
MPI ATtipClipper

olga24912
authored and
asl
committed
reduce code dublication

olga24912
authored and
asl
committed
fix pair_info_counter

olga24912
authored and
asl
committed
detach edge index on load

olga24912
authored and
asl
committed
detach edge index at the end of pair_info_counter

olga24912
authored and
asl
committed
create lib Spades-MPI

olga24912
authored and
asl
committed
separate distEst

olga24912
authored and
asl
committed
separate construction_mpi

olga24912
authored and
asl
committed
separate test_mpi

olga24912
authored and
asl
committed
update distEst Arhitecture

olga24912
authored and
asl
committed
make SeqMapperNot in GCMPI as in GC

olga24912
authored and
asl
committed
GapCloserBase

olga24912
authored and
asl
committed
Separate MPI gap closer

olga24912
authored and
asl
committed
mismatch_corrector with functor

olga24912
authored and
asl
committed
declarate MismatchShallNotPass to hpp

olga24912
authored and
asl
committed
separate MPI mismatch_correction

olga24912
authored and
asl
committed
make pair_info_count consistent with master version

olga24912
authored and
asl
committed
functor for FillEdgePairFilter

olga24912
authored and
asl
committed
separate PairInfoCount MPI

olga24912
authored and
asl
committed
MapLibFabric

olga24912
authored and
asl
committed
Separate SeqMapperNotifier

olga24912
authored and
asl
committed
PerfectHashMapperBuilder MPI

olga24912
authored and
asl
committed
move mpi_kmer_index_builder to mpi dir

olga24912
authored and
asl
committed
separate perfect_hash_map_builder_mpi

olga24912
authored and
asl
committed
move kmer_extension_index_builder_mpi to mpi dir

olga24912
authored and
asl
committed
Move partask and mpi stage to hpcSPAdes
asl
committed
run_on_load stage type

olga24912
authored and
asl
committed
namespace spaces

olga24912
authored and
asl
committed
rename function by code style

olga24912
authored and
asl
committed
delete mpi from local pipeline

olga24912
authored and
asl
committed
separate logger mpi

olga24912
authored and
asl
committed
Add time tracer annotations for MPI stage manager. Some cleanup here and there
asl
committed
Do not overwrite time traces from different nodes
asl
committed
Cleanup
asl
committed
Time tracing for partask
asl
committed
Better time tracing
asl
committed
More information
asl
committed
Annotate PathExtend
asl
committed
A bit more verbosity
asl
committed
More events + some cleanups
asl
committed
call process lib func in Mismatch Corrector

olga24912
authored and
asl
committed
fix: allreduce only in sync

olga24912
authored and
asl
committed
fix: block size/NNodes

olga24912
authored and
asl
committed
sequence mapper notifier MPI in paired info counter

olga24912
authored and
asl
committed
PairedInfoCounter SeqMapNot MPI

olga24912
authored and
asl
committed
split streams on allthreads cnt

olga24912
authored and
asl
committed
Move MPI detection down to project
asl
committed
Normalize include paths
asl
committed
Add spades.py bits
asl
committed
Add hpcSPAdes to list of known projects
asl
committed
Add SLURM executor
asl
committed
Better job names
asl
committed
Fix some defaults
asl
committed
Run stuff via srun by default
asl
committed
Fix some broken deps
asl
committed

Commits

Commits on Dec 18, 2024

Commits on Dec 20, 2024

Commits on Dec 23, 2024

Commits on Dec 24, 2024