2022 10 24 Open NEST Developer Video Conference

previous | list | next

Agenda

1. Welcome
2. Review of NEST User Mailing List
3. Project team round
4. In-depth discussion
   • Compressed spikes

Mailing list

• https://nest-simulator.org/mailinglist

Project team round

Here we discuss topics that need broader attention, for example questions that came up but are outside a single project's scope, larger planned changes/PRs that affect all teams or pending work that is blocked by external factors.

1. Models / NESTML
   • working on refactoring "printers" for SpiNNaker back-end
     • design pattern: compositionality over inheritance
   • working on refactoring input-ports
2. NEST GPU
   • unifying python interface with pynest
   • working on 2.0 release
   • Q: API-server available?
     A: API is same as NEST, no difference from nest-server perspective.
3. PyNEST
   • pynest branch by @jougs and @hakonsmb with many accumulated changes due to kernel refactoring
   • Python wheel by @helveg and @jougs possible since last Friday, thanks to SLI-less NEST, merge coming
   • #2170 to be updated
4. NEST Desktop
   • issues causing the release to be postponed to November
   • looking for beta-testers! Contact @babsey!
   • moved default port of nest-server from 5000 to 52425
5. Kernel
   • fixed a number of bugs
   • see in-depth discussion
6. Installation & Infrastructure
   • worked on Spack package
   • started looking into containerization with Apptainer (former Singularity)
   • Testing
     • shift towards pytest'ing also kernel
     • huge test matrix (CO2 footprint!!)
     • smarter setup of tests required
     • interoperability testing
7. Documentation
   • C++ comments #2113 re-review
   • model updates #2416 re-review
   • Thu 2pm, HiRSE seminar on "Docs as Code"
8. EBRAINS
   • token handling for cloud+HPC installations with NEST Desktop is being worked on
   • test option requested for HPC operations staff (as discussed under Infrastructure)

In-depth discussion

Compressed Spikes (@heplesser)

• with some updates bug that connection information was not passed to pre-synaptic side
• lengthy debugging due to issues only showing up in multi-threaded mode
• conclusion: need a new aproach handling compressed spikes
  • till now update connection infrastructure postponed "compressed/not-compressed" decision to the latest point
  • not optimal, many things "come for free" with compressed spikes
• Discussed code in temporary branch of @heplesser. Look for:
  • update_connection_infrastructure()
  • EventDeliveryManager::gather target_data_compressed() L795
  • EventDeliveryManager::collocate_target_data_buffers_() L871
    • Now exploit that when preparing compression all selection work was already done: only need to call new method fill_target_buffer() in ConnectionManager L1643
      • avoids multiple insertion
      • only needs compressed spike data map (csd_maps)
      • no find/search operations required
      • Works on single thread
    • failing weight recorder test fixed
    • failing volume transmitter test fixed
  • SourceTable::collect_compressible_sources() L455, thread parallel, check if multiple targets on thread, now done for free
  • SourceTable::fill_compressed_spike_data() L481, check if source neuron has connection on any thread, later used for spike delivery to unpack the structure
    • still requires a "find" operation, which might not be strictly necessary since we have sorted data (potential for optimization) Conceptually one currently has the choice of
  • "sorted connections" yes/no (per thread compression)
  • "spike compression" yes/no (per rank compression) Maybe both or none is enough choice. In principle heuristic possible, e.g. "use compression only if 99% of neurons have targets on other threads".

Next steps:

• re-enable multiple rounds of connectivity communication
• with MPI some tests still fail