Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce overhead #436

Merged
merged 190 commits into from
Jul 7, 2022
Merged

Reduce overhead #436

merged 190 commits into from
Jul 7, 2022

Conversation

rowleya
Copy link
Member

@rowleya rowleya commented Jun 27, 2022

Reduce the memory footprint and improve the operation with new algorithms. Includes:

  1. Drop some constraints that no longer make sense or are not used.
  2. Deal with iteration of keys elsewhere (as it is always the same anyway)
  3. There is no machine graph! There are still machine objects linked though.
  4. Edge partitions can exist outside of graphs (indeed machine partitions are now used directly in splitters)
  5. Splitters now handle same-chip-as "constraint" through explicit grouping obtained by calling a method (this means that things to go on the same chip must be handled by a single splitter, but this was already the case in all examples).
  6. Splitters can filter their incoming vertices to match with a source. This takes on the equivalent role of filtering edges.
  7. There is a new placer that ensures that vertices are placed on chips adjacent to each other. This is needed by the new routing algorithm so all other placers have been removed!
  8. There is a new routing algorithm that takes advantage of the placer above, and also makes edges from the same source application vertex take the same path to their destinations. Other routing algorithms are still available because there is no further downstream issues.
  9. Some compression of routing can be done on routing table generation. This compresses adjacent keys with the same route.
  10. Routing information generation is always done per-application vertex, though some additional per-machine-vertex information is also produced. Both exist downstream. Other routing information algorithms have been removed.
  11. Partitioning is always done to max_atoms_per_core. Placement will fail if this is not good enough.
  12. Partitioning uses a much simpler "chip counter" to estimate boards required (which appears more successful than master despite its simplicity).
  13. Virtual vertices are handled directly rather than allocating them to a place on the machine (i.e. they are dealt with by routing down the link to them from the real chip rather than adding a virtual chip).
  14. Some useful functions have been added to placements which are now ordered by chip.
  15. Routing tables have additional checks to ensure they don't contain both application and machine versions of a vertex (which can be fatal).
  16. Lots of data structures have been removed and their consequences dealt with (removing functions, using different ordering etc.).

Depends on SpiNNakerManchester/SpiNNMachine#175

This was referenced Jul 5, 2022
Copy link
Member

@Christian-B Christian-B left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

created a number of issue for follow up work.

Including getting coverage back up.

pacman/model/placements/placement.py Outdated Show resolved Hide resolved
pacman/model/routing_info/vertex_routing_info.py Outdated Show resolved Hide resolved
pacman/model/routing_info/vertex_routing_info.py Outdated Show resolved Hide resolved
pacman/model/routing_info/routing_info.py Outdated Show resolved Hide resolved
pacman/model/routing_info/routing_info.py Outdated Show resolved Hide resolved
pacman/operations/placer_algorithms/application_placer.py Outdated Show resolved Hide resolved
pacman/operations/placer_algorithms/application_placer.py Outdated Show resolved Hide resolved
pacman/utilities/utility_objs/chip_counter.py Show resolved Hide resolved
@rowleya rowleya merged commit 8571a72 into master Jul 7, 2022
@rowleya rowleya deleted the reduce_overhead branch July 7, 2022 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants