feat: Completely Revamped Clustering Algorithm

[TurtIeSocks]

Changes

Algorithms

• New algorithm that leverages RTrees to be lightning fast and provides huge improvements to the final result
• Previous Balanced and BruteForce algorithms have been removed/replaced entirely, as they're worse in every way and are no longer worth maintaining
• Now there are:
  • Fastest (Previously fast algorithm)
  • Fast
  • Balanced (utilizes new algorithm though)
  • Better
• Logs provide live % status of the algorithm progress

Stats

• Stats have been completely reworked to be calculated more consistently, also utilizing an RTree
• route_time is now returned from the API
• mygod_score is also now returned

UI

• Route is no longer a selectable mode, rather you choose TSP for the value of Sort By in clustering mode
• Fixes the double scrollbar in the menu that sometimes appeared
• View full route stats when exporting a route, calculated on the server

API

• For cluster_mode, BruteForce now maps to Better
• only_unique has been deprecated and is no longer used
• /api/v1/calc/route/{category} is still active, but you're encouraged to just use the cluster endpoint with the sort_by argument set to TSP
• Adds max_clusters arg, this limits the algorithm to only accept the top X amount of clusters (by number of points they cover)

Misc

• Some struct declarations were moved around to more appropriate crates
• External crates were updated
• Minimum Rust version is now 1.68

Benchmarks

## Fastest
      [STATS] =================================================================
      || [AREA] Aalst | Fastest | Radius                                     ||
      || [POINTS] Total: 11064 | Covered: 10858                              ||
      || [CLUSTERS] Total: 1359 | Avg Points: 7                              ||
      || [BEST_CLUSTER] Amount: 1 | Point Count: 35                          ||
      || [TIMES] Clustering: 0.02s | Routing: 0.00s | Stats: 0.09s           ||
      || [MYGOD_SCORE] 4283                                                  ||
      =========================================================================
## Balanced (old)
      [STATS] =================================================================
      || [AREA] Aalst | Balanced | Radius                                    ||
      || [POINTS] Total: 11064 | Covered: 10998                              ||
      || [CLUSTERS] Total: 1196 | Avg Points: 9                              ||
      || [BEST_CLUSTER] Amount: 1 | Point Count: 32                          ||
      || [TIMES] Clustering: 3.85s | Routing: 0.00s | Stats: 0.06s           ||
      || [MYGOD_SCORE] 3654                                                  ||
      =========================================================================
## Brute Force (old)
      [STATS] =================================================================
      || [AREA] Aalst | BruteForce | Radius                                  ||
      || [POINTS] Total: 11064 | Covered: 10487                              ||
      || [CLUSTERS] Total: 931 | Avg Points: 11                              ||
      || [BEST_CLUSTER] Amount: 0 | Point Count: 0                           ||
      || [TIMES] Clustering: 8.00s | Routing: 0.00s | Stats: 0.06s           ||
      || [MYGOD_SCORE] 3370                                                  ||
      =========================================================================
## Fast
      [STATS] =================================================================
      || [AREA] Aalst | Fast | Radius                                        ||
      || [POINTS] Total: 11064 | Covered: 10631                              ||
      || [CLUSTERS] Total: 865 | Avg Points: 12                              ||
      || [BEST_CLUSTER] Amount: 1 | Point Count: 45                          ||
      || [TIMES] Clustering: 2.35s | Routing: 0.00s | Stats: 0.06s           ||
      || [MYGOD_SCORE] 3028                                                  ||
      =========================================================================
## Balanced (new)
      [STATS] =================================================================
      || [AREA] Aalst | Balanced | Radius                                    ||
      || [POINTS] Total: 11064 | Covered: 10724                              ||
      || [CLUSTERS] Total: 864 | Avg Points: 12                              ||
      || [BEST_CLUSTER] Amount: 1 | Point Count: 44                          ||
      || [TIMES] Clustering: 5.02s | Routing: 0.00s | Stats: 0.06s           ||
      || [MYGOD_SCORE] 2932                                                  ||
      =========================================================================
## Better
      [STATS] =================================================================
      || [AREA] Aalst | Better | Radius                                      ||
      || [POINTS] Total: 11064 | Covered: 10734                              ||
      || [CLUSTERS] Total: 841 | Avg Points: 12                              ||
      || [BEST_CLUSTER] Amount: 1 | Point Count: 45                          ||
      || [TIMES] Clustering: 78.58s | Routing: 0.00s | Stats: 0.07s          ||
      || [MYGOD_SCORE] 2853                                                  ||
      =========================================================================

Resolves #169
Resolves #162
Closes #121
Resolves #116 (hopefully)

[jfberry]

You may find switching allocator allows process to shrink better. Eg https://lib.rs/crates/jemallocator or https://lib.rs/crates/mimalloc (mimalloc sounds potentially more attractive)
Found by reading this web page https://www.reddit.com/r/rust/comments/fffth0/memory_not_released_after_allocate_a_lot_of/

[TurtIeSocks]

You may find switching allocator allows process to shrink better. Eg https://lib.rs/crates/jemallocator or https://lib.rs/crates/mimalloc (mimalloc sounds potentially more attractive) Found by reading this web page https://www.reddit.com/r/rust/comments/fffth0/memory_not_released_after_allocate_a_lot_of/

Thanks! I'll give it a try tomorrow morning.

[TurtIeSocks]

Latest benchmarks:

      [STATS] =================================================================
      || [AREA] Amsterdam | Fastest | Radius                                 ||
      || [POINTS] Total: 76692 | Covered: 76198                              ||
      || [CLUSTERS] Total: 9549 | Avg Points: 7                              ||
      || [BEST_CLUSTER] Amount: 1 | Point Count: 42                          ||
      || [TIMES] Clustering: 0.09s | Routing: 0.00s | Stats: 0.47s           ||
      || [MYGOD_SCORE] 29141                                                 ||
      =========================================================================

      [STATS] =================================================================
      || [AREA] Amsterdam | Fast | Radius                                    ||
      || [POINTS] Total: 76692 | Covered: 73941                              ||
      || [CLUSTERS] Total: 5857 | Avg Points: 12                             ||
      || [BEST_CLUSTER] Amount: 1 | Point Count: 53                          ||
      || [TIMES] Clustering: 2.93s | Routing: 0.00s | Stats: 0.47s           ||
      || [MYGOD_SCORE] 20322                                                 ||
      =========================================================================

      [STATS] =================================================================
      || [AREA] Amsterdam | Balanced | Radius                                ||
      || [POINTS] Total: 76692 | Covered: 74582                              ||
      || [CLUSTERS] Total: 5834 | Avg Points: 12                             ||
      || [BEST_CLUSTER] Amount: 1 | Point Count: 53                          ||
      || [TIMES] Clustering: 8.31s | Routing: 0.00s | Stats: 0.47s           ||
      || [MYGOD_SCORE] 19612                                                 ||
      =========================================================================

      [STATS] =================================================================
      || [AREA] Amsterdam | Better | Radius                                  ||
      || [POINTS] Total: 76692 | Covered: 74641                              ||
      || [CLUSTERS] Total: 5690 | Avg Points: 13                             ||
      || [BEST_CLUSTER] Amount: 1 | Point Count: 54                          ||
      || [TIMES] Clustering: 41.12s | Routing: 0.00s | Stats: 0.47s          ||
      || [MYGOD_SCORE] 19121                                                 ||
      =========================================================================

[TurtIeSocks]

You may find switching allocator allows process to shrink better. Eg https://lib.rs/crates/jemallocator or https://lib.rs/crates/mimalloc (mimalloc sounds potentially more attractive) Found by reading this web page https://www.reddit.com/r/rust/comments/fffth0/memory_not_released_after_allocate_a_lot_of/

Thanks for the suggestion! I tested both and mimalloc does indeed seem to perform quite a bit better with Linux, seems to be junk for MacOS but easy to exclude it in that case. Jemallocator worked pretty nicely too but on average mimalloc retained less after operations completed.