Multi node tweaks

[Delaunay]

No description provided.

[Delaunay]

=================
Benchmark results
=================
bench                          | fail |   n |       perf |   sem% |   std% | peak_memory |      score | weight
brax                           |    0 |   1 |  705221.22 |   0.2% |   1.4% |        2670 |  705221.22 |   1.00
dinov2-giant-gpus              |    0 |   1 |     290.27 |   1.4% |  11.0% |       69546 |     290.27 |   1.00
dinov2-giant-nodes             |    1 |   2 |        nan |   nan% |   nan% |         nan |        nan |   0.00
dinov2-giant-single            |    0 |   8 |      45.93 |   0.9% |  19.5% |       74646 |     371.12 |   0.00
bf16                           |    0 |   8 |     293.43 |   0.2% |   6.2% |        1788 |    2360.98 |   0.00
fp16                           |    0 |   8 |     290.16 |   0.1% |   3.8% |        1788 |    2329.58 |   0.00
fp32                           |    0 |   8 |      19.13 |   0.0% |   0.7% |        2166 |     153.10 |   0.00
tf32                           |    0 |   8 |     146.89 |   0.1% |   3.7% |        2166 |    1179.13 |   0.00
bert-fp16                      |    0 |   8 |     260.00 |   1.1% |  17.4% |         nan |    2134.01 |   0.00
bert-fp32                      |    0 |   8 |      44.57 |   0.7% |  10.4% |       21170 |     362.77 |   0.00
bert-tf32                      |    0 |   8 |     140.28 |   1.0% |  15.0% |        1704 |    1148.08 |   0.00
bert-tf32-fp16                 |    0 |   8 |     259.09 |   1.2% |  18.3% |         nan |    2125.99 |   3.00
reformer                       |    0 |   8 |      61.96 |   0.3% |   6.4% |       25404 |     499.36 |   1.00
t5                             |    0 |   8 |      50.01 |   0.6% |  12.9% |       34390 |     404.81 |   2.00
whisper                        |    0 |   8 |     425.67 |   1.1% |  24.5% |         nan |    3438.69 |   1.00
lightning                      |    0 |   8 |     467.95 |   1.5% |  33.5% |       27360 |    3772.13 |   0.00
lightning-gpus                 |    0 |   1 |    1937.94 |   4.4% |  34.8% |       28178 |    1937.94 |   1.00
llama                          |    0 |   8 |     429.17 |   4.5% |  81.4% |       27820 |    3276.65 |   1.00
llm-full-mp-gpus               |    0 |   1 |      86.50 |   2.6% |  13.7% |       48400 |      86.50 |   1.00
llm-full-mp-nodes              |    1 |   2 |      49.97 |   1.6% |   8.4% |       35658 |      24.99 |   1.00
llm-lora-ddp-gpus              |    0 |   1 |    8995.26 |   1.2% |   6.2% |       36988 |    8995.26 |   1.00
llm-lora-ddp-nodes             |    1 |   2 |    4493.43 |   0.5% |   2.3% |       35678 |    2246.71 |   1.00
llm-lora-mp-gpus               |    0 |   1 |    1146.41 |   1.5% |   7.7% |       55972 |    1146.41 |   1.00
llm-lora-single                |    0 |   8 |    2717.90 |   0.2% |   3.1% |       49926 |   21797.19 |   1.00
super-slomo                    |    0 |   8 |      44.44 |   1.2% |  26.1% |       65928 |     358.16 |   1.00
focalnet                       |    0 |   8 |     341.79 |   0.9% |  20.2% |       23536 |    2765.88 |   2.00
convnext_large-fp16            |    0 |   8 |     331.04 |   1.2% |  18.6% |       27376 |    2714.04 |   0.00
convnext_large-fp32            |    0 |   8 |      59.16 |   0.7% |  11.2% |       55950 |     482.25 |   0.00
convnext_large-tf32            |    0 |   8 |     154.15 |   1.0% |  14.9% |       49650 |    1263.82 |   0.00
convnext_large-tf32-fp16       |    0 |   8 |     331.46 |   1.2% |  18.5% |       27376 |    2717.71 |   3.00
regnet_y_128gf                 |    0 |   8 |     115.71 |   0.6% |  12.4% |       29762 |     936.62 |   2.00
resnet152-ddp-gpus             |    0 |   1 |    2040.41 |   5.1% |  38.9% |       27988 |    2040.41 |   0.00
resnet50                       |    0 |   8 |     782.16 |   1.6% |  34.6% |       14848 |    6298.85 |   1.00
resnet50-noio                  |    0 |   8 |    1161.02 |   0.3% |   6.8% |       27480 |    9363.26 |   0.00

Scores
------
Failure rate:       1.46% (FAIL)
Score:            1711.32

Errors
------
3 errors, details in HTML report.