Llama 3

[Delaunay]

No description provided.

[Delaunay]

Issue is pytorch 2.3, works for 2.4

=================
Benchmark results
=================
bench                          | fail |   n |       perf |   sem% |   std% | peak_memory |      score | weight
brax                           |    0 |   1 |  725222.36 |   0.1% |   0.6% |        2676 |  725222.36 |   1.00
diffusion-gpus                 |    0 |   1 |     113.37 |   1.7% |  13.1% |       57798 |     113.37 |   1.00
dinov2-giant-gpus              |    0 |   1 |     446.19 |   0.3% |   2.7% |       69994 |     446.19 |   1.00
bf16                           |    0 |   8 |     293.82 |   0.3% |   6.7% |        1788 |    2365.54 |   0.00
fp16                           |    0 |   8 |     289.36 |   0.2% |   4.3% |        1788 |    2324.21 |   0.00
fp32                           |    0 |   8 |      19.15 |   0.0% |   0.7% |        2166 |     153.31 |   0.00
tf32                           |    0 |   8 |     146.63 |   0.1% |   3.7% |        2166 |    1177.05 |   0.00
bert-fp16                      |    0 |   8 |     264.89 |   1.0% |  15.4% |         nan |    2174.26 |   0.00
bert-fp32                      |    0 |   8 |      44.84 |   0.6% |   9.8% |       21170 |     364.66 |   0.00
bert-tf32                      |    0 |   8 |     142.00 |   0.9% |  13.6% |         nan |    1162.31 |   0.00
bert-tf32-fp16                 |    0 |   8 |     264.64 |   1.0% |  15.3% |         nan |    2171.79 |   3.00
reformer                       |    0 |   8 |      62.32 |   0.3% |   6.1% |       25404 |     502.16 |   1.00
t5                             |    0 |   8 |      51.34 |   0.5% |  10.0% |       34390 |     415.60 |   2.00
whisper                        |    0 |   8 |     415.89 |   1.4% |  30.9% |        9278 |    3352.85 |   1.00
lightning                      |    0 |   8 |     185.85 |   0.9% |  21.1% |        4862 |    1502.57 |   0.00
lightning-gpus                 |    0 |   1 |    1275.39 |   3.3% |  25.9% |        5690 |    1275.39 |   1.00
llama                          |    0 |   8 |     431.60 |   4.5% |  81.8% |       27820 |    3270.45 |   1.00
llm-full-mp-gpus               |    1 |   1 |        nan |   nan% |   nan% |         nan |        nan |   1.00
llm-lora-ddp-gpus              |    0 |   1 |   16842.49 |   0.4% |   2.1% |       36988 |   16842.49 |   1.00
llm-lora-mp-gpus               |    0 |   1 |    1987.22 |   2.3% |  11.9% |       55972 |    1987.22 |   1.00
llm-lora-single                |    0 |   8 |    2728.04 |   0.2% |   2.9% |       49926 |   21877.63 |   1.00
super-slomo                    |    0 |   8 |      44.85 |   1.2% |  26.7% |       66072 |     362.02 |   1.00
focalnet                       |    0 |   8 |     384.48 |   0.6% |  14.4% |       23586 |    3115.32 |   2.00
convnext_large-fp16            |    0 |   8 |     342.21 |   1.2% |  19.1% |       27392 |    2805.60 |   0.00
convnext_large-fp32            |    0 |   8 |      60.85 |   0.7% |  11.1% |       47300 |     495.89 |   0.00
convnext_large-tf32            |    0 |   8 |     157.93 |   0.9% |  14.7% |       49666 |    1294.95 |   0.00
convnext_large-tf32-fp16       |    0 |   8 |     347.88 |   1.0% |  16.3% |       27392 |    2853.46 |   3.00
regnet_y_128gf                 |    0 |   8 |     115.90 |   0.4% |   9.9% |       31614 |     938.02 |   2.00
resnet152-ddp-gpus             |    0 |   1 |    4284.58 |   3.7% |  28.4% |       28964 |    4284.58 |   0.00
resnet50                       |    0 |   8 |     916.45 |   2.3% |  51.8% |       14686 |    7393.95 |   1.00
resnet50-noio                  |    0 |   8 |    1161.65 |   0.3% |   7.3% |       27450 |    9374.04 |   0.00

Scores
------
Failure rate:       0.52% (PASS)
Score:            1621.60

Errors
------
1 errors, details in HTML report.

[Delaunay]

=================
Benchmark results
=================
bench                          | fail |   n |       perf |   sem% |   std% | peak_memory |      score | weight
brax                           |    0 |   1 |  729446.26 |   0.1% |   0.6% |        2670 |  729446.26 |   1.00
diffusion-gpus                 |    0 |   1 |     113.76 |   1.7% |  12.9% |       57786 |     113.76 |   1.00
diffusion-single               |    0 |   1 |     121.45 |   2.3% |  17.6% |       61884 |     121.45 |   0.00
dinov2-giant-gpus              |    0 |   1 |     447.65 |   0.3% |   2.5% |       69614 |     447.65 |   1.00
dinov2-giant-single            |    0 |   8 |      53.38 |   0.4% |   9.9% |       74646 |     431.61 |   0.00
bf16                           |    0 |   8 |     293.48 |   0.2% |   6.6% |        1788 |    2362.04 |   0.00
fp16                           |    0 |   8 |     289.12 |   0.1% |   3.8% |        1788 |    2321.12 |   0.00
fp32                           |    0 |   8 |      19.15 |   0.0% |   0.6% |        2166 |     153.30 |   0.00
tf32                           |    0 |   8 |     146.59 |   0.1% |   3.8% |        2166 |    1176.95 |   0.00
bert-fp16                      |    0 |   8 |     265.18 |   1.0% |  15.3% |         nan |    2176.03 |   0.00
bert-fp32                      |    0 |   8 |      44.79 |   0.6% |  10.1% |       21170 |     364.48 |   0.00
bert-tf32                      |    0 |   8 |     141.85 |   0.9% |  14.6% |        1984 |    1162.93 |   0.00
bert-tf32-fp16                 |    0 |   8 |     264.60 |   1.0% |  15.6% |         nan |    2172.08 |   3.00
reformer                       |    0 |   8 |      62.32 |   0.3% |   5.8% |       25404 |     502.05 |   1.00
t5                             |    0 |   8 |      51.38 |   0.4% |   9.9% |       34390 |     415.88 |   2.00
whisper                        |    0 |   8 |     435.08 |   1.1% |  24.9% |        9262 |    3513.18 |   1.00
lightning                      |    0 |   8 |     189.96 |   0.8% |  19.1% |        4802 |    1535.06 |   0.00
lightning-gpus                 |    0 |   1 |    1318.61 |   3.3% |  25.9% |        5546 |    1318.61 |   1.00
llama                          |    0 |   8 |     490.57 |   4.5% |  80.8% |       27820 |    3726.80 |   1.00
llm-full-mp-gpus               |    0 |   1 |     150.18 |   2.7% |  14.1% |       48780 |     150.18 |   1.00
llm-lora-ddp-gpus              |    0 |   1 |   16756.80 |   0.4% |   2.2% |       36988 |   16756.80 |   1.00
llm-lora-mp-gpus               |    0 |   1 |    1989.71 |   2.3% |  11.9% |       55972 |    1989.71 |   1.00
llm-lora-single                |    0 |   8 |    2722.30 |   0.2% |   3.0% |       49926 |   21834.06 |   1.00
super-slomo                    |    0 |   8 |      46.17 |   1.5% |  32.7% |       65928 |     372.07 |   1.00
focalnet                       |    0 |   8 |     372.87 |   0.7% |  15.3% |       23536 |    3020.88 |   2.00
convnext_large-fp16            |    0 |   8 |     335.74 |   1.0% |  16.1% |       27376 |    2754.57 |   0.00
convnext_large-fp32            |    0 |   8 |      59.45 |   0.7% |  10.5% |       55950 |     484.00 |   0.00
convnext_large-tf32            |    0 |   8 |     155.59 |   0.9% |  14.2% |       49650 |    1274.81 |   0.00
convnext_large-tf32-fp16       |    0 |   8 |     329.61 |   1.2% |  18.4% |       27376 |    2702.85 |   3.00
regnet_y_128gf                 |    0 |   8 |     117.67 |   0.5% |  10.1% |       29762 |     952.25 |   2.00
resnet152-ddp-gpus             |    0 |   1 |    3565.00 |   6.3% |  48.3% |       27980 |    3565.00 |   0.00
resnet50                       |    0 |   8 |     905.00 |   2.5% |  55.6% |       14848 |    7294.01 |   1.00
resnet50-noio                  |    0 |   8 |    1164.45 |   0.3% |   6.3% |       27480 |    9385.17 |   0.00