Benchmark Batch

[Delaunay]

No description provided.

[Delaunay]

=================
Benchmark results
=================
bench                          | fail |   n |       perf |   sem% |   std% | peak_memory |      score | weight
brax                           |    1 |   1 |    6469.30 |   0.0% |   0.1% |         nan |       0.00 |   1.00
diffusion-gpus                 |    1 |   1 |        nan |   nan% |   nan% |       28401 |        nan |   1.00
diffusion-single               |    1 |   1 |        nan |   nan% |   nan% |        3505 |        nan |   0.00
dinov2-giant-gpus              |    1 |   1 |        nan |   nan% |   nan% |       46021 |        nan |   1.00
dinov2-giant-single            |    4 |   4 |        nan |   nan% |   nan% |       28417 |        nan |   0.00
bf16                           |    0 |   4 |     282.41 |   0.2% |   4.4% |        1391 |    1134.03 |   0.00
fp16                           |    0 |   4 |     276.40 |   0.1% |   2.5% |        1391 |    1107.57 |   0.00
fp32                           |    0 |   4 |      49.29 |   0.1% |   1.3% |        1769 |     197.18 |   0.00
tf32                           |    0 |   4 |     143.37 |   0.1% |   2.3% |        1769 |     574.32 |   0.00
dimenet                        |    0 |   4 |     363.82 |   1.5% |  23.4% |        6183 |    1467.64 |   0.00
bert-fp16                      |    0 |   4 |     200.23 |   1.7% |  18.7% |         nan |     816.20 |   0.00
bert-fp32                      |    0 |   4 |      72.41 |   1.1% |  12.3% |        1491 |     293.17 |   0.00
bert-tf32                      |    0 |   4 |     117.56 |   1.4% |  15.0% |        1491 |     477.17 |   0.00
bert-tf32-fp16                 |    0 |   4 |     202.08 |   1.6% |  17.5% |         nan |     823.87 |   3.00
reformer                       |    0 |   4 |      29.75 |   0.2% |   2.8% |       25005 |     119.37 |   1.00
t5                             |    0 |   4 |      30.33 |   0.5% |   8.2% |       33991 |     122.19 |   2.00
whisper                        |    0 |   4 |     392.12 |   1.2% |  18.7% |        8837 |    1582.22 |   1.00
lightning                      |    0 |   4 |     351.16 |   2.2% |  35.4% |       26491 |    1412.54 |   0.00
lightning-gpus                 |    0 |   1 |    1080.26 |   4.5% |  35.6% |       26313 |    1080.26 |   1.00
llama                          |    0 |   4 |     275.94 |   6.9% |  88.2% |       27423 |    1047.76 |   1.00
llm-full-mp-gpus               |    1 |   1 |        nan |   nan% |   nan% |       26373 |        nan |   1.00
llm-lora-ddp-gpus              |    0 |   1 |    3327.81 |   1.0% |   5.6% |       32681 |    3327.81 |   1.00
llm-lora-mp-gpus               |    1 |   1 |        nan |   nan% |   nan% |       44739 |        nan |   1.00
llm-lora-single                |    0 |   4 |    2581.76 |   0.2% |   1.7% |       32649 |   10346.37 |   1.00
recursiongfn                   |    0 |   4 |    4693.88 |   2.3% |  36.4% |        8901 |   18718.74 |   0.00
super-slomo                    |    4 |   4 |        nan |   nan% |   nan% |         nan |        nan |   1.00
focalnet                       |    0 |   4 |     329.59 |   0.9% |  14.1% |       23149 |    1331.78 |   2.00
torchatari                     |    4 |   4 |        nan |   nan% |   nan% |        3377 |        nan |   0.00
convnext_large-fp16            |    0 |   4 |     253.50 |   1.5% |  16.9% |       26981 |    1034.76 |   0.00
convnext_large-fp32            |    0 |   4 |      76.77 |   1.0% |  11.4% |       45103 |     312.47 |   0.00
convnext_large-tf32            |    0 |   4 |     109.22 |   1.5% |  16.2% |       45695 |     445.01 |   0.00
convnext_large-tf32-fp16       |    0 |   4 |     252.13 |   1.6% |  17.9% |       26981 |    1029.01 |   3.00
regnet_y_128gf                 |    0 |   4 |      89.53 |   0.6% |   9.8% |       28923 |     361.48 |   2.00
resnet152-ddp-gpus             |    0 |   1 |    1239.94 |   5.4% |  40.9% |       26113 |    1239.94 |   0.00
resnet50                       |    0 |   4 |     446.42 |   2.1% |  33.4% |       13981 |    1798.23 |   1.00
resnet50-noio                  |    0 |   4 |     853.60 |   0.4% |   6.2% |       26997 |    3436.95 |   0.00

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 59.13043% with 94 lines in your changes missing coverage. Please review.

Project coverage is 71.27%. Comparing base (bdf7705) to head (c2ad383).
Report is 16 commits behind head on master.

Files with missing lines	Patch %	Lines
milabench/remote.py	17.24%	24 Missing ⚠️
milabench/system.py	73.23%	19 Missing ⚠️
milabench/cli/slurm.py	30.00%	14 Missing ⚠️
milabench/multi.py	22.22%	14 Missing ⚠️
milabench/cli/env.py	30.76%	9 Missing ⚠️
milabench/sizer.py	82.75%	5 Missing ⚠️
milabench/cli/prepare_run.py	55.55%	4 Missing ⚠️
milabench/cli/__init__.py	66.66%	2 Missing ⚠️
milabench/pack.py	0.00%	2 Missing ⚠️
milabench/utils.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #252      +/-   ##
==========================================
- Coverage   72.42%   71.27%   -1.15%     
==========================================
  Files          58       60       +2     
  Lines        5505     5661     +156     
==========================================
+ Hits         3987     4035      +48     
- Misses       1518     1626     +108     

Flag	Coverage Δ
unittests	71.27% <59.13%> (-1.15%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.