Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README.md #195

Merged
merged 1 commit into from
Feb 20, 2024
Merged

Update README.md #195

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 30 additions & 32 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,40 +43,38 @@ It will include all of the necessary data
=================
Benchmark results
=================
perf: items/sec, (sum batchsize) / (elapsed time), higher is better.
peak_memory: MiB
fail n perf sem% std% peak_memory score weight
bert-fp16 0 1 49.82 0.0% 0.2% 23952 49.815508 0.00
bert-fp32 0 1 20.78 0.0% 0.2% 30922 20.783989 0.00
bert-tf32 0 1 20.79 0.0% 0.2% 30922 20.787725 0.00
bert-tf32-fp16 0 1 49.70 0.1% 0.3% 23952 49.697091 3.00
bf16 0 1 7.91 0.0% 0.1% 1140 7.910341 0.00
convnext_large-fp16 0 1 123.77 2.5% 13.6% 26632 123.767014 0.00
convnext_large-fp32 0 1 32.69 0.5% 2.6% 45356 32.687851 0.00
convnext_large-tf32 0 1 32.64 0.5% 2.6% 45356 32.636185 0.00
convnext_large-tf32-fp16 0 1 124.93 2.5% 13.4% 26632 124.930007 3.00
davit_large 0 1 114.54 1.3% 9.8% 32374 114.539282 1.00
davit_large-multi 0 1 115.18 1.2% 9.3% 32374 115.176873 5.00
dlrm 0 1 255977.96 0.5% 4.0% 6354 255977.960840 1.00
focalnet 0 1 151.78 1.6% 12.4% 24098 151.775544 2.00
fp16 0 1 101.03 0.1% 0.6% 1142 101.025637 0.00
fp32 0 1 14.42 0.0% 0.2% 1524 14.418942 0.00
reformer 0 1 10.22 0.0% 0.1% 24756 10.222305 1.00
regnet_y_128gf 0 1 30.52 0.3% 1.9% 30748 30.518845 2.00
resnet152 0 1 232.63 1.1% 8.1% 29904 232.629851 1.00
resnet152-multi 0 1 232.14 1.0% 7.7% 30614 232.144301 5.00
resnet50 0 1 490.08 2.5% 19.0% 4166 490.076388 1.00
rwkv 0 1 109.45 0.3% 2.0% 4944 109.449712 1.00
stargan 0 1 11.40 4.2% 31.9% 35648 11.399463 1.00
super-slomo 0 1 11.46 0.1% 0.5% 36364 11.463760 1.00
t5 0 1 13.91 0.6% 4.5% 34794 13.913109 2.00
tf32 0 1 14.43 0.0% 0.2% 1524 14.430707 0.00
whisper 0 1 81.71 0.1% 0.6% 35968 81.705971 1.00

fail n perf sem% std% peak_memory score weight
bert-fp16 0 8 155.08 0.3% 4.3% 24552 1241.260310 0.00
bert-fp32 0 8 29.52 0.0% 0.5% 31524 236.337218 0.00
bert-tf32 0 8 120.46 0.4% 6.1% 31524 964.713297 0.00
bert-tf32-fp16 0 8 154.76 0.3% 4.1% 24552 1238.477257 3.00
convnext_large-fp16 0 8 337.48 0.9% 14.0% 27658 2741.604444 0.00
convnext_large-fp32 0 8 44.61 0.8% 12.6% 49786 354.207225 0.00
convnext_large-tf32 0 8 135.99 0.7% 11.2% 49786 1089.394916 0.00
convnext_large-tf32-fp16 0 8 338.58 0.8% 13.0% 27658 2744.325170 3.00
davit_large 0 8 312.79 0.3% 6.7% 35058 2515.326450 1.00
davit_large-multi 0 1 2401.65 1.0% 7.7% 42232 2401.651720 5.00
dlrm 0 1 188777.20 1.8% 14.0% 3194 188777.203190 1.00
focalnet 0 8 400.47 0.2% 5.4% 26604 3215.431924 2.00
opt-1_3b 0 1 26.71 0.1% 0.4% 44116 26.714365 5.00
opt-1_3b-multinode 0 2 34.62 0.2% 1.0% 43552 34.618292 10.00
opt-6_7b 0 1 14.32 0.0% 0.1% 55750 14.319587 5.00
opt-6_7b-multinode 0 2 10.79 0.1% 0.7% 49380 10.792595 10.00
reformer 0 8 61.70 0.0% 0.9% 25376 494.110834 1.00
regnet_y_128gf 0 8 99.96 0.2% 5.0% 31840 803.012507 2.00
resnet152 0 8 710.18 0.3% 6.2% 36732 5710.828608 1.00
resnet152-multi 0 1 5367.34 1.0% 8.1% 38638 5367.338469 5.00
resnet50 0 8 984.43 0.9% 19.1% 5026 7927.257351 1.00
rwkv 0 8 428.65 0.2% 3.8% 5546 3435.097716 1.00
stargan 0 8 51.32 1.8% 40.8% 37848 413.238870 1.00
super-slomo 0 8 41.63 0.1% 2.3% 34082 332.395065 1.00
t5 0 8 48.05 0.2% 3.9% 35466 384.317023 2.00
whisper 0 8 248.16 0.0% 0.6% 37006 1985.861017 1.00

Scores
------
Failure rate: 0.00% (OK)
Score: 10.68
Failure rate: 0.00% (PASS)
Score: 219.06


## Details
Expand Down
Loading