Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use latest dataset #204

Merged
merged 1 commit into from
Feb 27, 2024
Merged

Use latest dataset #204

merged 1 commit into from
Feb 27, 2024

Conversation

Delaunay
Copy link
Collaborator

No description provided.

@Delaunay Delaunay changed the base branch from master to stable February 27, 2024 15:23
@Delaunay Delaunay merged commit 77fa0bc into stable Feb 27, 2024
4 checks passed
@Delaunay Delaunay deleted the use_latest_dataset branch February 27, 2024 16:47
@Delaunay
Copy link
Collaborator Author

=================
Benchmark results
=================
                         fail n       perf   sem%   std% peak_memory          score weight
bert-fp16                   0 2     154.87   0.4%   3.5%       24421     309.763287   0.00
bert-fp32                   0 2      29.77   0.0%   0.3%       31385      59.544515   0.00
bert-tf32                   0 2     115.77   0.5%   3.5%       31387     231.177012   0.00
bert-tf32-fp16              0 2     154.09   0.4%   3.2%       24421     308.144333   3.00
convnext_large-fp16         0 2     320.36   2.8%  21.5%       27283     646.182755   0.00
convnext_large-fp32         0 2      45.36   0.7%   5.3%       49403      90.372289   0.00
convnext_large-tf32         0 2     149.07   2.2%  17.0%       49403     298.093913   0.00
convnext_large-tf32-fp16    0 2     322.29   2.9%  22.6%       27283     650.679920   3.00
davit_large                 0 2     303.11   0.7%   7.2%       34081     607.432733   1.00
davit_large-multi           0 1     611.20   0.9%   7.0%       37327     611.196632   5.00
dlrm                        0 1  354429.77   1.8%  13.8%        4795  354429.774921   1.00
focalnet                    0 2     371.65   0.6%   6.6%       25941     743.363201   2.00
opt-1_3b                    0 1       7.44   0.0%   0.2%       41319       7.444252   5.00
opt-6_7b                    1 1        NaN    NaN    NaN       14369            NaN   5.00
reformer                    0 2      61.85   0.1%   1.0%       25225     123.759787   1.00
regnet_y_128gf              0 2      87.74   0.4%   4.9%       31375     176.117266   2.00
resnet152                   0 2     648.46   0.9%   9.6%       34307    1298.643977   1.00
resnet152-multi             0 1    1294.53   1.3%  10.2%       41219    1294.527109   5.00
resnet50                    0 2     542.44   4.4%  48.5%        4551    1082.240601   1.00
stargan                     0 2      45.11   2.5%  27.3%       37247      90.594035   1.00
super-slomo                 0 2      42.79   0.2%   1.8%       33621      85.616477   1.00
t5                          0 2      48.30   0.4%   4.2%       34209      96.590255   2.00
whisper                     0 2     560.35   0.3%   2.7%        9099    1120.819978   1.00

Scores
------
Failure rate:       2.44% (FAIL)
Score:             153.76

Errors
------
1 errors, details in HTML report.
----
Done after 2755

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant