Skip to content

Commit

Permalink
Update the transformer block microbenchmark with SDPA nodes.
Browse files Browse the repository at this point in the history
For #2199

```
$ pytest benchmarks/python/test_transformer.py
```

Before:
```
--------------------------------------------------------------------------------------- benchmark: 2 tests --------------------------------------------------------------------------------------
Name (time in ms)                  Min                 Max                Mean            StdDev              Median               IQR            Outliers      OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_transformer_forward       53.0883 (1.0)       69.7684 (1.0)       65.8204 (1.0)      6.0816 (1.62)      68.9426 (1.0)      4.2709 (2.16)          2;2  15.1929 (1.0)          10           1
test_transformer_backward     174.3857 (3.28)     187.1334 (2.68)     184.6143 (2.80)     3.7561 (1.0)      185.1308 (2.69)     1.9769 (1.0)           1;1   5.4167 (0.36)         10           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
```

After:
```
--------------------------------------------------------------------------------------- benchmark: 2 tests --------------------------------------------------------------------------------------
Name (time in ms)                  Min                 Max                Mean            StdDev              Median               IQR            Outliers      OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_transformer_forward       53.3807 (1.0)       66.7263 (1.0)       63.6231 (1.0)      3.7131 (1.15)      64.7397 (1.0)      1.0460 (1.0)           1;2  15.7176 (1.0)          10           1
test_transformer_backward     160.4337 (3.01)     171.0229 (2.56)     168.4271 (2.65)     3.2160 (1.0)      169.6143 (2.62)     3.7713 (3.61)          1;1   5.9373 (0.38)         10           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
```
  • Loading branch information
wujingyue committed Aug 10, 2024
1 parent 48a17c4 commit 731be12
Showing 1 changed file with 676 additions and 856 deletions.
Loading

0 comments on commit 731be12

Please sign in to comment.