Skip to content
This repository has been archived by the owner on Nov 4, 2024. It is now read-only.

feat: auto-training mode and strict checks #145

Merged
merged 1 commit into from
Aug 29, 2024
Merged

feat: auto-training mode and strict checks #145

merged 1 commit into from
Aug 29, 2024

Conversation

avik-pal
Copy link
Member

@avik-pal avik-pal commented Aug 29, 2024 •
edited
Loading

initial part of #98. We start off with a warning but in 1.0, we will transition to an error

github-actions[bot]
github-actions bot reviewed Aug 29, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LuxLib Benchmarks

Benchmark suite Current: fb000d0 Previous: 56e40d8 Ratio
layernorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/2 thread(s) 5874.5 ns 6083.5 ns 0.97
layernorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/4 thread(s) 5791 ns 5729 ns 1.01
layernorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/8 thread(s) 7334 ns 8208 ns 0.89
layernorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/1 thread(s) 5771 ns 7417 ns 0.78
layernorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/CUDA 115557 ns 119536 ns 0.97
layernorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/oneAPI 2721197 ns 2858698 ns 0.95
layernorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/Metal 751875 ns 774000 ns 0.97
layernorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/AMDGPU 414784 ns 413554 ns 1.00
layernorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/2 thread(s) 10250 ns 9750 ns 1.05
layernorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/4 thread(s) 9708 ns 9541.5 ns 1.02
layernorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/8 thread(s) 9791 ns 9833 ns 1.00
layernorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/1 thread(s) 10042 ns 10000 ns 1.00
layernorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/CUDA 535648 ns 548421 ns 0.98
layernorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/oneAPI 6522514 ns 6391891 ns 1.02
layernorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/Metal 2497541 ns 13032833 ns 0.19
layernorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/AMDGPU 682006 ns 680216 ns 1.00
bias_activation(32, act=relu)(32 x 128)/forward/CPU/2 thread(s) 1542 ns 1333 ns 1.16
bias_activation(32, act=relu)(32 x 128)/forward/CPU/4 thread(s) 1667 ns 3125 ns 0.53
bias_activation(32, act=relu)(32 x 128)/forward/CPU/8 thread(s) 2000 ns 2750 ns 0.73
bias_activation(32, act=relu)(32 x 128)/forward/CPU/1 thread(s) 1458 ns 1646 ns 0.89
bias_activation(32, act=relu)(32 x 128)/forward/GPU/CUDA 21274 ns 21670 ns 0.98
bias_activation(32, act=relu)(32 x 128)/forward/GPU/oneAPI 1314672 ns 1360152 ns 0.97
bias_activation(32, act=relu)(32 x 128)/forward/GPU/Metal 202792 ns 200500 ns 1.01
bias_activation(32, act=relu)(32 x 128)/forward/GPU/AMDGPU 31600 ns 30925.5 ns 1.02
bias_activation(32, act=relu)(32 x 128)/zygote/CPU/2 thread(s) 3666 ns 4000.5 ns 0.92
bias_activation(32, act=relu)(32 x 128)/zygote/CPU/4 thread(s) 4000 ns 3542 ns 1.13
bias_activation(32, act=relu)(32 x 128)/zygote/CPU/8 thread(s) 4083.5 ns 4250 ns 0.96
bias_activation(32, act=relu)(32 x 128)/zygote/CPU/1 thread(s) 3667 ns 3979 ns 0.92
bias_activation(32, act=relu)(32 x 128)/zygote/GPU/CUDA 142562 ns 146351 ns 0.97
bias_activation(32, act=relu)(32 x 128)/zygote/GPU/oneAPI 8818919 ns 9417207 ns 0.94
bias_activation(32, act=relu)(32 x 128)/zygote/GPU/Metal 1444500 ns 1465833.5 ns 0.99
bias_activation(32, act=relu)(32 x 128)/zygote/GPU/AMDGPU 150661.5 ns 148801 ns 1.01
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 57417 ns 57959 ns 0.99
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 47062.5 ns 46875 ns 1.00
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 46958 ns 46666 ns 1.01
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 82666 ns 82958 ns 1.00
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/CUDA 36368 ns 37604.5 ns 0.97
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 573767 ns 581086 ns 0.99
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/Metal 1032834 ns 1034333.5 ns 1.00
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 79390 ns 79736 ns 1.00
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 2308000 ns 2032625 ns 1.14
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 2219209 ns 2089666 ns 1.06
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 2265792 ns 2087125 ns 1.09
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 2271083 ns 1994500 ns 1.14
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 458763 ns 234123 ns 1.96
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 7651123 ns 7389099 ns 1.04
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/Metal 7033375 ns 5422500 ns 1.30
batchnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1308453 ns 1219571 ns 1.07
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 147833 ns 164187.5 ns 0.90
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 173437.5 ns 153833 ns 1.13
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 151270.5 ns 150458 ns 1.01
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 146208.5 ns 167666.5 ns 0.87
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/CUDA 165164 ns 166286 ns 0.99
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 7366575 ns 7718471.5 ns 0.95
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/Metal 1388500 ns 1555229.5 ns 0.89
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 191773 ns 190051 ns 1.01
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 1118292 ns 1111646 ns 1.01
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 1109000 ns 1113250 ns 1.00
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 1120333 ns 1116271 ns 1.00
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 1118750 ns 1115771 ns 1.00
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 681370.5 ns 700098 ns 0.97
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 36156481 ns 33735820.5 ns 1.07
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/Metal 6091666 ns 6479708 ns 0.94
layernorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 922886 ns 1025985 ns 0.90
layernorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/2 thread(s) 5666.5 ns 5708.5 ns 0.99
layernorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/4 thread(s) 4229 ns 4292 ns 0.99
layernorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/8 thread(s) 6416.5 ns 5708 ns 1.12
layernorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/1 thread(s) 5312.5 ns 6395.5 ns 0.83
layernorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/CUDA 89522 ns 93278.5 ns 0.96
layernorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/oneAPI 5362277 ns 5349506.5 ns 1.00
layernorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/Metal 454125 ns 453417 ns 1.00
layernorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/AMDGPU 72051 ns 59270 ns 1.22
layernorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/2 thread(s) 9208 ns 8625 ns 1.07
layernorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/4 thread(s) 8584 ns 8667 ns 0.99
layernorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/8 thread(s) 9000 ns 9000 ns 1
layernorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/1 thread(s) 9083 ns 8792 ns 1.03
layernorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/CUDA 589330 ns 614663 ns 0.96
layernorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/oneAPI 37693204.5 ns 34665842.5 ns 1.09
layernorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/Metal 5442709 ns 5535062.5 ns 0.98
layernorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/AMDGPU 391815 ns 384114 ns 1.02
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 17937.5 ns 19000 ns 0.94
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 18250 ns 18708.5 ns 0.98
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 21500 ns 20209 ns 1.06
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 18291.5 ns 18542 ns 0.99
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/CUDA 65529 ns 66269 ns 0.99
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 2847915 ns 2900988 ns 0.98
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/Metal 1318875 ns 1296084 ns 1.02
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 73415.5 ns 73291 ns 1.00
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 213209 ns 222000 ns 0.96
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 224292 ns 211750 ns 1.06
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 216083.5 ns 223125 ns 0.97
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 223875 ns 221021 ns 1.01
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 344520 ns 354565.5 ns 0.97
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 12592357.5 ns 13542856.5 ns 0.93
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/Metal 5804917 ns 6042354 ns 0.96
groupnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 477606 ns 480655 ns 0.99
bias_activation(2, act=relu)(2 x 128)/forward/CPU/2 thread(s) 625 ns 667 ns 0.94
bias_activation(2, act=relu)(2 x 128)/forward/CPU/4 thread(s) 625 ns 625 ns 1
bias_activation(2, act=relu)(2 x 128)/forward/CPU/8 thread(s) 1042 ns 916 ns 1.14
bias_activation(2, act=relu)(2 x 128)/forward/CPU/1 thread(s) 625 ns 791 ns 0.79
bias_activation(2, act=relu)(2 x 128)/forward/GPU/CUDA 20212 ns 20668 ns 0.98
bias_activation(2, act=relu)(2 x 128)/forward/GPU/oneAPI 1171756.5 ns 1159361.5 ns 1.01
bias_activation(2, act=relu)(2 x 128)/forward/GPU/Metal 286125 ns 278000 ns 1.03
bias_activation(2, act=relu)(2 x 128)/forward/GPU/AMDGPU 34560 ns 34450 ns 1.00
bias_activation(2, act=relu)(2 x 128)/zygote/CPU/2 thread(s) 1417 ns 1417 ns 1
bias_activation(2, act=relu)(2 x 128)/zygote/CPU/4 thread(s) 1458 ns 1520.5 ns 0.96
bias_activation(2, act=relu)(2 x 128)/zygote/CPU/8 thread(s) 1667 ns 1583 ns 1.05
bias_activation(2, act=relu)(2 x 128)/zygote/CPU/1 thread(s) 1375 ns 1375 ns 1
bias_activation(2, act=relu)(2 x 128)/zygote/GPU/CUDA 122479 ns 125954.5 ns 0.97
bias_activation(2, act=relu)(2 x 128)/zygote/GPU/oneAPI 8995739 ns 8769872 ns 1.03
bias_activation(2, act=relu)(2 x 128)/zygote/GPU/Metal 1460625 ns 1445458 ns 1.01
bias_activation(2, act=relu)(2 x 128)/zygote/GPU/AMDGPU 130722 ns 128201.5 ns 1.02
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 7333 ns 7417 ns 0.99
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 6125 ns 6125 ns 1
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 6166 ns 6167 ns 1.00
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 10333 ns 10292 ns 1.00
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/CUDA 23423.5 ns 24236 ns 0.97
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 1238741 ns 1337008 ns 0.93
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/Metal 621896.5 ns 513333 ns 1.21
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 49041 ns 48301 ns 1.02
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 371104 ns 220959 ns 1.68
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 413916 ns 269208.5 ns 1.54
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 422208 ns 263750 ns 1.60
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 363833.5 ns 225583 ns 1.61
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 407257 ns 192759 ns 2.11
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 31598972 ns 29975947 ns 1.05
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/Metal 9262166.5 ns 8958750 ns 1.03
batchnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 730489 ns 608296 ns 1.20
dense(32, bias=false, act=relu)(32 x 128)/forward/CPU/2 thread(s) 4084 ns 4083 ns 1.00
dense(32, bias=false, act=relu)(32 x 128)/forward/CPU/4 thread(s) 4084 ns 4083 ns 1.00
dense(32, bias=false, act=relu)(32 x 128)/forward/CPU/8 thread(s) 4125 ns 4083 ns 1.01
dense(32, bias=false, act=relu)(32 x 128)/forward/CPU/1 thread(s) 4083 ns 4042 ns 1.01
dense(32, bias=false, act=relu)(32 x 128)/forward/GPU/CUDA 23414 ns 23710 ns 0.99
dense(32, bias=false, act=relu)(32 x 128)/forward/GPU/oneAPI 2073024 ns 1967682 ns 1.05
dense(32, bias=false, act=relu)(32 x 128)/forward/GPU/Metal 222041.5 ns 220708 ns 1.01
dense(32, bias=false, act=relu)(32 x 128)/forward/GPU/AMDGPU 52231 ns 48820 ns 1.07
dense(32, bias=false, act=relu)(32 x 128)/zygote/CPU/2 thread(s) 16834 ns 16625 ns 1.01
dense(32, bias=false, act=relu)(32 x 128)/zygote/CPU/4 thread(s) 17333 ns 16667 ns 1.04
dense(32, bias=false, act=relu)(32 x 128)/zygote/CPU/8 thread(s) 17125 ns 17125 ns 1
dense(32, bias=false, act=relu)(32 x 128)/zygote/CPU/1 thread(s) 16917 ns 16750 ns 1.01
dense(32, bias=false, act=relu)(32 x 128)/zygote/GPU/CUDA 188995.5 ns 195325 ns 0.97
dense(32, bias=false, act=relu)(32 x 128)/zygote/GPU/oneAPI 11366884 ns 9992661.5 ns 1.14
dense(32, bias=false, act=relu)(32 x 128)/zygote/GPU/Metal 932458.5 ns 956125 ns 0.98
dense(32, bias=false, act=relu)(32 x 128)/zygote/GPU/AMDGPU 178432 ns 177522 ns 1.01
dense(512, bias=false, act=gelu)(512 x 128)/forward/CPU/2 thread(s) 510041 ns 509750 ns 1.00
dense(512, bias=false, act=gelu)(512 x 128)/forward/CPU/4 thread(s) 405334 ns 404666 ns 1.00
dense(512, bias=false, act=gelu)(512 x 128)/forward/CPU/8 thread(s) 404000 ns 405500 ns 1.00
dense(512, bias=false, act=gelu)(512 x 128)/forward/CPU/1 thread(s) 865041 ns 864500 ns 1.00
dense(512, bias=false, act=gelu)(512 x 128)/forward/GPU/CUDA 113309 ns 113934 ns 0.99
dense(512, bias=false, act=gelu)(512 x 128)/forward/GPU/oneAPI 393832 ns 399968.5 ns 0.98
dense(512, bias=false, act=gelu)(512 x 128)/forward/GPU/Metal 385354 ns 452750 ns 0.85
dense(512, bias=false, act=gelu)(512 x 128)/forward/GPU/AMDGPU 249783 ns 248142 ns 1.01
dense(512, bias=false, act=gelu)(512 x 128)/zygote/CPU/2 thread(s) 2327625 ns 2323875 ns 1.00
dense(512, bias=false, act=gelu)(512 x 128)/zygote/CPU/4 thread(s) 2029083 ns 2027687 ns 1.00
dense(512, bias=false, act=gelu)(512 x 128)/zygote/CPU/8 thread(s) 2034187 ns 2035333 ns 1.00
dense(512, bias=false, act=gelu)(512 x 128)/zygote/CPU/1 thread(s) 3282959 ns 3278166 ns 1.00
dense(512, bias=false, act=gelu)(512 x 128)/zygote/GPU/CUDA 236546 ns 240558 ns 0.98
dense(512, bias=false, act=gelu)(512 x 128)/zygote/GPU/oneAPI 11904169 ns 9061829 ns 1.31
dense(512, bias=false, act=gelu)(512 x 128)/zygote/GPU/Metal 1917833 ns 1864375 ns 1.03
dense(512, bias=false, act=gelu)(512 x 128)/zygote/GPU/AMDGPU 762089 ns 762112 ns 1.00
layernorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/2 thread(s) 6500 ns 6791.5 ns 0.96
layernorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/4 thread(s) 6500 ns 6354.5 ns 1.02
layernorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/8 thread(s) 8229.5 ns 8021 ns 1.03
layernorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/1 thread(s) 6333 ns 7520.5 ns 0.84
layernorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/CUDA 88687.5 ns 92014.5 ns 0.96
layernorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/oneAPI 5523378 ns 5475275 ns 1.01
layernorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/Metal 719916 ns 726667 ns 0.99
layernorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/AMDGPU 62181 ns 60220 ns 1.03
layernorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/2 thread(s) 12375 ns 11083.5 ns 1.12
layernorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/4 thread(s) 11417 ns 11729 ns 0.97
layernorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/8 thread(s) 12500 ns 12750 ns 0.98
layernorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/1 thread(s) 12458.5 ns 11291.5 ns 1.10
layernorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/CUDA 611796 ns 656827.5 ns 0.93
layernorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/oneAPI 37468397.5 ns 40222366 ns 0.93
layernorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/Metal 5362250 ns 5480792 ns 0.98
layernorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/AMDGPU 418230 ns 413864 ns 1.01
dense(2, bias=true, act=relu)(2 x 128)/forward/CPU/2 thread(s) 500 ns 500 ns 1
dense(2, bias=true, act=relu)(2 x 128)/forward/CPU/4 thread(s) 500 ns 500 ns 1
dense(2, bias=true, act=relu)(2 x 128)/forward/CPU/8 thread(s) 500 ns 542 ns 0.92
dense(2, bias=true, act=relu)(2 x 128)/forward/CPU/1 thread(s) 500 ns 500 ns 1
dense(2, bias=true, act=relu)(2 x 128)/forward/GPU/CUDA 22927 ns 23122 ns 0.99
dense(2, bias=true, act=relu)(2 x 128)/forward/GPU/oneAPI 2183765 ns 2175700 ns 1.00
dense(2, bias=true, act=relu)(2 x 128)/forward/GPU/Metal 222437.5 ns 322625 ns 0.69
dense(2, bias=true, act=relu)(2 x 128)/forward/GPU/AMDGPU 53110 ns 53980 ns 0.98
dense(2, bias=true, act=relu)(2 x 128)/zygote/CPU/2 thread(s) 2125 ns 2083 ns 1.02
dense(2, bias=true, act=relu)(2 x 128)/zygote/CPU/4 thread(s) 2167 ns 2083 ns 1.04
dense(2, bias=true, act=relu)(2 x 128)/zygote/CPU/8 thread(s) 2167 ns 2208 ns 0.98
dense(2, bias=true, act=relu)(2 x 128)/zygote/CPU/1 thread(s) 2125 ns 2125 ns 1
dense(2, bias=true, act=relu)(2 x 128)/zygote/GPU/CUDA 210803 ns 223205.5 ns 0.94
dense(2, bias=true, act=relu)(2 x 128)/zygote/GPU/oneAPI 11049931 ns 11597723.5 ns 0.95
dense(2, bias=true, act=relu)(2 x 128)/zygote/GPU/Metal 1947417 ns 1948500 ns 1.00
dense(2, bias=true, act=relu)(2 x 128)/zygote/GPU/AMDGPU 176312 ns 183602 ns 0.96
groupnorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/2 thread(s) 9042 ns 8812 ns 1.03
groupnorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/4 thread(s) 8083.5 ns 8583.5 ns 0.94
groupnorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/8 thread(s) 11895.5 ns 10667 ns 1.12
groupnorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/1 thread(s) 8875 ns 8604.5 ns 1.03
groupnorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/CUDA 91893.5 ns 99076.5 ns 0.93
groupnorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/oneAPI 3103648 ns 2921578.5 ns 1.06
groupnorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/Metal 777541.5 ns 808625 ns 0.96
groupnorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/AMDGPU 78651 ns 78481 ns 1.00
groupnorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/2 thread(s) 16958 ns 17750.5 ns 0.96
groupnorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/4 thread(s) 18145.5 ns 18334 ns 0.99
groupnorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/8 thread(s) 19000 ns 18792 ns 1.01
groupnorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/1 thread(s) 17750 ns 18104.5 ns 0.98
groupnorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/CUDA 534997 ns 609459 ns 0.88
groupnorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/oneAPI 15733339.5 ns 16546891.5 ns 0.95
groupnorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/Metal 4951333.5 ns 5201875 ns 0.95
groupnorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/AMDGPU 394905 ns 392264 ns 1.01
batchnorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/2 thread(s) 541 ns 542 ns 1.00
batchnorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/4 thread(s) 583 ns 541 ns 1.08
batchnorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/8 thread(s) 583 ns 625 ns 0.93
batchnorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/1 thread(s) 500 ns 541 ns 0.92
batchnorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/CUDA 34529 ns 35893 ns 0.96
batchnorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/oneAPI 1235951 ns 1234257.5 ns 1.00
batchnorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/Metal 287292 ns 308542 ns 0.93
batchnorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/AMDGPU 48301 ns 45990.5 ns 1.05
batchnorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/2 thread(s) 147000 ns 10208 ns 14.40
batchnorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/4 thread(s) 148417 ns 9042 ns 16.41
batchnorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/8 thread(s) 161208.5 ns 11208 ns 14.38
batchnorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/1 thread(s) 148333 ns 9666 ns 15.35
batchnorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/CUDA 451818 ns 268063.5 ns 1.69
batchnorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/oneAPI 18323518 ns 18168061 ns 1.01
batchnorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/Metal 4753042 ns 4946250 ns 0.96
batchnorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/AMDGPU 537806 ns 374863 ns 1.43
dense(512, bias=false, act=identity)(512 x 128)/forward/CPU/2 thread(s) 396958 ns 397125 ns 1.00
dense(512, bias=false, act=identity)(512 x 128)/forward/CPU/4 thread(s) 288125 ns 287584 ns 1.00
dense(512, bias=false, act=identity)(512 x 128)/forward/CPU/8 thread(s) 287979.5 ns 288125 ns 1.00
dense(512, bias=false, act=identity)(512 x 128)/forward/CPU/1 thread(s) 755542 ns 756000 ns 1.00
dense(512, bias=false, act=identity)(512 x 128)/forward/GPU/CUDA 111265.5 ns 112334 ns 0.99
dense(512, bias=false, act=identity)(512 x 128)/forward/GPU/oneAPI 328662 ns 331349 ns 0.99
dense(512, bias=false, act=identity)(512 x 128)/forward/GPU/Metal 367000 ns 453166 ns 0.81
dense(512, bias=false, act=identity)(512 x 128)/forward/GPU/AMDGPU 78721 ns 78321 ns 1.01
dense(512, bias=false, act=identity)(512 x 128)/zygote/CPU/2 thread(s) 1454062.5 ns 1442312.5 ns 1.01
dense(512, bias=false, act=identity)(512 x 128)/zygote/CPU/4 thread(s) 1136416 ns 1128583 ns 1.01
dense(512, bias=false, act=identity)(512 x 128)/zygote/CPU/8 thread(s) 1132354.5 ns 1136375 ns 1.00
dense(512, bias=false, act=identity)(512 x 128)/zygote/CPU/1 thread(s) 2439916 ns 2441021 ns 1.00
dense(512, bias=false, act=identity)(512 x 128)/zygote/GPU/CUDA 204364.5 ns 207111 ns 0.99
dense(512, bias=false, act=identity)(512 x 128)/zygote/GPU/oneAPI 9824415 ns 10702686 ns 0.92
dense(512, bias=false, act=identity)(512 x 128)/zygote/GPU/Metal 1570541.5 ns 1560271 ns 1.01
dense(512, bias=false, act=identity)(512 x 128)/zygote/GPU/AMDGPU 325414 ns 324973 ns 1.00
layernorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/2 thread(s) 7395.5 ns 7208.5 ns 1.03
layernorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/4 thread(s) 7666.5 ns 7166.5 ns 1.07
layernorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/8 thread(s) 9792 ns 8250 ns 1.19
layernorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/1 thread(s) 7145.5 ns 7750 ns 0.92
layernorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/CUDA 129489 ns 148537.5 ns 0.87
layernorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/oneAPI 5955624 ns 5887503 ns 1.01
layernorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/Metal 439000 ns 464209 ns 0.95
layernorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/AMDGPU 60971 ns 59820 ns 1.02
layernorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/2 thread(s) 17125 ns 16500 ns 1.04
layernorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/4 thread(s) 15771 ns 15041.5 ns 1.05
layernorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/8 thread(s) 16520.5 ns 15708 ns 1.05
layernorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/1 thread(s) 15375 ns 15041 ns 1.02
layernorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/CUDA 855248.5 ns 975911 ns 0.88
layernorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/oneAPI 44381652.5 ns 46239281 ns 0.96
layernorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/Metal 5527750 ns 5635271 ns 0.98
layernorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/AMDGPU 435840 ns 439474 ns 0.99
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 30500 ns 25083 ns 1.22
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 27834 ns 26354.5 ns 1.06
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 31292 ns 29833 ns 1.05
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 25771 ns 25291 ns 1.02
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/CUDA 191688 ns 200872.5 ns 0.95
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 7437045.5 ns 7942712.5 ns 0.94
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/Metal 586292 ns 976813 ns 0.60
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 118032 ns 117671 ns 1.00
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 154208 ns 103917 ns 1.48
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 150125 ns 154250 ns 0.97
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 153709 ns 143979 ns 1.07
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 104021 ns 112208 ns 0.93
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 1029587 ns 1080618 ns 0.95
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 43318293.5 ns 45956117 ns 0.94
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/Metal 5712583.5 ns 5734166.5 ns 1.00
layernorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 600207 ns 598555 ns 1.00
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 74500 ns 77125 ns 0.97
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 85875 ns 74229 ns 1.16
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 84000 ns 79687.5 ns 1.05
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 75208 ns 75770.5 ns 0.99
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/CUDA 203652.5 ns 207270 ns 0.98
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 7714505.5 ns 7792417.5 ns 0.99
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/Metal 526209 ns 522646 ns 1.01
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 125851 ns 123966 ns 1.02
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 221270.5 ns 287541.5 ns 0.77
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 295000 ns 301792 ns 0.98
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 302375 ns 295041 ns 1.02
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 221417 ns 218208 ns 1.01
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 1101887 ns 1107042 ns 1.00
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 42953193 ns 43388234 ns 0.99
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/Metal 6252458.5 ns 6243958 ns 1.00
layernorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 702428 ns 701281.5 ns 1.00
layernorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/2 thread(s) 16375 ns 17834 ns 0.92
layernorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/4 thread(s) 18000 ns 16625 ns 1.08
layernorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/8 thread(s) 18374.5 ns 18104.5 ns 1.01
layernorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/1 thread(s) 17083 ns 16729.5 ns 1.02
layernorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/CUDA 148672.5 ns 150748.5 ns 0.99
layernorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/oneAPI 5794282 ns 5427549 ns 1.07
layernorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/Metal 443708 ns 452333 ns 0.98
layernorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/AMDGPU 240983 ns 237672 ns 1.01
layernorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/2 thread(s) 27270.5 ns 27500.5 ns 0.99
layernorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/4 thread(s) 26687.5 ns 27583 ns 0.97
layernorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/8 thread(s) 27333 ns 28875 ns 0.95
layernorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/1 thread(s) 27167 ns 25146 ns 1.08
layernorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/CUDA 956079 ns 981795 ns 0.97
layernorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/oneAPI 42186646 ns 43049751.5 ns 0.98
layernorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/Metal 5738708 ns 5608208 ns 1.02
layernorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/AMDGPU 716559 ns 715207 ns 1.00
groupnorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/2 thread(s) 11291 ns 12313 ns 0.92
groupnorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/4 thread(s) 10812.5 ns 10020.5 ns 1.08
groupnorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/8 thread(s) 13646 ns 12417 ns 1.10
groupnorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/1 thread(s) 11833 ns 11208.5 ns 1.06
groupnorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/CUDA 121242 ns 122999 ns 0.99
groupnorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/oneAPI 3475987.5 ns 3856591 ns 0.90
groupnorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/Metal 857542 ns 783354.5 ns 1.09
groupnorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/AMDGPU 243332.5 ns 244302 ns 1.00
groupnorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/2 thread(s) 22416 ns 22250 ns 1.01
groupnorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/4 thread(s) 22458 ns 21396 ns 1.05
groupnorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/8 thread(s) 21833 ns 23000 ns 0.95
groupnorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/1 thread(s) 22333 ns 21687.5 ns 1.03
groupnorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/CUDA 685433 ns 704827 ns 0.97
groupnorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/oneAPI 21324187 ns 19822430 ns 1.08
groupnorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/Metal 5433000 ns 5200895.5 ns 1.04
groupnorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/AMDGPU 684518 ns 687056 ns 1.00
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 64291 ns 63625.5 ns 1.01
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 62291 ns 62583 ns 1.00
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 68000 ns 66646 ns 1.02
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 63458 ns 66937.5 ns 0.95
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/CUDA 106200.5 ns 105671 ns 1.01
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 3278706 ns 3419840 ns 0.96
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/Metal 1343125 ns 1336188 ns 1.01
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 240213 ns 238667 ns 1.01
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 449625 ns 475666 ns 0.95
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 450041.5 ns 448750 ns 1.00
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 452875 ns 446208 ns 1.01
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 473708 ns 478625 ns 0.99
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 506804 ns 516873.5 ns 0.98
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 20711437 ns 20193366 ns 1.03
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/Metal 6340770.5 ns 6184938 ns 1.03
groupnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 734073.5 ns 717327 ns 1.02
layernorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/2 thread(s) 6750 ns 7271 ns 0.93
layernorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/4 thread(s) 8333.5 ns 7083 ns 1.18
layernorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/8 thread(s) 9583 ns 8250 ns 1.16
layernorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/1 thread(s) 8084 ns 7521 ns 1.07
layernorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/CUDA 144433.5 ns 146807 ns 0.98
layernorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/oneAPI 5559405 ns 5447467.5 ns 1.02
layernorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/Metal 442333 ns 462959 ns 0.96
layernorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/AMDGPU 59460.5 ns 61520 ns 0.97
layernorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/2 thread(s) 15271 ns 15854.5 ns 0.96
layernorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/4 thread(s) 15042 ns 13895.5 ns 1.08
layernorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/8 thread(s) 16687.5 ns 15458 ns 1.08
layernorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/1 thread(s) 17292 ns 14041 ns 1.23
layernorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/CUDA 925112 ns 952735 ns 0.97
layernorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/oneAPI 38208450 ns 39171319.5 ns 0.98
layernorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/Metal 5445417 ns 5387667 ns 1.01
layernorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/AMDGPU 403345 ns 406764 ns 0.99
batchedmm(512, Bsize=4)/forward/CPU/2 thread(s) 6148375 ns 6150187.5 ns 1.00
batchedmm(512, Bsize=4)/forward/CPU/4 thread(s) 6378583 ns 6375084 ns 1.00
batchedmm(512, Bsize=4)/forward/CPU/8 thread(s) 6377979.5 ns 6377896 ns 1.00
batchedmm(512, Bsize=4)/forward/CPU/1 thread(s) 11916021 ns 11916958 ns 1.00
batchedmm(512, Bsize=4)/forward/GPU/CUDA 347485 ns 345906.5 ns 1.00
batchedmm(512, Bsize=4)/forward/GPU/AMDGPU 294433.5 ns 293393 ns 1.00
batchedmm(512, Bsize=4)/zygote/CPU/2 thread(s) 19132250 ns 19109896 ns 1.00
batchedmm(512, Bsize=4)/zygote/CPU/4 thread(s) 19973479 ns 19969688 ns 1.00
batchedmm(512, Bsize=4)/zygote/CPU/8 thread(s) 20014417 ns 19911667 ns 1.01
batchedmm(512, Bsize=4)/zygote/CPU/1 thread(s) 36557250 ns 36665438 ns 1.00
batchedmm(512, Bsize=4)/zygote/GPU/CUDA 1025341.5 ns 1011944.5 ns 1.01
batchedmm(512, Bsize=4)/zygote/GPU/AMDGPU 1151734 ns 1168811 ns 0.99
dense(2, bias=true, act=gelu)(2 x 128)/forward/CPU/2 thread(s) 958 ns 958 ns 1
dense(2, bias=true, act=gelu)(2 x 128)/forward/CPU/4 thread(s) 1000 ns 917 ns 1.09
dense(2, bias=true, act=gelu)(2 x 128)/forward/CPU/8 thread(s) 959 ns 959 ns 1
dense(2, bias=true, act=gelu)(2 x 128)/forward/CPU/1 thread(s) 959 ns 917 ns 1.05
dense(2, bias=true, act=gelu)(2 x 128)/forward/GPU/CUDA 23037 ns 23377 ns 0.99
dense(2, bias=true, act=gelu)(2 x 128)/forward/GPU/oneAPI 2116275 ns 2088445 ns 1.01
dense(2, bias=true, act=gelu)(2 x 128)/forward/GPU/Metal 288334 ns 218062.5 ns 1.32
dense(2, bias=true, act=gelu)(2 x 128)/forward/GPU/AMDGPU 215612 ns 214272.5 ns 1.01
dense(2, bias=true, act=gelu)(2 x 128)/zygote/CPU/2 thread(s) 3625 ns 3667 ns 0.99
dense(2, bias=true, act=gelu)(2 x 128)/zygote/CPU/4 thread(s) 3750 ns 3667 ns 1.02
dense(2, bias=true, act=gelu)(2 x 128)/zygote/CPU/8 thread(s) 3875 ns 3791 ns 1.02
dense(2, bias=true, act=gelu)(2 x 128)/zygote/CPU/1 thread(s) 3667 ns 3667 ns 1
dense(2, bias=true, act=gelu)(2 x 128)/zygote/GPU/CUDA 275557 ns 284240 ns 0.97
dense(2, bias=true, act=gelu)(2 x 128)/zygote/GPU/oneAPI 11382415 ns 10831129 ns 1.05
dense(2, bias=true, act=gelu)(2 x 128)/zygote/GPU/Metal 2104917 ns 2013458 ns 1.05
dense(2, bias=true, act=gelu)(2 x 128)/zygote/GPU/AMDGPU 651302.5 ns 642396 ns 1.01
groupnorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/2 thread(s) 8167 ns 7834 ns 1.04
groupnorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/4 thread(s) 7792 ns 8250 ns 0.94
groupnorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/8 thread(s) 9542 ns 9208.5 ns 1.04
groupnorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/1 thread(s) 8563 ns 9125 ns 0.94
groupnorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/CUDA 119002 ns 120248 ns 0.99
groupnorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/oneAPI 3379292 ns 3419283 ns 0.99
groupnorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/Metal 784000 ns 777875.5 ns 1.01
groupnorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/AMDGPU 72261 ns 68341 ns 1.06
groupnorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/2 thread(s) 11041 ns 11875 ns 0.93
groupnorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/4 thread(s) 11209 ns 12167 ns 0.92
groupnorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/8 thread(s) 12958 ns 12625 ns 1.03
groupnorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/1 thread(s) 10812.5 ns 12709 ns 0.85
groupnorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/CUDA 626600.5 ns 645406 ns 0.97
groupnorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/oneAPI 22838448 ns 20750949 ns 1.10
groupnorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/Metal 4545250 ns 4833541 ns 0.94
groupnorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/AMDGPU 371774 ns 362853 ns 1.02
dense(2, bias=false, act=relu)(2 x 128)/forward/CPU/2 thread(s) 250 ns 291 ns 0.86
dense(2, bias=false, act=relu)(2 x 128)/forward/CPU/4 thread(s) 292 ns 291 ns 1.00
dense(2, bias=false, act=relu)(2 x 128)/forward/CPU/8 thread(s) 292 ns 292 ns 1
dense(2, bias=false, act=relu)(2 x 128)/forward/CPU/1 thread(s) 250 ns 250 ns 1
dense(2, bias=false, act=relu)(2 x 128)/forward/GPU/CUDA 22136 ns 22453 ns 0.99
dense(2, bias=false, act=relu)(2 x 128)/forward/GPU/oneAPI 2088688 ns 2036768.5 ns 1.03
dense(2, bias=false, act=relu)(2 x 128)/forward/GPU/Metal 221666.5 ns 218625 ns 1.01
dense(2, bias=false, act=relu)(2 x 128)/forward/GPU/AMDGPU 53240 ns 51501 ns 1.03
dense(2, bias=false, act=relu)(2 x 128)/zygote/CPU/2 thread(s) 2833 ns 2833 ns 1
dense(2, bias=false, act=relu)(2 x 128)/zygote/CPU/4 thread(s) 2792 ns 2833 ns 0.99
dense(2, bias=false, act=relu)(2 x 128)/zygote/CPU/8 thread(s) 3166 ns 3209 ns 0.99
dense(2, bias=false, act=relu)(2 x 128)/zygote/CPU/1 thread(s) 2792 ns 2834 ns 0.99
dense(2, bias=false, act=relu)(2 x 128)/zygote/GPU/CUDA 198602.5 ns 203537.5 ns 0.98
dense(2, bias=false, act=relu)(2 x 128)/zygote/GPU/oneAPI 9486294 ns 9796958 ns 0.97
dense(2, bias=false, act=relu)(2 x 128)/zygote/GPU/Metal 1514917 ns 1523062.5 ns 0.99
dense(2, bias=false, act=relu)(2 x 128)/zygote/GPU/AMDGPU 161262 ns 161481 ns 1.00
groupnorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/2 thread(s) 12084 ns 11875 ns 1.02
groupnorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/4 thread(s) 11792 ns 10875 ns 1.08
groupnorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/8 thread(s) 14292 ns 13208 ns 1.08
groupnorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/1 thread(s) 12000 ns 11875 ns 1.01
groupnorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/CUDA 119093 ns 120852 ns 0.99
groupnorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/oneAPI 3487542 ns 3578611 ns 0.97
groupnorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/Metal 826000 ns 824000 ns 1.00
groupnorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/AMDGPU 243253 ns 240102 ns 1.01
groupnorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/2 thread(s) 20416.5 ns 21792 ns 0.94
groupnorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/4 thread(s) 20479.5 ns 21834 ns 0.94
groupnorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/8 thread(s) 20729 ns 22271 ns 0.93
groupnorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/1 thread(s) 19479.5 ns 20584 ns 0.95
groupnorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/CUDA 579311 ns 600222 ns 0.97
groupnorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/oneAPI 20554218 ns 20377991.5 ns 1.01
groupnorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/Metal 4772875 ns 4668250 ns 1.02
groupnorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/AMDGPU 652368 ns 663226 ns 0.98
dense(32, bias=true, act=relu)(32 x 128)/forward/CPU/2 thread(s) 4375 ns 4375 ns 1
dense(32, bias=true, act=relu)(32 x 128)/forward/CPU/4 thread(s) 4375 ns 4375 ns 1
dense(32, bias=true, act=relu)(32 x 128)/forward/CPU/8 thread(s) 4417 ns 4416 ns 1.00
dense(32, bias=true, act=relu)(32 x 128)/forward/CPU/1 thread(s) 4375 ns 4375 ns 1
dense(32, bias=true, act=relu)(32 x 128)/forward/GPU/CUDA 24066.5 ns 24569 ns 0.98
dense(32, bias=true, act=relu)(32 x 128)/forward/GPU/oneAPI 2256365 ns 2264680 ns 1.00
dense(32, bias=true, act=relu)(32 x 128)/forward/GPU/Metal 223000 ns 222854.5 ns 1.00
dense(32, bias=true, act=relu)(32 x 128)/forward/GPU/AMDGPU 52711 ns 52551 ns 1.00
dense(32, bias=true, act=relu)(32 x 128)/zygote/CPU/2 thread(s) 16667 ns 16542 ns 1.01
dense(32, bias=true, act=relu)(32 x 128)/zygote/CPU/4 thread(s) 16417 ns 16458 ns 1.00
dense(32, bias=true, act=relu)(32 x 128)/zygote/CPU/8 thread(s) 16500 ns 16750 ns 0.99
dense(32, bias=true, act=relu)(32 x 128)/zygote/CPU/1 thread(s) 16625 ns 16584 ns 1.00
dense(32, bias=true, act=relu)(32 x 128)/zygote/GPU/CUDA 325746.5 ns 329740.5 ns 0.99
dense(32, bias=true, act=relu)(32 x 128)/zygote/GPU/oneAPI 12327891 ns 12210269.5 ns 1.01
dense(32, bias=true, act=relu)(32 x 128)/zygote/GPU/Metal 1081354.5 ns 1074708 ns 1.01
dense(32, bias=true, act=relu)(32 x 128)/zygote/GPU/AMDGPU 211922.5 ns 212612 ns 1.00
batchnorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/2 thread(s) 1958 ns 2083 ns 0.94
batchnorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/4 thread(s) 2083 ns 2084 ns 1.00
batchnorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/8 thread(s) 2041 ns 2167 ns 0.94
batchnorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/1 thread(s) 1958 ns 1958 ns 1
batchnorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/CUDA 35040 ns 36693 ns 0.95
batchnorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/oneAPI 1241357 ns 1172885.5 ns 1.06
batchnorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/Metal 286125 ns 289959 ns 0.99
batchnorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/AMDGPU 207902 ns 206982 ns 1.00
batchnorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/2 thread(s) 148896 ns 17541.5 ns 8.49
batchnorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/4 thread(s) 144771 ns 19584 ns 7.39
batchnorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/8 thread(s) 149292 ns 18875 ns 7.91
batchnorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/1 thread(s) 145000 ns 19896 ns 7.29
batchnorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/CUDA 484165.5 ns 291009.5 ns 1.66
batchnorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/oneAPI 10024099 ns 19691709 ns 0.51
batchnorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/Metal 4663146 ns 4873604 ns 0.96
batchnorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/AMDGPU 819939 ns 691816 ns 1.19
batchedmm(16, Bsize=512)/forward/CPU/2 thread(s) 58833.5 ns 59750 ns 0.98
batchedmm(16, Bsize=512)/forward/CPU/4 thread(s) 66541.5 ns 65125 ns 1.02
batchedmm(16, Bsize=512)/forward/CPU/8 thread(s) 66854 ns 66229 ns 1.01
batchedmm(16, Bsize=512)/forward/CPU/1 thread(s) 51250 ns 51250 ns 1
batchedmm(16, Bsize=512)/forward/GPU/CUDA 66577 ns 66341 ns 1.00
batchedmm(16, Bsize=512)/forward/GPU/AMDGPU 101381 ns 96856 ns 1.05
batchedmm(16, Bsize=512)/zygote/CPU/2 thread(s) 134562.5 ns 149084 ns 0.90
batchedmm(16, Bsize=512)/zygote/CPU/4 thread(s) 166750 ns 109437.5 ns 1.52
batchedmm(16, Bsize=512)/zygote/CPU/8 thread(s) 164792 ns 142625 ns 1.16
batchedmm(16, Bsize=512)/zygote/CPU/1 thread(s) 233729 ns 252625 ns 0.93
batchedmm(16, Bsize=512)/zygote/GPU/CUDA 213970 ns 218082 ns 0.98
batchedmm(16, Bsize=512)/zygote/GPU/AMDGPU 586847 ns 579290.5 ns 1.01
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 83583 ns 128229.5 ns 0.65
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 86166 ns 124458 ns 0.69
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 86791 ns 121520.5 ns 0.71
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 84666 ns 84354 ns 1.00
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/CUDA 193431 ns 193150.5 ns 1.00
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 5266325 ns 5581378 ns 0.94
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/Metal 1961875 ns 1913292 ns 1.03
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 170922 ns 170532 ns 1.00
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 1910750 ns 1825541.5 ns 1.05
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 1913479.5 ns 1917500 ns 1.00
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 1930458.5 ns 1726708 ns 1.12
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 1911667 ns 1896375 ns 1.01
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 526485 ns 531416 ns 0.99
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 25811415 ns 25804121 ns 1.00
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/Metal 8890041 ns 9091041.5 ns 0.98
groupnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1086682 ns 1081700 ns 1.00
dense(2, bias=true, act=identity)(2 x 128)/forward/CPU/2 thread(s) 292 ns 291 ns 1.00
dense(2, bias=true, act=identity)(2 x 128)/forward/CPU/4 thread(s) 292 ns 292 ns 1
dense(2, bias=true, act=identity)(2 x 128)/forward/CPU/8 thread(s) 292 ns 333 ns 0.88
dense(2, bias=true, act=identity)(2 x 128)/forward/CPU/1 thread(s) 291 ns 291 ns 1
dense(2, bias=true, act=identity)(2 x 128)/forward/GPU/CUDA 21239 ns 21564 ns 0.98
dense(2, bias=true, act=identity)(2 x 128)/forward/GPU/oneAPI 2029526 ns 2150170.5 ns 0.94
dense(2, bias=true, act=identity)(2 x 128)/forward/GPU/Metal 329625 ns 322646 ns 1.02
dense(2, bias=true, act=identity)(2 x 128)/forward/GPU/AMDGPU 45115.5 ns 44940 ns 1.00
dense(2, bias=true, act=identity)(2 x 128)/zygote/CPU/2 thread(s) 1792 ns 1833 ns 0.98
dense(2, bias=true, act=identity)(2 x 128)/zygote/CPU/4 thread(s) 1875 ns 1791 ns 1.05
dense(2, bias=true, act=identity)(2 x 128)/zygote/CPU/8 thread(s) 1833 ns 1834 ns 1.00
dense(2, bias=true, act=identity)(2 x 128)/zygote/CPU/1 thread(s) 1792 ns 1792 ns 1
dense(2, bias=true, act=identity)(2 x 128)/zygote/GPU/CUDA 249589.5 ns 253017.5 ns 0.99
dense(2, bias=true, act=identity)(2 x 128)/zygote/GPU/oneAPI 10044931 ns 9954512 ns 1.01
dense(2, bias=true, act=identity)(2 x 128)/zygote/GPU/Metal 1052750 ns 1489959 ns 0.71
dense(2, bias=true, act=identity)(2 x 128)/zygote/GPU/AMDGPU 183272 ns 184662 ns 0.99
groupnorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/2 thread(s) 10083 ns 8208 ns 1.23
groupnorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/4 thread(s) 8854.5 ns 8354 ns 1.06
groupnorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/8 thread(s) 11041.5 ns 11062.5 ns 1.00
groupnorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/1 thread(s) 9000 ns 11375 ns 0.79
groupnorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/CUDA 116531 ns 117709 ns 0.99
groupnorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/oneAPI 3670049 ns 3377279 ns 1.09
groupnorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/Metal 818083.5 ns 841167 ns 0.97
groupnorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/AMDGPU 239873 ns 237972 ns 1.01
groupnorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/2 thread(s) 8541 ns 10750 ns 0.79
groupnorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/4 thread(s) 8479.5 ns 9625 ns 0.88
groupnorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/8 thread(s) 8854.5 ns 10333 ns 0.86
groupnorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/1 thread(s) 8084 ns 9437.5 ns 0.86
groupnorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/CUDA 513528 ns 528567.5 ns 0.97
groupnorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/oneAPI 20228071.5 ns 20481875 ns 0.99
groupnorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/Metal 3987521 ns 4066875 ns 0.98
groupnorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/AMDGPU 650677.5 ns 650956 ns 1.00
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 57958 ns 58458 ns 0.99
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 46500 ns 46834 ns 0.99
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 46417 ns 46541 ns 1.00
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 83000 ns 82770.5 ns 1.00
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/CUDA 38759 ns 40116 ns 0.97
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 1415113 ns 1353318 ns 1.05
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/Metal 1142604 ns 1107646 ns 1.03
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 76641 ns 75891 ns 1.01
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 2016958 ns 1830958 ns 1.10
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 2108125 ns 1987709 ns 1.06
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 2126125 ns 1806000 ns 1.18
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 2031604.5 ns 1902167 ns 1.07
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 403273 ns 224182 ns 1.80
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 12592175 ns 33930875 ns 0.37
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/Metal 11201375 ns 11292291.5 ns 0.99
batchnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1027411 ns 1025890 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 418375 ns 418083 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 418875 ns 418854.5 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 421396 ns 419624.5 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 431167 ns 418083 ns 1.03
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/CUDA 207671.5 ns 210311 ns 0.99
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 7803352.5 ns 7920822 ns 0.99
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/Metal 527542 ns 525521 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 287424 ns 284163 ns 1.01
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 757041.5 ns 669416.5 ns 1.13
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 668625 ns 671291.5 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 677167 ns 684750 ns 0.99
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 669208 ns 684021 ns 0.98
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 1038754 ns 1058312 ns 0.98
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 45709188 ns 44385100.5 ns 1.03
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/Metal 6421375 ns 6341125 ns 1.01
layernorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 924830 ns 918153 ns 1.01
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 3445500 ns 3455395.5 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 3467104 ns 3437542 ns 1.01
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 3440104 ns 3456500 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 3397583 ns 3441812 ns 0.99
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/CUDA 169978 ns 173936 ns 0.98
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 8429258 ns 8236547 ns 1.02
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/Metal 1333667 ns 1383541.5 ns 0.96
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 431205 ns 408024 ns 1.06
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 6187042 ns 6212292 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 6193833.5 ns 6192374.5 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 6188917 ns 6230104.5 ns 0.99
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 6203437.5 ns 6210542 ns 1.00
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 985127.5 ns 1001699 ns 0.98
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 50910191.5 ns 52343757.5 ns 0.97
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/Metal 7089083 ns 7314167 ns 0.97
layernorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1562022.5 ns 1560500 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/forward/CPU/2 thread(s) 471645.5 ns 471667 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/forward/CPU/4 thread(s) 341416 ns 341500 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/forward/CPU/8 thread(s) 341958 ns 341250 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/forward/CPU/1 thread(s) 903541 ns 901083.5 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/forward/GPU/CUDA 46127 ns 46237 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/forward/GPU/oneAPI 834768.5 ns 841979 ns 0.99
dense(512, bias=true, act=gelu)(512 x 128)/forward/GPU/Metal 398208 ns 403541.5 ns 0.99
dense(512, bias=true, act=gelu)(512 x 128)/forward/GPU/AMDGPU 251432 ns 251513 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/zygote/CPU/2 thread(s) 2328124.5 ns 2304875 ns 1.01
dense(512, bias=true, act=gelu)(512 x 128)/zygote/CPU/4 thread(s) 2034625 ns 2036291 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/zygote/CPU/8 thread(s) 2036333 ns 2035208 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/zygote/CPU/1 thread(s) 3318312.5 ns 3278208.5 ns 1.01
dense(512, bias=true, act=gelu)(512 x 128)/zygote/GPU/CUDA 265130.5 ns 256609 ns 1.03
dense(512, bias=true, act=gelu)(512 x 128)/zygote/GPU/oneAPI 16240147 ns 13028144 ns 1.25
dense(512, bias=true, act=gelu)(512 x 128)/zygote/GPU/Metal 2194667 ns 2192084 ns 1.00
dense(512, bias=true, act=gelu)(512 x 128)/zygote/GPU/AMDGPU 792439 ns 788718 ns 1.00
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 57583 ns 57833 ns 1.00
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 46500 ns 46584 ns 1.00
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 46291 ns 46083 ns 1.00
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 82542 ns 83709 ns 0.99
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/CUDA 27969 ns 28664 ns 0.98
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 1424400 ns 1387212 ns 1.03
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/Metal 1133750 ns 1120333 ns 1.01
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 76011 ns 77001 ns 0.99
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 2193084 ns 1999146 ns 1.10
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 2214979 ns 2075834 ns 1.07
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 2249042 ns 1881917 ns 1.20
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 2140854 ns 1993250 ns 1.07
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 445719.5 ns 229523 ns 1.94
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 38789932 ns 36882455.5 ns 1.05
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/Metal 11702791.5 ns 11806542 ns 0.99
batchnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1094522 ns 1046160 ns 1.05
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 57750 ns 57917 ns 1.00
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 47083 ns 47250 ns 1.00
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 46500 ns 46750 ns 0.99
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 82833.5 ns 83250 ns 0.99
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/CUDA 47723 ns 49927 ns 0.96
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 785704 ns 787820 ns 1.00
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/Metal 1095166 ns 1080583 ns 1.01
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 77990.5 ns 73870 ns 1.06
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 2086062.5 ns 1891083.5 ns 1.10
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 2116208 ns 1970208 ns 1.07
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 2144666.5 ns 1955187.5 ns 1.10
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 2046292 ns 1904291 ns 1.07
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 487919.5 ns 234662 ns 2.08
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 28265363.5 ns 18211943.5 ns 1.55
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/Metal 10193083.5 ns 10103542 ns 1.01
batchnorm(4, act=identity, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1010551 ns 933389 ns 1.08
batchnorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/2 thread(s) 292 ns 292 ns 1
batchnorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/4 thread(s) 375 ns 292 ns 1.28
batchnorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/8 thread(s) 375 ns 417 ns 0.90
batchnorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/1 thread(s) 291 ns 333 ns 0.87
batchnorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/CUDA 33775 ns 34917 ns 0.97
batchnorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/oneAPI 1145972 ns 1226168.5 ns 0.93
batchnorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/Metal 269646 ns 272625 ns 0.99
batchnorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/AMDGPU 50361 ns 47950 ns 1.05
batchnorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/2 thread(s) 143208 ns 7479.5 ns 19.15
batchnorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/4 thread(s) 141625 ns 6792 ns 20.85
batchnorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/8 thread(s) 144500 ns 8375 ns 17.25
batchnorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/1 thread(s) 140292 ns 7291 ns 19.24
batchnorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/CUDA 430404 ns 203469.5 ns 2.12
batchnorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/oneAPI 21186553 ns 20240791 ns 1.05
batchnorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/Metal 4566833 ns 4583500 ns 1.00
batchnorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/AMDGPU 517175 ns 374783 ns 1.38
dense(2, bias=false, act=identity)(2 x 128)/forward/CPU/2 thread(s) 250 ns 250 ns 1
dense(2, bias=false, act=identity)(2 x 128)/forward/CPU/4 thread(s) 291 ns 250 ns 1.16
dense(2, bias=false, act=identity)(2 x 128)/forward/CPU/8 thread(s) 291 ns 292 ns 1.00
dense(2, bias=false, act=identity)(2 x 128)/forward/CPU/1 thread(s) 250 ns 250 ns 1
dense(2, bias=false, act=identity)(2 x 128)/forward/GPU/CUDA 32093 ns 31986 ns 1.00
dense(2, bias=false, act=identity)(2 x 128)/forward/GPU/oneAPI 1228680 ns 1276838 ns 0.96
dense(2, bias=false, act=identity)(2 x 128)/forward/GPU/Metal 251125 ns 250958.5 ns 1.00
dense(2, bias=false, act=identity)(2 x 128)/forward/GPU/AMDGPU 40461 ns 39251 ns 1.03
dense(2, bias=false, act=identity)(2 x 128)/zygote/CPU/2 thread(s) 2667 ns 3000 ns 0.89
dense(2, bias=false, act=identity)(2 x 128)/zygote/CPU/4 thread(s) 2834 ns 2666 ns 1.06
dense(2, bias=false, act=identity)(2 x 128)/zygote/CPU/8 thread(s) 2875 ns 3000 ns 0.96
dense(2, bias=false, act=identity)(2 x 128)/zygote/CPU/1 thread(s) 2709 ns 3250 ns 0.83
dense(2, bias=false, act=identity)(2 x 128)/zygote/GPU/CUDA 184759.5 ns 193112 ns 0.96
dense(2, bias=false, act=identity)(2 x 128)/zygote/GPU/oneAPI 7793763 ns 7648888.5 ns 1.02
dense(2, bias=false, act=identity)(2 x 128)/zygote/GPU/Metal 963604.5 ns 1228042 ns 0.78
dense(2, bias=false, act=identity)(2 x 128)/zygote/GPU/AMDGPU 159696.5 ns 155301 ns 1.03
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 443666 ns 423250 ns 1.05
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 426750 ns 422000 ns 1.01
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 427375 ns 426584 ns 1.00
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 425854.5 ns 433042 ns 0.98
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/CUDA 137307.5 ns 138742 ns 0.99
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 4108067 ns 6023505.5 ns 0.68
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/Metal 2074313 ns 2150209 ns 0.96
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 326613 ns 350454 ns 0.93
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 3803750 ns 3765146 ns 1.01
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 3802666 ns 3779584 ns 1.01
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 3822250 ns 3801667 ns 1.01
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 3801875 ns 3781770.5 ns 1.01
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 703504 ns 710296 ns 0.99
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 33026409 ns 31895528 ns 1.04
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/Metal 10878208 ns 10614458 ns 1.02
groupnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1505796 ns 1323602 ns 1.14
batchedmm(512, Bsize=32)/forward/CPU/2 thread(s) 49864187 ns 49864000 ns 1.00
batchedmm(512, Bsize=32)/forward/CPU/4 thread(s) 35504062.5 ns 35497062 ns 1.00
batchedmm(512, Bsize=32)/forward/CPU/8 thread(s) 35538333.5 ns 35537125 ns 1.00
batchedmm(512, Bsize=32)/forward/CPU/1 thread(s) 96916667 ns 96997520.5 ns 1.00
batchedmm(512, Bsize=32)/forward/GPU/CUDA 1592214 ns 1604687 ns 0.99
batchedmm(512, Bsize=32)/forward/GPU/AMDGPU 998606.5 ns 1017349 ns 0.98
batchedmm(512, Bsize=32)/zygote/CPU/2 thread(s) 154562395.5 ns 154531062.5 ns 1.00
batchedmm(512, Bsize=32)/zygote/CPU/4 thread(s) 112345958 ns 112258062 ns 1.00
batchedmm(512, Bsize=32)/zygote/CPU/8 thread(s) 112532584 ns 112366667 ns 1.00
batchedmm(512, Bsize=32)/zygote/CPU/1 thread(s) 297051812.5 ns 299279978.5 ns 0.99
batchedmm(512, Bsize=32)/zygote/GPU/CUDA 6520428 ns 6477003 ns 1.01
batchedmm(512, Bsize=32)/zygote/GPU/AMDGPU 5676402.5 ns 5749519.5 ns 0.99
bias_activation(32, act=tanh)(32 x 128)/forward/CPU/2 thread(s) 18229 ns 19666.5 ns 0.93
bias_activation(32, act=tanh)(32 x 128)/forward/CPU/4 thread(s) 17083.5 ns 18542 ns 0.92
bias_activation(32, act=tanh)(32 x 128)/forward/CPU/8 thread(s) 16042 ns 17562.5 ns 0.91
bias_activation(32, act=tanh)(32 x 128)/forward/CPU/1 thread(s) 15625 ns 15437.5 ns 1.01
bias_activation(32, act=tanh)(32 x 128)/forward/GPU/CUDA 21334 ns 21582 ns 0.99
bias_activation(32, act=tanh)(32 x 128)/forward/GPU/oneAPI 1139016 ns 1137074 ns 1.00
bias_activation(32, act=tanh)(32 x 128)/forward/GPU/Metal 217520.5 ns 219625 ns 0.99
bias_activation(32, act=tanh)(32 x 128)/forward/GPU/AMDGPU 26211 ns 27981 ns 0.94
bias_activation(32, act=tanh)(32 x 128)/zygote/CPU/2 thread(s) 10834 ns 10854.5 ns 1.00
bias_activation(32, act=tanh)(32 x 128)/zygote/CPU/4 thread(s) 9042 ns 8916.5 ns 1.01
bias_activation(32, act=tanh)(32 x 128)/zygote/CPU/8 thread(s) 9291 ns 9292 ns 1.00
bias_activation(32, act=tanh)(32 x 128)/zygote/CPU/1 thread(s) 17521 ns 17417 ns 1.01
bias_activation(32, act=tanh)(32 x 128)/zygote/GPU/CUDA 257401 ns 261948.5 ns 0.98
bias_activation(32, act=tanh)(32 x 128)/zygote/GPU/oneAPI 10553951.5 ns 9733148 ns 1.08
bias_activation(32, act=tanh)(32 x 128)/zygote/GPU/Metal 1502875 ns 1502000 ns 1.00
bias_activation(32, act=tanh)(32 x 128)/zygote/GPU/AMDGPU 154702 ns 152941 ns 1.01
groupnorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/2 thread(s) 8937.5 ns 8021 ns 1.11
groupnorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/4 thread(s) 9459 ns 8458 ns 1.12
groupnorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/8 thread(s) 10208 ns 10375 ns 0.98
groupnorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/1 thread(s) 8520.5 ns 9583 ns 0.89
groupnorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/CUDA 122970.5 ns 125031 ns 0.98
groupnorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/oneAPI 3704089 ns 3572138 ns 1.04
groupnorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/Metal 708166 ns 766396 ns 0.92
groupnorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/AMDGPU 242933 ns 239027.5 ns 1.02
groupnorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/2 thread(s) 9270.5 ns 10270.5 ns 0.90
groupnorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/4 thread(s) 9791 ns 9125 ns 1.07
groupnorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/8 thread(s) 9792 ns 9833 ns 1.00
groupnorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/1 thread(s) 9229.5 ns 9562 ns 0.97
groupnorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/CUDA 608318 ns 626181 ns 0.97
groupnorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/oneAPI 25004839 ns 23291818.5 ns 1.07
groupnorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/Metal 4935416.5 ns 5110500 ns 0.97
groupnorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/AMDGPU 655337 ns 669926 ns 0.98
groupnorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/2 thread(s) 8812.5 ns 9021 ns 0.98
groupnorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/4 thread(s) 9791.5 ns 9292 ns 1.05
groupnorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/8 thread(s) 12083 ns 10792 ns 1.12
groupnorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/1 thread(s) 8771 ns 9624.5 ns 0.91
groupnorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/CUDA 118050 ns 119634.5 ns 0.99
groupnorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/oneAPI 3451608 ns 3516934 ns 0.98
groupnorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/Metal 875291.5 ns 854750 ns 1.02
groupnorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/AMDGPU 72861 ns 69771 ns 1.04
groupnorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/2 thread(s) 13250 ns 16583 ns 0.80
groupnorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/4 thread(s) 13375 ns 12583 ns 1.06
groupnorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/8 thread(s) 13916.5 ns 14020.5 ns 0.99
groupnorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/1 thread(s) 12958 ns 14104 ns 0.92
groupnorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/CUDA 580235.5 ns 597754 ns 0.97
groupnorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/oneAPI 19330798 ns 19866015 ns 0.97
groupnorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/Metal 4607833.5 ns 4399958 ns 1.05
groupnorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/AMDGPU 354624 ns 354973.5 ns 1.00
batchnorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/2 thread(s) 500 ns 459 ns 1.09
batchnorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/4 thread(s) 584 ns 500 ns 1.17
batchnorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/8 thread(s) 583 ns 584 ns 1.00
batchnorm(2, act=gelu, affine=false)(4 x 32)/forward/CPU/1 thread(s) 459 ns 459 ns 1
batchnorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/CUDA 34114 ns 35591 ns 0.96
batchnorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/oneAPI 1260842 ns 1301172 ns 0.97
batchnorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/Metal 274542 ns 273021 ns 1.01
batchnorm(2, act=gelu, affine=false)(4 x 32)/forward/GPU/AMDGPU 209992 ns 208092 ns 1.01
batchnorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/2 thread(s) 138958 ns 9667 ns 14.37
batchnorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/4 thread(s) 136542 ns 7917 ns 17.25
batchnorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/8 thread(s) 140000 ns 8667 ns 16.15
batchnorm(2, act=gelu, affine=false)(4 x 32)/zygote/CPU/1 thread(s) 136083 ns 10542 ns 12.91
batchnorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/CUDA 438919.5 ns 228879.5 ns 1.92
batchnorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/oneAPI 22996988 ns 22093342 ns 1.04
batchnorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/Metal 4752042 ns 4715584 ns 1.01
batchnorm(2, act=gelu, affine=false)(4 x 32)/zygote/GPU/AMDGPU 792008 ns 665037 ns 1.19
bias_activation(32, act=gelu)(32 x 128)/forward/CPU/2 thread(s) 16250 ns 16792 ns 0.97
bias_activation(32, act=gelu)(32 x 128)/forward/CPU/4 thread(s) 15333 ns 18042 ns 0.85
bias_activation(32, act=gelu)(32 x 128)/forward/CPU/8 thread(s) 14625 ns 15104 ns 0.97
bias_activation(32, act=gelu)(32 x 128)/forward/CPU/1 thread(s) 11792 ns 10520.5 ns 1.12
bias_activation(32, act=gelu)(32 x 128)/forward/GPU/CUDA 21202 ns 21410 ns 0.99
bias_activation(32, act=gelu)(32 x 128)/forward/GPU/oneAPI 1151785 ns 1212361 ns 0.95
bias_activation(32, act=gelu)(32 x 128)/forward/GPU/Metal 211187.5 ns 204104 ns 1.03
bias_activation(32, act=gelu)(32 x 128)/forward/GPU/AMDGPU 188407 ns 189022 ns 1.00
bias_activation(32, act=gelu)(32 x 128)/zygote/CPU/2 thread(s) 31708 ns 31875 ns 0.99
bias_activation(32, act=gelu)(32 x 128)/zygote/CPU/4 thread(s) 31958 ns 31709 ns 1.01
bias_activation(32, act=gelu)(32 x 128)/zygote/CPU/8 thread(s) 32375 ns 32312.5 ns 1.00
bias_activation(32, act=gelu)(32 x 128)/zygote/CPU/1 thread(s) 32166 ns 32000 ns 1.01
bias_activation(32, act=gelu)(32 x 128)/zygote/GPU/CUDA 271317 ns 276685 ns 0.98
bias_activation(32, act=gelu)(32 x 128)/zygote/GPU/oneAPI 10945554 ns 10782012.5 ns 1.02
bias_activation(32, act=gelu)(32 x 128)/zygote/GPU/Metal 1602812.5 ns 1597917 ns 1.00
bias_activation(32, act=gelu)(32 x 128)/zygote/GPU/AMDGPU 607056 ns 603936 ns 1.01
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 479875 ns 444417 ns 1.08
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 483521 ns 440729.5 ns 1.10
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 446874.5 ns 483875 ns 0.92
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 484500 ns 487833 ns 0.99
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/CUDA 195239 ns 194859 ns 1.00
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 6117630 ns 6150501 ns 0.99
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/Metal 1972500 ns 1973354.5 ns 1.00
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 353204 ns 352013 ns 1.00
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 3824750 ns 3829083 ns 1.00
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 3813709 ns 3817542 ns 1.00
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 3827041 ns 3807333.5 ns 1.01
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 3827375 ns 3833750 ns 1.00
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 535235 ns 543447.5 ns 0.98
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 29684309 ns 29268457 ns 1.01
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/Metal 9492709 ns 9074479 ns 1.05
groupnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1391296 ns 1381293 ns 1.01
batchedmm(512, Bsize=512)/forward/CPU/2 thread(s) 783309416 ns 782808250 ns 1.00
batchedmm(512, Bsize=512)/forward/CPU/4 thread(s) 542819541 ns 542955458 ns 1.00
batchedmm(512, Bsize=512)/forward/CPU/8 thread(s) 544218417 ns 543245416 ns 1.00
batchedmm(512, Bsize=512)/forward/CPU/1 thread(s) 1569457250 ns 1526913187.5 ns 1.03
batchedmm(512, Bsize=512)/forward/GPU/CUDA 22543017 ns 22538913 ns 1.00
batchedmm(512, Bsize=512)/forward/GPU/AMDGPU 14103431 ns 14166095 ns 1.00
batchedmm(512, Bsize=512)/zygote/CPU/2 thread(s) 3007940334 ns 2518672041 ns 1.19
batchedmm(512, Bsize=512)/zygote/CPU/4 thread(s) 1800866291 ns 2247031041 ns 0.80
batchedmm(512, Bsize=512)/zygote/CPU/8 thread(s) 1792982875 ns 2268043292 ns 0.79
batchedmm(512, Bsize=512)/zygote/CPU/1 thread(s) 5315494791 ns 4817775208 ns 1.10
batchedmm(512, Bsize=512)/zygote/GPU/CUDA 379420999 ns 370296484 ns 1.02
batchedmm(512, Bsize=512)/zygote/GPU/AMDGPU 89218848 ns 89108951 ns 1.00
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 76770.5 ns 78291.5 ns 0.98
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 80583 ns 76292 ns 1.06
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 79646 ns 78708.5 ns 1.01
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 76625 ns 75666.5 ns 1.01
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/CUDA 205479.5 ns 209649 ns 0.98
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 8543893 ns 7907666.5 ns 1.08
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/Metal 527542 ns 527271 ns 1.00
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 109941 ns 110221 ns 1.00
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 289687.5 ns 267125 ns 1.08
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 277688 ns 192750 ns 1.44
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 193750 ns 228291 ns 0.85
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 264166.5 ns 274042 ns 0.96
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 1027978 ns 1049164 ns 0.98
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 44758239 ns 42798073 ns 1.05
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/Metal 6106375 ns 5942583 ns 1.03
layernorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 645427 ns 646896 ns 1.00
batchedmm(512, Bsize=128)/forward/CPU/2 thread(s) 199655375 ns 199999187.5 ns 1.00
batchedmm(512, Bsize=128)/forward/CPU/4 thread(s) 138887125 ns 139287958 ns 1.00
batchedmm(512, Bsize=128)/forward/CPU/8 thread(s) 139073250 ns 139251125 ns 1.00
batchedmm(512, Bsize=128)/forward/CPU/1 thread(s) 394615000 ns 388390459 ns 1.02
batchedmm(512, Bsize=128)/forward/GPU/CUDA 5834346 ns 5842600.5 ns 1.00
batchedmm(512, Bsize=128)/forward/GPU/AMDGPU 3411391.5 ns 3422748 ns 1.00
batchedmm(512, Bsize=128)/zygote/CPU/2 thread(s) 619240375 ns 618321291.5 ns 1.00
batchedmm(512, Bsize=128)/zygote/CPU/4 thread(s) 440309458 ns 440516458 ns 1.00
batchedmm(512, Bsize=128)/zygote/CPU/8 thread(s) 440130812.5 ns 441449562.5 ns 1.00
batchedmm(512, Bsize=128)/zygote/CPU/1 thread(s) 1195765667 ns 1184281125 ns 1.01
batchedmm(512, Bsize=128)/zygote/GPU/CUDA 26511719.5 ns 26535363 ns 1.00
batchedmm(512, Bsize=128)/zygote/GPU/AMDGPU 22284299 ns 22224253 ns 1.00
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 7250 ns 7292 ns 0.99
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 6209 ns 6041 ns 1.03
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 6166 ns 6042 ns 1.02
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 10000 ns 9959 ns 1.00
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/CUDA 27372 ns 28005 ns 0.98
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 1220411 ns 1278165.5 ns 0.95
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/Metal 513792 ns 585916.5 ns 0.88
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 48370 ns 47711 ns 1.01
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 380229.5 ns 214417 ns 1.77
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 359459 ns 220791 ns 1.63
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 363875 ns 221750 ns 1.64
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 335125 ns 208354 ns 1.61
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 418348.5 ns 227388 ns 1.84
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 31583019 ns 32450442 ns 0.97
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/Metal 9289000 ns 9078291.5 ns 1.02
batchnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 668577 ns 531855 ns 1.26
groupnorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/2 thread(s) 10187.5 ns 9708.5 ns 1.05
groupnorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/4 thread(s) 8667 ns 7875 ns 1.10
groupnorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/8 thread(s) 10459 ns 10334 ns 1.01
groupnorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/1 thread(s) 8166.5 ns 8896 ns 0.92
groupnorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/CUDA 117261.5 ns 116991 ns 1.00
groupnorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/oneAPI 3477087 ns 3379932 ns 1.03
groupnorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/Metal 755208 ns 844125 ns 0.89
groupnorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/AMDGPU 68860 ns 78200 ns 0.88
groupnorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/2 thread(s) 7541.5 ns 10500 ns 0.72
groupnorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/4 thread(s) 7667 ns 7209 ns 1.06
groupnorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/8 thread(s) 8292 ns 7958 ns 1.04
groupnorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/1 thread(s) 7521 ns 8125 ns 0.93
groupnorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/CUDA 510936.5 ns 524925 ns 0.97
groupnorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/oneAPI 19179630 ns 19844305 ns 0.97
groupnorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/Metal 4013292 ns 4066333.5 ns 0.99
groupnorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/AMDGPU 321554 ns 322173 ns 1.00
batchnorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/2 thread(s) 459 ns 584 ns 0.79
batchnorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/4 thread(s) 541 ns 500 ns 1.08
batchnorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/8 thread(s) 542 ns 625 ns 0.87
batchnorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/1 thread(s) 500 ns 625 ns 0.80
batchnorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/CUDA 25702 ns 26471.5 ns 0.97
batchnorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/oneAPI 1237540.5 ns 1215499 ns 1.02
batchnorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/Metal 303395.5 ns 366750 ns 0.83
batchnorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/AMDGPU 49140 ns 48170.5 ns 1.02
batchnorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/2 thread(s) 148125 ns 12709 ns 11.66
batchnorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/4 thread(s) 145187.5 ns 9291 ns 15.63
batchnorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/8 thread(s) 150208.5 ns 10083.5 ns 14.90
batchnorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/1 thread(s) 145416.5 ns 9750 ns 14.91
batchnorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/CUDA 461372 ns 258485 ns 1.78
batchnorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/oneAPI 23030274 ns 22618575 ns 1.02
batchnorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/Metal 5062875 ns 5040750 ns 1.00
batchnorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/AMDGPU 565486 ns 393458.5 ns 1.44
bias_activation(512, act=gelu)(512 x 128)/forward/CPU/2 thread(s) 115125 ns 108458 ns 1.06
bias_activation(512, act=gelu)(512 x 128)/forward/CPU/4 thread(s) 99688 ns 98875 ns 1.01
bias_activation(512, act=gelu)(512 x 128)/forward/CPU/8 thread(s) 101270.5 ns 100521 ns 1.01
bias_activation(512, act=gelu)(512 x 128)/forward/CPU/1 thread(s) 146604 ns 146417 ns 1.00
bias_activation(512, act=gelu)(512 x 128)/forward/GPU/CUDA 24185 ns 24425.5 ns 0.99
bias_activation(512, act=gelu)(512 x 128)/forward/GPU/oneAPI 1199345 ns 1223728 ns 0.98
bias_activation(512, act=gelu)(512 x 128)/forward/GPU/Metal 261959 ns 258208 ns 1.01
bias_activation(512, act=gelu)(512 x 128)/forward/GPU/AMDGPU 191147 ns 191276.5 ns 1.00
bias_activation(512, act=gelu)(512 x 128)/zygote/CPU/2 thread(s) 498209 ns 478583 ns 1.04
bias_activation(512, act=gelu)(512 x 128)/zygote/CPU/4 thread(s) 517875 ns 480437 ns 1.08
bias_activation(512, act=gelu)(512 x 128)/zygote/CPU/8 thread(s) 478937.5 ns 482104 ns 0.99
bias_activation(512, act=gelu)(512 x 128)/zygote/CPU/1 thread(s) 477958 ns 478875 ns 1.00
bias_activation(512, act=gelu)(512 x 128)/zygote/GPU/CUDA 229118 ns 234461 ns 0.98
bias_activation(512, act=gelu)(512 x 128)/zygote/GPU/oneAPI 11518633.5 ns 11870954.5 ns 0.97
bias_activation(512, act=gelu)(512 x 128)/zygote/GPU/Metal 2113625 ns 2153250 ns 0.98
bias_activation(512, act=gelu)(512 x 128)/zygote/GPU/AMDGPU 624696.5 ns 620366 ns 1.01
batchedmm(16, Bsize=32)/forward/CPU/2 thread(s) 4833 ns 5416 ns 0.89
batchedmm(16, Bsize=32)/forward/CPU/4 thread(s) 6979 ns 5021 ns 1.39
batchedmm(16, Bsize=32)/forward/CPU/8 thread(s) 7416.5 ns 7000 ns 1.06
batchedmm(16, Bsize=32)/forward/CPU/1 thread(s) 4646 ns 4375 ns 1.06
batchedmm(16, Bsize=32)/forward/GPU/CUDA 15972 ns 16254 ns 0.98
batchedmm(16, Bsize=32)/forward/GPU/AMDGPU 78941 ns 79120 ns 1.00
batchedmm(16, Bsize=32)/zygote/CPU/2 thread(s) 11916 ns 13209 ns 0.90
batchedmm(16, Bsize=32)/zygote/CPU/4 thread(s) 10833.5 ns 10333 ns 1.05
batchedmm(16, Bsize=32)/zygote/CPU/8 thread(s) 11125 ns 11187.5 ns 0.99
batchedmm(16, Bsize=32)/zygote/CPU/1 thread(s) 16562.5 ns 16875 ns 0.98
batchedmm(16, Bsize=32)/zygote/GPU/CUDA 211125 ns 214352 ns 0.98
batchedmm(16, Bsize=32)/zygote/GPU/AMDGPU 370104 ns 369103.5 ns 1.00
batchedmm(16, Bsize=128)/forward/CPU/2 thread(s) 40292 ns 39084 ns 1.03
batchedmm(16, Bsize=128)/forward/CPU/4 thread(s) 50709 ns 51604 ns 0.98
batchedmm(16, Bsize=128)/forward/CPU/8 thread(s) 53437.5 ns 52875 ns 1.01
batchedmm(16, Bsize=128)/forward/CPU/1 thread(s) 13395.5 ns 13500 ns 0.99
batchedmm(16, Bsize=128)/forward/GPU/CUDA 20045 ns 21418 ns 0.94
batchedmm(16, Bsize=128)/forward/GPU/AMDGPU 86141 ns 78891 ns 1.09
batchedmm(16, Bsize=128)/zygote/CPU/2 thread(s) 41708 ns 37875 ns 1.10
batchedmm(16, Bsize=128)/zygote/CPU/4 thread(s) 32375 ns 31625 ns 1.02
batchedmm(16, Bsize=128)/zygote/CPU/8 thread(s) 30895.5 ns 31125 ns 0.99
batchedmm(16, Bsize=128)/zygote/CPU/1 thread(s) 56916.5 ns 57625 ns 0.99
batchedmm(16, Bsize=128)/zygote/GPU/CUDA 188929 ns 194392 ns 0.97
batchedmm(16, Bsize=128)/zygote/GPU/AMDGPU 417859.5 ns 418224 ns 1.00
bias_activation(2, act=tanh)(2 x 128)/forward/CPU/2 thread(s) 1979.5 ns 1854.5 ns 1.07
bias_activation(2, act=tanh)(2 x 128)/forward/CPU/4 thread(s) 1833.5 ns 1709 ns 1.07
bias_activation(2, act=tanh)(2 x 128)/forward/CPU/8 thread(s) 2375 ns 2209 ns 1.08
bias_activation(2, act=tanh)(2 x 128)/forward/CPU/1 thread(s) 1791.5 ns 1958 ns 0.91
bias_activation(2, act=tanh)(2 x 128)/forward/GPU/CUDA 20678 ns 21178 ns 0.98
bias_activation(2, act=tanh)(2 x 128)/forward/GPU/oneAPI 1163936 ns 1109464 ns 1.05
bias_activation(2, act=tanh)(2 x 128)/forward/GPU/Metal 291541 ns 296916 ns 0.98
bias_activation(2, act=tanh)(2 x 128)/forward/GPU/AMDGPU 28881 ns 28730 ns 1.01
bias_activation(2, act=tanh)(2 x 128)/zygote/CPU/2 thread(s) 2104.5 ns 2208 ns 0.95
bias_activation(2, act=tanh)(2 x 128)/zygote/CPU/4 thread(s) 2125 ns 2084 ns 1.02
bias_activation(2, act=tanh)(2 x 128)/zygote/CPU/8 thread(s) 2334 ns 2270.5 ns 1.03
bias_activation(2, act=tanh)(2 x 128)/zygote/CPU/1 thread(s) 2167 ns 2083 ns 1.04
bias_activation(2, act=tanh)(2 x 128)/zygote/GPU/CUDA 201809 ns 204626 ns 0.99
bias_activation(2, act=tanh)(2 x 128)/zygote/GPU/oneAPI 9295907.5 ns 9185930.5 ns 1.01
bias_activation(2, act=tanh)(2 x 128)/zygote/GPU/Metal 1432187.5 ns 1373917 ns 1.04
bias_activation(2, act=tanh)(2 x 128)/zygote/GPU/AMDGPU 137651 ns 136741 ns 1.01
layernorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/2 thread(s) 5229.5 ns 5187.5 ns 1.01
layernorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/4 thread(s) 5687.5 ns 5021 ns 1.13
layernorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/8 thread(s) 6375 ns 6708 ns 0.95
layernorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/1 thread(s) 4375.5 ns 5292 ns 0.83
layernorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/CUDA 142965.5 ns 146489.5 ns 0.98
layernorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/oneAPI 5606105 ns 5824136 ns 0.96
layernorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/Metal 449020.5 ns 461083 ns 0.97
layernorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/AMDGPU 62570 ns 63631 ns 0.98
layernorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/2 thread(s) 8271 ns 9208.5 ns 0.90
layernorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/4 thread(s) 8417 ns 8000 ns 1.05
layernorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/8 thread(s) 8625 ns 8667 ns 1.00
layernorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/1 thread(s) 8604.5 ns 8708 ns 0.99
layernorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/CUDA 858818 ns 883915 ns 0.97
layernorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/oneAPI 40092469.5 ns 38448137 ns 1.04
layernorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/Metal 5355812.5 ns 5404750 ns 0.99
layernorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/AMDGPU 384874 ns 389893 ns 0.99
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 56834 ns 56750 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 57708 ns 57625 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 57666 ns 57750 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 58125 ns 58292 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/CUDA 37255 ns 37898 ns 0.98
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 1223472.5 ns 1225239 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/Metal 533354.5 ns 482042 ns 1.11
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 208377.5 ns 206812 ns 1.01
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 605750 ns 461021.5 ns 1.31
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 623562.5 ns 464812.5 ns 1.34
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 628313 ns 465375 ns 1.35
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 582834 ns 443771 ns 1.31
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 532888 ns 262541 ns 2.03
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 28144268 ns 26994597 ns 1.04
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/Metal 8798458.5 ns 8201187.5 ns 1.07
batchnorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 928810 ns 814918 ns 1.14
batchedmm(128, Bsize=128)/forward/CPU/2 thread(s) 3332667 ns 3312833 ns 1.01
batchedmm(128, Bsize=128)/forward/CPU/4 thread(s) 2330333 ns 2337166.5 ns 1.00
batchedmm(128, Bsize=128)/forward/CPU/8 thread(s) 2339292 ns 2336896 ns 1.00
batchedmm(128, Bsize=128)/forward/CPU/1 thread(s) 6320375 ns 6300021 ns 1.00
batchedmm(128, Bsize=128)/forward/GPU/CUDA 205868 ns 204708 ns 1.01
batchedmm(128, Bsize=128)/forward/GPU/AMDGPU 204322.5 ns 210502 ns 0.97
batchedmm(128, Bsize=128)/zygote/CPU/2 thread(s) 11449916 ns 11472333 ns 1.00
batchedmm(128, Bsize=128)/zygote/CPU/4 thread(s) 8300833.5 ns 8296521 ns 1.00
batchedmm(128, Bsize=128)/zygote/CPU/8 thread(s) 8348000 ns 8328708 ns 1.00
batchedmm(128, Bsize=128)/zygote/CPU/1 thread(s) 21108459 ns 21128979.5 ns 1.00
batchedmm(128, Bsize=128)/zygote/GPU/CUDA 742474 ns 742043 ns 1.00
batchedmm(128, Bsize=128)/zygote/GPU/AMDGPU 1071261 ns 1071150 ns 1.00
layernorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/2 thread(s) 7041 ns 6750 ns 1.04
layernorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/4 thread(s) 6125 ns 4709 ns 1.30
layernorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/8 thread(s) 6750 ns 6667 ns 1.01
layernorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/1 thread(s) 5791 ns 7083 ns 0.82
layernorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/CUDA 136817.5 ns 140008 ns 0.98
layernorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/oneAPI 5779963 ns 5632229.5 ns 1.03
layernorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/Metal 723333 ns 742458 ns 0.97
layernorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/AMDGPU 58111 ns 58641 ns 0.99
layernorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/2 thread(s) 7166 ns 7209 ns 0.99
layernorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/4 thread(s) 7334 ns 6959 ns 1.05
layernorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/8 thread(s) 7292 ns 7541 ns 0.97
layernorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/1 thread(s) 7250 ns 7042 ns 1.03
layernorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/CUDA 740847.5 ns 764718 ns 0.97
layernorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/oneAPI 37404171.5 ns 36130678 ns 1.04
layernorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/Metal 5084958 ns 5226958.5 ns 0.97
layernorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/AMDGPU 381784 ns 383774 ns 0.99
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 124250 ns 138354.5 ns 0.90
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 124166.5 ns 98145.5 ns 1.27
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 101708.5 ns 101167 ns 1.01
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 113520.5 ns 106375 ns 1.07
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/CUDA 150861 ns 151797 ns 0.99
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 6135447 ns 5992457 ns 1.02
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/Metal 2029958 ns 2019062.5 ns 1.01
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 210087 ns 168952 ns 1.24
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 1997854.5 ns 1834167 ns 1.09
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 1999916 ns 2017208 ns 0.99
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 2027166 ns 2009167 ns 1.01
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 2018708 ns 2029979.5 ns 0.99
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 701304.5 ns 712230.5 ns 0.98
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 32626383 ns 31045366 ns 1.05
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/Metal 10780312.5 ns 10914875 ns 0.99
groupnorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1123801.5 ns 1252337 ns 0.90
batchedmm(2, Bsize=4)/forward/CPU/2 thread(s) 33041 ns 34437.5 ns 0.96
batchedmm(2, Bsize=4)/forward/CPU/4 thread(s) 35959 ns 37312.5 ns 0.96
batchedmm(2, Bsize=4)/forward/CPU/8 thread(s) 35604.5 ns 35812 ns 0.99
batchedmm(2, Bsize=4)/forward/CPU/1 thread(s) 708 ns 667 ns 1.06
batchedmm(2, Bsize=4)/forward/GPU/CUDA 15322 ns 15573 ns 0.98
batchedmm(2, Bsize=4)/forward/GPU/AMDGPU 81061 ns 72020 ns 1.13
batchedmm(2, Bsize=4)/zygote/CPU/2 thread(s) 2542 ns 2604.5 ns 0.98
batchedmm(2, Bsize=4)/zygote/CPU/4 thread(s) 2875 ns 2792 ns 1.03
batchedmm(2, Bsize=4)/zygote/CPU/8 thread(s) 2937.5 ns 2959 ns 0.99
batchedmm(2, Bsize=4)/zygote/CPU/1 thread(s) 2125 ns 2291 ns 0.93
batchedmm(2, Bsize=4)/zygote/GPU/CUDA 136167 ns 141728 ns 0.96
batchedmm(2, Bsize=4)/zygote/GPU/AMDGPU 365774 ns 348083 ns 1.05
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 7292 ns 7000 ns 1.04
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 6083 ns 5916 ns 1.03
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 6000 ns 6000 ns 1
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 10125 ns 10333 ns 0.98
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/CUDA 35823 ns 36891 ns 0.97
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 1228555 ns 1194445.5 ns 1.03
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/Metal 389833 ns 487520.5 ns 0.80
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 51200 ns 49011 ns 1.04
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 377104 ns 241166.5 ns 1.56
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 385167 ns 221000 ns 1.74
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 381167 ns 221542 ns 1.72
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 351916 ns 206542 ns 1.70
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 512269.5 ns 240362 ns 2.13
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 27785411 ns 25649976.5 ns 1.08
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/Metal 8478687.5 ns 7897958.5 ns 1.07
batchnorm(4, act=identity, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 666837 ns 521174 ns 1.28
dense(32, bias=true, act=identity)(32 x 128)/forward/CPU/2 thread(s) 3917 ns 3917 ns 1
dense(32, bias=true, act=identity)(32 x 128)/forward/CPU/4 thread(s) 3958 ns 3917 ns 1.01
dense(32, bias=true, act=identity)(32 x 128)/forward/CPU/8 thread(s) 3958 ns 3917 ns 1.01
dense(32, bias=true, act=identity)(32 x 128)/forward/CPU/1 thread(s) 3917 ns 3917 ns 1
dense(32, bias=true, act=identity)(32 x 128)/forward/GPU/CUDA 21501 ns 21897 ns 0.98
dense(32, bias=true, act=identity)(32 x 128)/forward/GPU/oneAPI 2120479 ns 2185292 ns 0.97
dense(32, bias=true, act=identity)(32 x 128)/forward/GPU/Metal 241959 ns 242250 ns 1.00
dense(32, bias=true, act=identity)(32 x 128)/forward/GPU/AMDGPU 45910 ns 47251 ns 0.97
dense(32, bias=true, act=identity)(32 x 128)/zygote/CPU/2 thread(s) 14959 ns 14917 ns 1.00
dense(32, bias=true, act=identity)(32 x 128)/zygote/CPU/4 thread(s) 15000 ns 14875 ns 1.01
dense(32, bias=true, act=identity)(32 x 128)/zygote/CPU/8 thread(s) 14875 ns 15042 ns 0.99
dense(32, bias=true, act=identity)(32 x 128)/zygote/CPU/1 thread(s) 14917 ns 14875 ns 1.00
dense(32, bias=true, act=identity)(32 x 128)/zygote/GPU/CUDA 305862 ns 311422 ns 0.98
dense(32, bias=true, act=identity)(32 x 128)/zygote/GPU/oneAPI 11568964.5 ns 12486721 ns 0.93
dense(32, bias=true, act=identity)(32 x 128)/zygote/GPU/Metal 990500 ns 997375 ns 0.99
dense(32, bias=true, act=identity)(32 x 128)/zygote/GPU/AMDGPU 194662 ns 204526.5 ns 0.95
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 151292 ns 109666.5 ns 1.38
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 105542 ns 103749.5 ns 1.02
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 105666 ns 105500 ns 1.00
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 108334 ns 120375 ns 0.90
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/CUDA 135171.5 ns 152552 ns 0.89
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 5948811 ns 5849524.5 ns 1.02
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/Metal 2056750 ns 2042500 ns 1.01
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 187802 ns 185802 ns 1.01
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 1926208 ns 1782959 ns 1.08
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 1919708 ns 1919667 ns 1.00
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 1926500 ns 1893875 ns 1.02
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 1919333 ns 1914750 ns 1.00
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 684579 ns 695036 ns 0.98
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 30815788.5 ns 30814201.5 ns 1.00
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/Metal 10672792 ns 10692646 ns 1.00
groupnorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1239603 ns 1072840 ns 1.16
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 19584 ns 19917 ns 0.98
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 18583 ns 17584 ns 1.06
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 21292 ns 21125 ns 1.01
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 17833.5 ns 19291 ns 0.92
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/CUDA 107607.5 ns 109095.5 ns 0.99
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 3724684 ns 3459371.5 ns 1.08
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/Metal 1354625 ns 1363791.5 ns 0.99
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 75895.5 ns 81501 ns 0.93
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 218833 ns 216083 ns 1.01
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 216895.5 ns 249854 ns 0.87
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 217146 ns 216667 ns 1.00
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 215166.5 ns 215958.5 ns 1.00
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 512344 ns 521547.5 ns 0.98
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 21928657 ns 20634945 ns 1.06
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/Metal 6352541.5 ns 6272667 ns 1.01
groupnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 494060.5 ns 476034 ns 1.04
batchedmm(16, Bsize=4)/forward/CPU/2 thread(s) 25000 ns 24875 ns 1.01
batchedmm(16, Bsize=4)/forward/CPU/4 thread(s) 28708 ns 30916.5 ns 0.93
batchedmm(16, Bsize=4)/forward/CPU/8 thread(s) 28000 ns 30375 ns 0.92
batchedmm(16, Bsize=4)/forward/CPU/1 thread(s) 1187.5 ns 1250 ns 0.95
batchedmm(16, Bsize=4)/forward/GPU/CUDA 15742 ns 16240 ns 0.97
batchedmm(16, Bsize=4)/forward/GPU/AMDGPU 83041 ns 82701 ns 1.00
batchedmm(16, Bsize=4)/zygote/CPU/2 thread(s) 4791 ns 4375.5 ns 1.09
batchedmm(16, Bsize=4)/zygote/CPU/4 thread(s) 5167 ns 4500 ns 1.15
batchedmm(16, Bsize=4)/zygote/CPU/8 thread(s) 5208.5 ns 5271 ns 0.99
batchedmm(16, Bsize=4)/zygote/CPU/1 thread(s) 4750 ns 4750 ns 1
batchedmm(16, Bsize=4)/zygote/GPU/CUDA 204557 ns 208732 ns 0.98
batchedmm(16, Bsize=4)/zygote/GPU/AMDGPU 380439 ns 382674 ns 0.99
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 308208 ns 306583 ns 1.01
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 315750 ns 306500 ns 1.03
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 308333 ns 309375 ns 1.00
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 305833.5 ns 307083 ns 1.00
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/CUDA 227475.5 ns 229206.5 ns 0.99
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 7965121 ns 7783651 ns 1.02
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/Metal 593750 ns 1169270.5 ns 0.51
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 278443 ns 276543 ns 1.01
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 559583 ns 537021 ns 1.04
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 539500 ns 531791.5 ns 1.01
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 533625 ns 547104.5 ns 0.98
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 529812.5 ns 535708 ns 0.99
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 1062119 ns 1083213 ns 0.98
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 43579112.5 ns 45992002.5 ns 0.95
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/Metal 6188375 ns 6107583.5 ns 1.01
layernorm(4, act=gelu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 881479 ns 867099 ns 1.02
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 22708 ns 21292 ns 1.07
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 20083 ns 20083 ns 1
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 21417 ns 21458 ns 1.00
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 19542 ns 20416 ns 0.96
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/CUDA 112201 ns 113930 ns 0.98
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 3652323 ns 3636526.5 ns 1.00
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/Metal 1495604 ns 1444708.5 ns 1.04
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 79055.5 ns 77785.5 ns 1.02
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 212750 ns 215625 ns 0.99
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 213167 ns 215500 ns 0.99
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 215021 ns 213812 ns 1.01
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 215542 ns 216500 ns 1.00
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 752835.5 ns 748686 ns 1.01
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 24398027 ns 25547600 ns 0.96
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/Metal 7403000 ns 7444708 ns 0.99
groupnorm(4, act=identity, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 550166 ns 542025 ns 1.02
layernorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/2 thread(s) 7458 ns 6792 ns 1.10
layernorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/4 thread(s) 6583 ns 6875 ns 0.96
layernorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/8 thread(s) 8271 ns 8250 ns 1.00
layernorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/1 thread(s) 7104 ns 6750 ns 1.05
layernorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/CUDA 139804.5 ns 141039.5 ns 0.99
layernorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/oneAPI 5773128 ns 5493009 ns 1.05
layernorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/Metal 730937 ns 712166.5 ns 1.03
layernorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/AMDGPU 70340 ns 70750 ns 0.99
layernorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/2 thread(s) 10042 ns 10291 ns 0.98
layernorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/4 thread(s) 10167 ns 9708 ns 1.05
layernorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/8 thread(s) 10542 ns 10625 ns 0.99
layernorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/1 thread(s) 10499.5 ns 10333 ns 1.02
layernorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/CUDA 813031 ns 833353 ns 0.98
layernorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/oneAPI 37615031 ns 38773821 ns 0.97
layernorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/Metal 5138875 ns 5198104 ns 0.99
layernorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/AMDGPU 384474 ns 387004 ns 0.99
layernorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/2 thread(s) 6708 ns 6292 ns 1.07
layernorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/4 thread(s) 5229 ns 5125 ns 1.02
layernorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/8 thread(s) 6708.5 ns 6958 ns 0.96
layernorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/1 thread(s) 7125 ns 6875 ns 1.04
layernorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/CUDA 141719 ns 144971.5 ns 0.98
layernorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/oneAPI 5586545 ns 5893830.5 ns 0.95
layernorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/Metal 717396 ns 727542 ns 0.99
layernorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/AMDGPU 62111 ns 60830 ns 1.02
layernorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/2 thread(s) 7542 ns 7479 ns 1.01
layernorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/4 thread(s) 7292 ns 7333 ns 0.99
layernorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/8 thread(s) 7542 ns 7875 ns 0.96
layernorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/1 thread(s) 7520.5 ns 7292 ns 1.03
layernorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/CUDA 771432 ns 792524 ns 0.97
layernorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/oneAPI 41795223 ns 40455917 ns 1.03
layernorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/Metal 5408708 ns 5364583.5 ns 1.01
layernorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/AMDGPU 397734 ns 399744 ns 0.99
batchedmm(128, Bsize=512)/forward/CPU/2 thread(s) 14505854.5 ns 14468708 ns 1.00
batchedmm(128, Bsize=512)/forward/CPU/4 thread(s) 10125354.5 ns 10147583 ns 1.00
batchedmm(128, Bsize=512)/forward/CPU/8 thread(s) 10158937 ns 10085542 ns 1.01
batchedmm(128, Bsize=512)/forward/CPU/1 thread(s) 27749437.5 ns 27811041 ns 1.00
batchedmm(128, Bsize=512)/forward/GPU/CUDA 540274 ns 542278 ns 1.00
batchedmm(128, Bsize=512)/forward/GPU/AMDGPU 399624 ns 384534 ns 1.04
batchedmm(128, Bsize=512)/zygote/CPU/2 thread(s) 46276833 ns 46218541.5 ns 1.00
batchedmm(128, Bsize=512)/zygote/CPU/4 thread(s) 33361542 ns 33417104.5 ns 1.00
batchedmm(128, Bsize=512)/zygote/CPU/8 thread(s) 33503958 ns 33420958 ns 1.00
batchedmm(128, Bsize=512)/zygote/CPU/1 thread(s) 85348583 ns 85450792 ns 1.00
batchedmm(128, Bsize=512)/zygote/GPU/CUDA 2829546.5 ns 2643632 ns 1.07
batchedmm(128, Bsize=512)/zygote/GPU/AMDGPU 3291354.5 ns 3285991 ns 1.00
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 66667 ns 67625 ns 0.99
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 65750 ns 67500 ns 0.97
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 68833 ns 69791 ns 0.99
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 66541 ns 68750 ns 0.97
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/CUDA 105401.5 ns 119783 ns 0.88
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 3426262.5 ns 3660768 ns 0.94
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/Metal 1470791.5 ns 1428437.5 ns 1.03
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 234912 ns 230052 ns 1.02
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 446541 ns 439833 ns 1.02
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 445937.5 ns 443166.5 ns 1.01
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 442354 ns 444771 ns 0.99
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 446375 ns 454646 ns 0.98
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 728201 ns 731518 ns 1.00
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 26874397 ns 27861313 ns 0.96
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/Metal 7885708 ns 8299458.5 ns 0.95
groupnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 812439 ns 775003 ns 1.05
batchnorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/2 thread(s) 541 ns 583 ns 0.93
batchnorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/4 thread(s) 625 ns 542 ns 1.15
batchnorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/8 thread(s) 625 ns 666 ns 0.94
batchnorm(2, act=relu, affine=false)(32 x 32)/forward/CPU/1 thread(s) 500 ns 500 ns 1
batchnorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/CUDA 31896 ns 33044 ns 0.97
batchnorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/oneAPI 1266980.5 ns 1125170 ns 1.13
batchnorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/Metal 284750 ns 329250 ns 0.86
batchnorm(2, act=relu, affine=false)(32 x 32)/forward/GPU/AMDGPU 51781 ns 48860 ns 1.06
batchnorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/2 thread(s) 140375 ns 9104 ns 15.42
batchnorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/4 thread(s) 137583 ns 8625 ns 15.95
batchnorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/8 thread(s) 142625 ns 10250 ns 13.91
batchnorm(2, act=relu, affine=false)(32 x 32)/zygote/CPU/1 thread(s) 138042 ns 9896.5 ns 13.95
batchnorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/CUDA 486965 ns 284435 ns 1.71
batchnorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/oneAPI 14162730 ns 21375944 ns 0.66
batchnorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/Metal 4861750 ns 5566187.5 ns 0.87
batchnorm(2, act=relu, affine=false)(32 x 32)/zygote/GPU/AMDGPU 517885 ns 386843 ns 1.34
dense(32, bias=false, act=gelu)(32 x 128)/forward/CPU/2 thread(s) 9833 ns 9792 ns 1.00
dense(32, bias=false, act=gelu)(32 x 128)/forward/CPU/4 thread(s) 9792 ns 9792 ns 1
dense(32, bias=false, act=gelu)(32 x 128)/forward/CPU/8 thread(s) 9833 ns 9875 ns 1.00
dense(32, bias=false, act=gelu)(32 x 128)/forward/CPU/1 thread(s) 9792 ns 9833 ns 1.00
dense(32, bias=false, act=gelu)(32 x 128)/forward/GPU/CUDA 22840 ns 23403 ns 0.98
dense(32, bias=false, act=gelu)(32 x 128)/forward/GPU/oneAPI 2189392 ns 2092475 ns 1.05
dense(32, bias=false, act=gelu)(32 x 128)/forward/GPU/Metal 219750 ns 221416 ns 0.99
dense(32, bias=false, act=gelu)(32 x 128)/forward/GPU/AMDGPU 217022 ns 215732 ns 1.01
dense(32, bias=false, act=gelu)(32 x 128)/zygote/CPU/2 thread(s) 46167 ns 46167 ns 1
dense(32, bias=false, act=gelu)(32 x 128)/zygote/CPU/4 thread(s) 45916 ns 45875 ns 1.00
dense(32, bias=false, act=gelu)(32 x 128)/zygote/CPU/8 thread(s) 46542 ns 46458 ns 1.00
dense(32, bias=false, act=gelu)(32 x 128)/zygote/CPU/1 thread(s) 46042 ns 46125 ns 1.00
dense(32, bias=false, act=gelu)(32 x 128)/zygote/GPU/CUDA 285352 ns 290850 ns 0.98
dense(32, bias=false, act=gelu)(32 x 128)/zygote/GPU/oneAPI 11520821.5 ns 11312959 ns 1.02
dense(32, bias=false, act=gelu)(32 x 128)/zygote/GPU/Metal 946416.5 ns 1043959 ns 0.91
dense(32, bias=false, act=gelu)(32 x 128)/zygote/GPU/AMDGPU 624726.5 ns 616456 ns 1.01
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 56250 ns 56333 ns 1.00
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 57125 ns 57042 ns 1.00
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 57125 ns 57167 ns 1.00
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 57709 ns 57958 ns 1.00
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/CUDA 28529 ns 29487 ns 0.97
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 1225122 ns 1284760 ns 0.95
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/Metal 577958 ns 609396 ns 0.95
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 206332 ns 217177.5 ns 0.95
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 598458 ns 459916.5 ns 1.30
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 604375.5 ns 465375 ns 1.30
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 610292 ns 498229 ns 1.22
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 609166.5 ns 449000 ns 1.36
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 450153 ns 242456 ns 1.86
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 19947300 ns 33776271 ns 0.59
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/Metal 9553437.5 ns 9662458 ns 0.99
batchnorm(4, act=gelu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 984880.5 ns 842873 ns 1.17
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 648625 ns 647916 ns 1.00
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 647375 ns 650791.5 ns 0.99
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 617542 ns 652979 ns 0.95
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 636312 ns 664625 ns 0.96
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/CUDA 204431.5 ns 206160 ns 0.99
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 8163653 ns 8473221 ns 0.96
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/Metal 1348000 ns 1347124.5 ns 1.00
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 232182 ns 237013 ns 0.98
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 2233874.5 ns 2259250 ns 0.99
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 2235250 ns 2232542 ns 1.00
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 2243542 ns 2224833 ns 1.01
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 2238083.5 ns 2241083 ns 1.00
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 956768 ns 980993 ns 0.98
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 49121505 ns 46835859 ns 1.05
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/Metal 7001041 ns 7206958 ns 0.97
layernorm(4, act=identity, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1380304 ns 1391854 ns 0.99
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 19792 ns 20083.5 ns 0.99
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 19833 ns 20916.5 ns 0.95
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 21896 ns 22625 ns 0.97
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 20833 ns 21291.5 ns 0.98
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/CUDA 111733 ns 113434.5 ns 0.99
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 3625769.5 ns 3471029 ns 1.04
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/Metal 1464666.5 ns 1349084 ns 1.09
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 77731 ns 75101 ns 1.04
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 231708.5 ns 220895.5 ns 1.05
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 219791.5 ns 228042 ns 0.96
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 220833 ns 238875 ns 0.92
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 242250 ns 219500 ns 1.10
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 722782 ns 734488 ns 0.98
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 26022488 ns 26435758 ns 0.98
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/Metal 7764979.5 ns 7569709 ns 1.03
groupnorm(4, act=relu, affine=true)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 566586 ns 566315 ns 1.00
batchnorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/2 thread(s) 541 ns 583 ns 0.93
batchnorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/4 thread(s) 625 ns 500 ns 1.25
batchnorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/8 thread(s) 625 ns 666 ns 0.94
batchnorm(2, act=relu, affine=true)(32 x 32)/forward/CPU/1 thread(s) 500 ns 500 ns 1
batchnorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/CUDA 22661 ns 23420 ns 0.97
batchnorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/oneAPI 1248244 ns 1232395 ns 1.01
batchnorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/Metal 419333 ns 304916.5 ns 1.38
batchnorm(2, act=relu, affine=true)(32 x 32)/forward/GPU/AMDGPU 50031 ns 49271 ns 1.02
batchnorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/2 thread(s) 142416 ns 9166.5 ns 15.54
batchnorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/4 thread(s) 137875 ns 9833 ns 14.02
batchnorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/8 thread(s) 142583 ns 10833 ns 13.16
batchnorm(2, act=relu, affine=true)(32 x 32)/zygote/CPU/1 thread(s) 138500 ns 9896 ns 14.00
batchnorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/CUDA 445482 ns 263097 ns 1.69
batchnorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/oneAPI 15914244 ns 24623499 ns 0.65
batchnorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/Metal 5263708 ns 5512291 ns 0.95
batchnorm(2, act=relu, affine=true)(32 x 32)/zygote/GPU/AMDGPU 546865 ns 408514 ns 1.34
groupnorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/2 thread(s) 10000 ns 10083 ns 0.99
groupnorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/4 thread(s) 8792 ns 8209 ns 1.07
groupnorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/8 thread(s) 10709 ns 10542 ns 1.02
groupnorm(2, act=identity, affine=false)(4 x 32)/forward/CPU/1 thread(s) 9416 ns 10541 ns 0.89
groupnorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/CUDA 118544.5 ns 118922 ns 1.00
groupnorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/oneAPI 3536986 ns 3616112 ns 0.98
groupnorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/Metal 813750 ns 828896 ns 0.98
groupnorm(2, act=identity, affine=false)(4 x 32)/forward/GPU/AMDGPU 71570 ns 72900.5 ns 0.98
groupnorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/2 thread(s) 7417 ns 7500 ns 0.99
groupnorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/4 thread(s) 7792 ns 7333 ns 1.06
groupnorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/8 thread(s) 8000 ns 7959 ns 1.01
groupnorm(2, act=identity, affine=false)(4 x 32)/zygote/CPU/1 thread(s) 7542 ns 7667 ns 0.98
groupnorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/CUDA 494002 ns 513625 ns 0.96
groupnorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/oneAPI 17713765 ns 17225102 ns 1.03
groupnorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/Metal 3925291.5 ns 3863708 ns 1.02
groupnorm(2, act=identity, affine=false)(4 x 32)/zygote/GPU/AMDGPU 335028.5 ns 334143 ns 1.00
bias_activation(2, act=gelu)(2 x 128)/forward/CPU/2 thread(s) 1584 ns 1458.5 ns 1.09
bias_activation(2, act=gelu)(2 x 128)/forward/CPU/4 thread(s) 1708 ns 1541 ns 1.11
bias_activation(2, act=gelu)(2 x 128)/forward/CPU/8 thread(s) 2125 ns 2000 ns 1.06
bias_activation(2, act=gelu)(2 x 128)/forward/CPU/1 thread(s) 1459 ns 1354.5 ns 1.08
bias_activation(2, act=gelu)(2 x 128)/forward/GPU/CUDA 20784 ns 20983 ns 0.99
bias_activation(2, act=gelu)(2 x 128)/forward/GPU/oneAPI 1154727.5 ns 1163032 ns 0.99
bias_activation(2, act=gelu)(2 x 128)/forward/GPU/Metal 294792 ns 299208 ns 0.99
bias_activation(2, act=gelu)(2 x 128)/forward/GPU/AMDGPU 195392 ns 188466.5 ns 1.04
bias_activation(2, act=gelu)(2 x 128)/zygote/CPU/2 thread(s) 3334 ns 3333 ns 1.00
bias_activation(2, act=gelu)(2 x 128)/zygote/CPU/4 thread(s) 3291 ns 3375 ns 0.98
bias_activation(2, act=gelu)(2 x 128)/zygote/CPU/8 thread(s) 3500 ns 3458 ns 1.01
bias_activation(2, act=gelu)(2 x 128)/zygote/CPU/1 thread(s) 3292 ns 3334 ns 0.99
bias_activation(2, act=gelu)(2 x 128)/zygote/GPU/CUDA 218874.5 ns 222731 ns 0.98
bias_activation(2, act=gelu)(2 x 128)/zygote/GPU/oneAPI 10553778.5 ns 10495351.5 ns 1.01
bias_activation(2, act=gelu)(2 x 128)/zygote/GPU/Metal 1550604.5 ns 1564958 ns 0.99
bias_activation(2, act=gelu)(2 x 128)/zygote/GPU/AMDGPU 595936 ns 593390.5 ns 1.00
bias_activation(512, act=tanh)(512 x 128)/forward/CPU/2 thread(s) 147416.5 ns 149500 ns 0.99
bias_activation(512, act=tanh)(512 x 128)/forward/CPU/4 thread(s) 127875 ns 128375 ns 1.00
bias_activation(512, act=tanh)(512 x 128)/forward/CPU/8 thread(s) 130167 ns 129542 ns 1.00
bias_activation(512, act=tanh)(512 x 128)/forward/CPU/1 thread(s) 225084 ns 225104 ns 1.00
bias_activation(512, act=tanh)(512 x 128)/forward/GPU/CUDA 23928 ns 24535 ns 0.98
bias_activation(512, act=tanh)(512 x 128)/forward/GPU/oneAPI 1233082 ns 1174771 ns 1.05
bias_activation(512, act=tanh)(512 x 128)/forward/GPU/Metal 272167 ns 264416 ns 1.03
bias_activation(512, act=tanh)(512 x 128)/forward/GPU/AMDGPU 34770 ns 36880 ns 0.94
bias_activation(512, act=tanh)(512 x 128)/zygote/CPU/2 thread(s) 159708 ns 159541 ns 1.00
bias_activation(512, act=tanh)(512 x 128)/zygote/CPU/4 thread(s) 127375 ns 138583 ns 0.92
bias_activation(512, act=tanh)(512 x 128)/zygote/CPU/8 thread(s) 110458 ns 138979.5 ns 0.79
bias_activation(512, act=tanh)(512 x 128)/zygote/CPU/1 thread(s) 284500 ns 266500 ns 1.07
bias_activation(512, act=tanh)(512 x 128)/zygote/GPU/CUDA 214034 ns 218895 ns 0.98
bias_activation(512, act=tanh)(512 x 128)/zygote/GPU/oneAPI 10830578 ns 10544510 ns 1.03
bias_activation(512, act=tanh)(512 x 128)/zygote/GPU/Metal 2013958.5 ns 1984187.5 ns 1.02
bias_activation(512, act=tanh)(512 x 128)/zygote/GPU/AMDGPU 240127.5 ns 220122.5 ns 1.09
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 7291 ns 7292 ns 1.00
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 6000 ns 6041 ns 0.99
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 5959 ns 6000 ns 0.99
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 9917 ns 10208 ns 0.97
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/CUDA 32360 ns 33604 ns 0.96
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 1256519 ns 1212259 ns 1.04
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/Metal 343458 ns 349062.5 ns 0.98
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 50810 ns 52470 ns 0.97
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 384687.5 ns 256959 ns 1.50
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 399708 ns 230729 ns 1.73
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 388770.5 ns 238125 ns 1.63
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 399666 ns 223021 ns 1.79
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 526360 ns 258353 ns 2.04
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 29342862 ns 29336585 ns 1.00
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/Metal 8734166.5 ns 8308500 ns 1.05
batchnorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 685707 ns 529375 ns 1.30
layernorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/2 thread(s) 14500 ns 15979.5 ns 0.91
layernorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/4 thread(s) 15292 ns 14667 ns 1.04
layernorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/8 thread(s) 17084 ns 16895.5 ns 1.01
layernorm(2, act=gelu, affine=false)(32 x 32)/forward/CPU/1 thread(s) 14666 ns 15625 ns 0.94
layernorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/CUDA 138947.5 ns 140165.5 ns 0.99
layernorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/oneAPI 5806482 ns 5484385 ns 1.06
layernorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/Metal 722000 ns 722958.5 ns 1.00
layernorm(2, act=gelu, affine=false)(32 x 32)/forward/GPU/AMDGPU 238282 ns 238723 ns 1.00
layernorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/2 thread(s) 23667 ns 23250 ns 1.02
layernorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/4 thread(s) 23916 ns 23479 ns 1.02
layernorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/8 thread(s) 23958 ns 24125 ns 0.99
layernorm(2, act=gelu, affine=false)(32 x 32)/zygote/CPU/1 thread(s) 23125 ns 23604 ns 0.98
layernorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/CUDA 853569.5 ns 877758 ns 0.97
layernorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/oneAPI 40236184 ns 39144107 ns 1.03
layernorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/Metal 5415250 ns 5343687.5 ns 1.01
layernorm(2, act=gelu, affine=false)(32 x 32)/zygote/GPU/AMDGPU 693447 ns 692857 ns 1.00
groupnorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/2 thread(s) 9584 ns 9375 ns 1.02
groupnorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/4 thread(s) 9416 ns 9084 ns 1.04
groupnorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/8 thread(s) 12687.5 ns 10979.5 ns 1.16
groupnorm(2, act=identity, affine=true)(32 x 32)/forward/CPU/1 thread(s) 9500 ns 9396 ns 1.01
groupnorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/CUDA 121436.5 ns 122868.5 ns 0.99
groupnorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/oneAPI 3573217 ns 3589561 ns 1.00
groupnorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/Metal 809542 ns 743916 ns 1.09
groupnorm(2, act=identity, affine=true)(32 x 32)/forward/GPU/AMDGPU 73381 ns 70681 ns 1.04
groupnorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/2 thread(s) 13334 ns 14583 ns 0.91
groupnorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/4 thread(s) 14208 ns 13709 ns 1.04
groupnorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/8 thread(s) 14438 ns 14125 ns 1.02
groupnorm(2, act=identity, affine=true)(32 x 32)/zygote/CPU/1 thread(s) 13708.5 ns 13083 ns 1.05
groupnorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/CUDA 655124 ns 673546 ns 0.97
groupnorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/oneAPI 21439559 ns 21471684.5 ns 1.00
groupnorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/Metal 5342125 ns 5194375 ns 1.03
groupnorm(2, act=identity, affine=true)(32 x 32)/zygote/GPU/AMDGPU 370548.5 ns 370953 ns 1.00
groupnorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/2 thread(s) 9250 ns 8833.5 ns 1.05
groupnorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/4 thread(s) 8959 ns 8917 ns 1.00
groupnorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/8 thread(s) 12042 ns 11500 ns 1.05
groupnorm(2, act=identity, affine=false)(32 x 32)/forward/CPU/1 thread(s) 9375 ns 9875 ns 0.95
groupnorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/CUDA 120539 ns 121948.5 ns 0.99
groupnorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/oneAPI 3310613 ns 4256003 ns 0.78
groupnorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/Metal 875959 ns 844125 ns 1.04
groupnorm(2, act=identity, affine=false)(32 x 32)/forward/GPU/AMDGPU 70721 ns 69331 ns 1.02
groupnorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/2 thread(s) 12709 ns 12729 ns 1.00
groupnorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/4 thread(s) 12958 ns 12416 ns 1.04
groupnorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/8 thread(s) 13125 ns 12792 ns 1.03
groupnorm(2, act=identity, affine=false)(32 x 32)/zygote/CPU/1 thread(s) 12541 ns 12604.5 ns 0.99
groupnorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/CUDA 540883.5 ns 557458 ns 0.97
groupnorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/oneAPI 18769186 ns 20700383 ns 0.91
groupnorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/Metal 4285771 ns 4280750 ns 1.00
groupnorm(2, act=identity, affine=false)(32 x 32)/zygote/GPU/AMDGPU 342373 ns 345693 ns 0.99
batchedmm(2, Bsize=128)/forward/CPU/2 thread(s) 30417 ns 30875.5 ns 0.99
batchedmm(2, Bsize=128)/forward/CPU/4 thread(s) 33937.5 ns 34167 ns 0.99
batchedmm(2, Bsize=128)/forward/CPU/8 thread(s) 29917 ns 31542 ns 0.95
batchedmm(2, Bsize=128)/forward/CPU/1 thread(s) 1792 ns 1792 ns 1
batchedmm(2, Bsize=128)/forward/GPU/CUDA 15983 ns 16044 ns 1.00
batchedmm(2, Bsize=128)/forward/GPU/AMDGPU 79550 ns 74080 ns 1.07
batchedmm(2, Bsize=128)/zygote/CPU/2 thread(s) 5333.5 ns 5375 ns 0.99
batchedmm(2, Bsize=128)/zygote/CPU/4 thread(s) 5188 ns 5084 ns 1.02
batchedmm(2, Bsize=128)/zygote/CPU/8 thread(s) 5417 ns 5459 ns 0.99
batchedmm(2, Bsize=128)/zygote/CPU/1 thread(s) 6666 ns 6833.5 ns 0.98
batchedmm(2, Bsize=128)/zygote/GPU/CUDA 137115 ns 140705.5 ns 0.97
batchedmm(2, Bsize=128)/zygote/GPU/AMDGPU 386874 ns 367554 ns 1.05
batchnorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/2 thread(s) 292 ns 250 ns 1.17
batchnorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/4 thread(s) 375 ns 291 ns 1.29
batchnorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/8 thread(s) 375 ns 375 ns 1
batchnorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/1 thread(s) 250 ns 250 ns 1
batchnorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/CUDA 25197 ns 26129 ns 0.96
batchnorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/oneAPI 1365775.5 ns 1277003 ns 1.07
batchnorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/Metal 399583 ns 290458 ns 1.38
batchnorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/AMDGPU 50531 ns 48125.5 ns 1.05
batchnorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/2 thread(s) 143958 ns 7125 ns 20.20
batchnorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/4 thread(s) 141958 ns 6833 ns 20.78
batchnorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/8 thread(s) 145771 ns 7542 ns 19.33
batchnorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/1 thread(s) 141666.5 ns 7041 ns 20.12
batchnorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/CUDA 405976 ns 192022.5 ns 2.11
batchnorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/oneAPI 24082892 ns 22542195.5 ns 1.07
batchnorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/Metal 5201396 ns 4967750 ns 1.05
batchnorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/AMDGPU 549066 ns 392884 ns 1.40
batchnorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/2 thread(s) 1958 ns 2000 ns 0.98
batchnorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/4 thread(s) 2083 ns 1958 ns 1.06
batchnorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/8 thread(s) 2042 ns 2084 ns 0.98
batchnorm(2, act=gelu, affine=true)(32 x 32)/forward/CPU/1 thread(s) 1958 ns 1959 ns 1.00
batchnorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/CUDA 25964 ns 27301 ns 0.95
batchnorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/oneAPI 1226664.5 ns 1311935 ns 0.94
batchnorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/Metal 440479.5 ns 441312.5 ns 1.00
batchnorm(2, act=gelu, affine=true)(32 x 32)/forward/GPU/AMDGPU 210742.5 ns 207332 ns 1.02
batchnorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/2 thread(s) 147458 ns 17271 ns 8.54
batchnorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/4 thread(s) 145583.5 ns 16729 ns 8.70
batchnorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/8 thread(s) 150146 ns 17521 ns 8.57
batchnorm(2, act=gelu, affine=true)(32 x 32)/zygote/CPU/1 thread(s) 145208 ns 16896 ns 8.59
batchnorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/CUDA 451191 ns 269988 ns 1.67
batchnorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/oneAPI 15538748.5 ns 25410174 ns 0.61
batchnorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/Metal 5521833 ns 5897125 ns 0.94
batchnorm(2, act=gelu, affine=true)(32 x 32)/zygote/GPU/AMDGPU 847204 ns 716817 ns 1.18
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 178770.5 ns 152500 ns 1.17
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 176292 ns 178084 ns 0.99
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 153833 ns 176437 ns 0.87
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 150375 ns 173584 ns 0.87
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/CUDA 199142.5 ns 203100 ns 0.98
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 7652852.5 ns 7890476 ns 0.97
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/Metal 1399625 ns 1351292 ns 1.04
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 198312 ns 177192 ns 1.12
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 1326625 ns 1312125 ns 1.01
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 1320083.5 ns 1329791 ns 0.99
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 1328187.5 ns 1323916 ns 1.00
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 1320541 ns 1330917 ns 0.99
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 894206 ns 913760 ns 0.98
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 46708540.5 ns 46233546.5 ns 1.01
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/Metal 6150333 ns 6497521 ns 0.95
layernorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1134836.5 ns 1120096 ns 1.01
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/2 thread(s) 25709 ns 24729 ns 1.04
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/4 thread(s) 25916 ns 24959 ns 1.04
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/8 thread(s) 27625 ns 27000 ns 1.02
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/CPU/1 thread(s) 25875 ns 25187.5 ns 1.03
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/CUDA 233752 ns 236630.5 ns 0.99
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/oneAPI 8257548 ns 7478324 ns 1.10
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/Metal 1008250 ns 992375 ns 1.02
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/forward/GPU/AMDGPU 120881 ns 118741 ns 1.02
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/2 thread(s) 158437.5 ns 118687.5 ns 1.33
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/4 thread(s) 182896 ns 118229 ns 1.55
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/8 thread(s) 118875 ns 119041 ns 1.00
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/CPU/1 thread(s) 117416 ns 162729 ns 0.72
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/CUDA 1061825.5 ns 1083875 ns 0.98
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/oneAPI 46743020 ns 46702412 ns 1.00
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/Metal 5873084 ns 6050937.5 ns 0.97
layernorm(4, act=relu, affine=false)(16 x 16 x 4 x 32)/zygote/GPU/AMDGPU 609097 ns 602125 ns 1.01
batchnorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/2 thread(s) 250 ns 291 ns 0.86
batchnorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/4 thread(s) 375 ns 250 ns 1.50
batchnorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/8 thread(s) 375 ns 375 ns 1
batchnorm(2, act=relu, affine=true)(4 x 32)/forward/CPU/1 thread(s) 291 ns 250 ns 1.16
batchnorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/CUDA 22335 ns 23359 ns 0.96
batchnorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/oneAPI 1169342 ns 1296199 ns 0.90
batchnorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/Metal 440167 ns 438834 ns 1.00
batchnorm(2, act=relu, affine=true)(4 x 32)/forward/GPU/AMDGPU 51530 ns 48171 ns 1.07
batchnorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/2 thread(s) 138000 ns 7187.5 ns 19.20
batchnorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/4 thread(s) 135458 ns 7083 ns 19.12
batchnorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/8 thread(s) 139250 ns 7792 ns 17.87
batchnorm(2, act=relu, affine=true)(4 x 32)/zygote/CPU/1 thread(s) 134896 ns 7292 ns 18.50
batchnorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/CUDA 393303 ns 198494 ns 1.98
batchnorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/oneAPI 25179341 ns 24144346 ns 1.04
batchnorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/Metal 5137437.5 ns 5445583 ns 0.94
batchnorm(2, act=relu, affine=true)(4 x 32)/zygote/GPU/AMDGPU 528076 ns 399424 ns 1.32
layernorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/2 thread(s) 6625 ns 6562 ns 1.01
layernorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/4 thread(s) 6166 ns 5916 ns 1.04
layernorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/8 thread(s) 7833 ns 7875 ns 0.99
layernorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/1 thread(s) 5542 ns 7312.5 ns 0.76
layernorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/CUDA 148231 ns 151846 ns 0.98
layernorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/oneAPI 5722106.5 ns 5647891 ns 1.01
layernorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/Metal 441646 ns 439083 ns 1.01
layernorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/AMDGPU 238152 ns 235952 ns 1.01
layernorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/2 thread(s) 9917 ns 9833.5 ns 1.01
layernorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/4 thread(s) 10166 ns 9812.5 ns 1.04
layernorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/8 thread(s) 10375 ns 10333 ns 1.00
layernorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/1 thread(s) 10042 ns 10000 ns 1.00
layernorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/CUDA 888256.5 ns 914543.5 ns 0.97
layernorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/oneAPI 41828927 ns 41915714.5 ns 1.00
layernorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/Metal 5636292 ns 5632541.5 ns 1.00
layernorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/AMDGPU 678427 ns 676221 ns 1.00
dense(2, bias=false, act=gelu)(2 x 128)/forward/CPU/2 thread(s) 625 ns 666 ns 0.94
dense(2, bias=false, act=gelu)(2 x 128)/forward/CPU/4 thread(s) 666 ns 625 ns 1.07
dense(2, bias=false, act=gelu)(2 x 128)/forward/CPU/8 thread(s) 666 ns 667 ns 1.00
dense(2, bias=false, act=gelu)(2 x 128)/forward/CPU/1 thread(s) 666 ns 625 ns 1.07
dense(2, bias=false, act=gelu)(2 x 128)/forward/GPU/CUDA 22088 ns 22836 ns 0.97
dense(2, bias=false, act=gelu)(2 x 128)/forward/GPU/oneAPI 2108789.5 ns 2080700 ns 1.01
dense(2, bias=false, act=gelu)(2 x 128)/forward/GPU/Metal 222959 ns 220167 ns 1.01
dense(2, bias=false, act=gelu)(2 x 128)/forward/GPU/AMDGPU 215243 ns 216072 ns 1.00
dense(2, bias=false, act=gelu)(2 x 128)/zygote/CPU/2 thread(s) 4542 ns 4625 ns 0.98
dense(2, bias=false, act=gelu)(2 x 128)/zygote/CPU/4 thread(s) 4584 ns 4500 ns 1.02
dense(2, bias=false, act=gelu)(2 x 128)/zygote/CPU/8 thread(s) 4791 ns 4833 ns 0.99
dense(2, bias=false, act=gelu)(2 x 128)/zygote/CPU/1 thread(s) 4542 ns 4625 ns 0.98
dense(2, bias=false, act=gelu)(2 x 128)/zygote/GPU/CUDA 222119.5 ns 226643.5 ns 0.98
dense(2, bias=false, act=gelu)(2 x 128)/zygote/GPU/oneAPI 9884248 ns 10356297.5 ns 0.95
dense(2, bias=false, act=gelu)(2 x 128)/zygote/GPU/Metal 1551583 ns 1566604.5 ns 0.99
dense(2, bias=false, act=gelu)(2 x 128)/zygote/GPU/AMDGPU 600866 ns 602506 ns 1.00
groupnorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/2 thread(s) 8292 ns 7833.5 ns 1.06
groupnorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/4 thread(s) 8167 ns 7833 ns 1.04
groupnorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/8 thread(s) 10167 ns 10187.5 ns 1.00
groupnorm(2, act=identity, affine=true)(4 x 32)/forward/CPU/1 thread(s) 7917 ns 8375 ns 0.95
groupnorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/CUDA 119558 ns 121209 ns 0.99
groupnorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/oneAPI 3445450 ns 3579394 ns 0.96
groupnorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/Metal 667562.5 ns 718583.5 ns 0.93
groupnorm(2, act=identity, affine=true)(4 x 32)/forward/GPU/AMDGPU 69441 ns 68421 ns 1.01
groupnorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/2 thread(s) 8209 ns 8500 ns 0.97
groupnorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/4 thread(s) 8833 ns 8250 ns 1.07
groupnorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/8 thread(s) 8959 ns 8895.5 ns 1.01
groupnorm(2, act=identity, affine=true)(4 x 32)/zygote/CPU/1 thread(s) 8208 ns 8250 ns 0.99
groupnorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/CUDA 581371.5 ns 596725 ns 0.97
groupnorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/oneAPI 20320168.5 ns 21444108.5 ns 0.95
groupnorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/Metal 4316250 ns 4307083 ns 1.00
groupnorm(2, act=identity, affine=true)(4 x 32)/zygote/GPU/AMDGPU 349214 ns 349143.5 ns 1.00
batchedmm(128, Bsize=4)/forward/CPU/2 thread(s) 126917 ns 126334 ns 1.00
batchedmm(128, Bsize=4)/forward/CPU/4 thread(s) 132875 ns 129959 ns 1.02
batchedmm(128, Bsize=4)/forward/CPU/8 thread(s) 130583 ns 130375 ns 1.00
batchedmm(128, Bsize=4)/forward/CPU/1 thread(s) 183042 ns 183500 ns 1.00
batchedmm(128, Bsize=4)/forward/GPU/CUDA 45635.5 ns 46170 ns 0.99
batchedmm(128, Bsize=4)/forward/GPU/AMDGPU 101126 ns 98026 ns 1.03
batchedmm(128, Bsize=4)/zygote/CPU/2 thread(s) 338166.5 ns 339041 ns 1.00
batchedmm(128, Bsize=4)/zygote/CPU/4 thread(s) 346812.5 ns 314708.5 ns 1.10
batchedmm(128, Bsize=4)/zygote/CPU/8 thread(s) 313875 ns 331646 ns 0.95
batchedmm(128, Bsize=4)/zygote/CPU/1 thread(s) 583145.5 ns 568833.5 ns 1.03
batchedmm(128, Bsize=4)/zygote/GPU/CUDA 188975 ns 193689 ns 0.98
batchedmm(128, Bsize=4)/zygote/GPU/AMDGPU 486375 ns 486014.5 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/forward/CPU/2 thread(s) 397541 ns 396875 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/forward/CPU/4 thread(s) 288312.5 ns 288542 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/forward/CPU/8 thread(s) 288000 ns 288416 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/forward/CPU/1 thread(s) 756125 ns 756291 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/forward/GPU/CUDA 43307.5 ns 43814 ns 0.99
dense(512, bias=true, act=identity)(512 x 128)/forward/GPU/oneAPI 1369209 ns 1380119 ns 0.99
dense(512, bias=true, act=identity)(512 x 128)/forward/GPU/Metal 417000 ns 406708 ns 1.03
dense(512, bias=true, act=identity)(512 x 128)/forward/GPU/AMDGPU 83400 ns 83471 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/zygote/CPU/2 thread(s) 1456666.5 ns 1458250 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/zygote/CPU/4 thread(s) 1135812 ns 1132042 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/zygote/CPU/8 thread(s) 1136958 ns 1134667 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/zygote/CPU/1 thread(s) 2446667 ns 2445396 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/zygote/GPU/CUDA 242723 ns 253213.5 ns 0.96
dense(512, bias=true, act=identity)(512 x 128)/zygote/GPU/oneAPI 11709621 ns 11703870 ns 1.00
dense(512, bias=true, act=identity)(512 x 128)/zygote/GPU/Metal 1775792 ns 1847229.5 ns 0.96
dense(512, bias=true, act=identity)(512 x 128)/zygote/GPU/AMDGPU 355854 ns 352553 ns 1.01
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 644646 ns 649083 ns 0.99
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 657333.5 ns 647625 ns 1.01
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 642583.5 ns 653666 ns 0.98
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 611437.5 ns 653416 ns 0.94
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/CUDA 198699.5 ns 195489.5 ns 1.02
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 8946325 ns 8194184 ns 1.09
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/Metal 1346479.5 ns 1355708 ns 0.99
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 257148 ns 256777.5 ns 1.00
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 2437416 ns 2416000 ns 1.01
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 2444333 ns 2443646 ns 1.00
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 2459917 ns 2452084 ns 1.00
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 2446646 ns 2458000 ns 1.00
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 975703 ns 1002651.5 ns 0.97
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 53584726.5 ns 52361708 ns 1.02
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/Metal 7227375 ns 7303334 ns 0.99
layernorm(4, act=relu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1475180.5 ns 1493424.5 ns 0.99
batchedmm(2, Bsize=32)/forward/CPU/2 thread(s) 32917 ns 32291.5 ns 1.02
batchedmm(2, Bsize=32)/forward/CPU/4 thread(s) 34958 ns 37125 ns 0.94
batchedmm(2, Bsize=32)/forward/CPU/8 thread(s) 34750 ns 34959 ns 0.99
batchedmm(2, Bsize=32)/forward/CPU/1 thread(s) 875 ns 958 ns 0.91
batchedmm(2, Bsize=32)/forward/GPU/CUDA 15505 ns 15494 ns 1.00
batchedmm(2, Bsize=32)/forward/GPU/AMDGPU 73541 ns 78911 ns 0.93
batchedmm(2, Bsize=32)/zygote/CPU/2 thread(s) 3167 ns 3125 ns 1.01
batchedmm(2, Bsize=32)/zygote/CPU/4 thread(s) 3271 ns 3084 ns 1.06
batchedmm(2, Bsize=32)/zygote/CPU/8 thread(s) 3375 ns 3333 ns 1.01
batchedmm(2, Bsize=32)/zygote/CPU/1 thread(s) 3041 ns 3083.5 ns 0.99
batchedmm(2, Bsize=32)/zygote/GPU/CUDA 135878 ns 140049.5 ns 0.97
batchedmm(2, Bsize=32)/zygote/GPU/AMDGPU 341823 ns 357403 ns 0.96
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 406167 ns 406729.5 ns 1.00
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 408250 ns 409167 ns 1.00
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 408750 ns 408916 ns 1.00
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 420042 ns 421291.5 ns 1.00
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/CUDA 42722 ns 43852 ns 0.97
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 1392129 ns 1426671 ns 0.98
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/Metal 1132750 ns 1145916 ns 0.99
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 240492.5 ns 242907 ns 0.99
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 4010917 ns 3884583 ns 1.03
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 4140708 ns 3997395.5 ns 1.04
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 4151999.5 ns 3997583.5 ns 1.04
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 3932958.5 ns 3773396 ns 1.04
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 429590.5 ns 238021 ns 1.80
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 40011947.5 ns 37654037 ns 1.06
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/Metal 11642250 ns 11658833 ns 1.00
batchnorm(4, act=gelu, affine=true)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1280768 ns 1239152 ns 1.03
dense(32, bias=false, act=identity)(32 x 128)/forward/CPU/2 thread(s) 3917 ns 3875 ns 1.01
dense(32, bias=false, act=identity)(32 x 128)/forward/CPU/4 thread(s) 3916 ns 3958 ns 0.99
dense(32, bias=false, act=identity)(32 x 128)/forward/CPU/8 thread(s) 3917 ns 3875 ns 1.01
dense(32, bias=false, act=identity)(32 x 128)/forward/CPU/1 thread(s) 3875 ns 3917 ns 0.99
dense(32, bias=false, act=identity)(32 x 128)/forward/GPU/CUDA 33393 ns 33239.5 ns 1.00
dense(32, bias=false, act=identity)(32 x 128)/forward/GPU/oneAPI 1245305.5 ns 1292975 ns 0.96
dense(32, bias=false, act=identity)(32 x 128)/forward/GPU/Metal 176917 ns 180000 ns 0.98
dense(32, bias=false, act=identity)(32 x 128)/forward/GPU/AMDGPU 41210 ns 40870 ns 1.01
dense(32, bias=false, act=identity)(32 x 128)/zygote/CPU/2 thread(s) 15708 ns 15709 ns 1.00
dense(32, bias=false, act=identity)(32 x 128)/zygote/CPU/4 thread(s) 16000 ns 15666 ns 1.02
dense(32, bias=false, act=identity)(32 x 128)/zygote/CPU/8 thread(s) 15959 ns 15958 ns 1.00
dense(32, bias=false, act=identity)(32 x 128)/zygote/CPU/1 thread(s) 15583 ns 15708 ns 0.99
dense(32, bias=false, act=identity)(32 x 128)/zygote/GPU/CUDA 251876 ns 255521 ns 0.99
dense(32, bias=false, act=identity)(32 x 128)/zygote/GPU/oneAPI 8767093.5 ns 9126745 ns 0.96
dense(32, bias=false, act=identity)(32 x 128)/zygote/GPU/Metal 864834 ns 866125 ns 1.00
dense(32, bias=false, act=identity)(32 x 128)/zygote/GPU/AMDGPU 169502 ns 168612 ns 1.01
dense(512, bias=false, act=relu)(512 x 128)/forward/CPU/2 thread(s) 404584 ns 404125 ns 1.00
dense(512, bias=false, act=relu)(512 x 128)/forward/CPU/4 thread(s) 295542 ns 295459 ns 1.00
dense(512, bias=false, act=relu)(512 x 128)/forward/CPU/8 thread(s) 295458 ns 295917 ns 1.00
dense(512, bias=false, act=relu)(512 x 128)/forward/CPU/1 thread(s) 760375 ns 760584 ns 1.00
dense(512, bias=false, act=relu)(512 x 128)/forward/GPU/CUDA 113075 ns 113603 ns 1.00
dense(512, bias=false, act=relu)(512 x 128)/forward/GPU/oneAPI 999782 ns 1047585 ns 0.95
dense(512, bias=false, act=relu)(512 x 128)/forward/GPU/Metal 389917 ns 403542 ns 0.97
dense(512, bias=false, act=relu)(512 x 128)/forward/GPU/AMDGPU 89371 ns 89361 ns 1.00
dense(512, bias=false, act=relu)(512 x 128)/zygote/CPU/2 thread(s) 1490875 ns 1484917 ns 1.00
dense(512, bias=false, act=relu)(512 x 128)/zygote/CPU/4 thread(s) 1158708 ns 1156042 ns 1.00
dense(512, bias=false, act=relu)(512 x 128)/zygote/CPU/8 thread(s) 1162750 ns 1159666 ns 1.00
dense(512, bias=false, act=relu)(512 x 128)/zygote/CPU/1 thread(s) 2503542 ns 2467083.5 ns 1.01
dense(512, bias=false, act=relu)(512 x 128)/zygote/GPU/CUDA 233635 ns 253490 ns 0.92
dense(512, bias=false, act=relu)(512 x 128)/zygote/GPU/oneAPI 12058492.5 ns 11945518 ns 1.01
dense(512, bias=false, act=relu)(512 x 128)/zygote/GPU/Metal 1844333 ns 1868708 ns 0.99
dense(512, bias=false, act=relu)(512 x 128)/zygote/GPU/AMDGPU 357394 ns 357078 ns 1.00
batchnorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/2 thread(s) 459 ns 458 ns 1.00
batchnorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/4 thread(s) 584 ns 500 ns 1.17
batchnorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/8 thread(s) 583 ns 583 ns 1
batchnorm(2, act=gelu, affine=true)(4 x 32)/forward/CPU/1 thread(s) 500 ns 459 ns 1.09
batchnorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/CUDA 25186 ns 26377 ns 0.95
batchnorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/oneAPI 1362627.5 ns 1231417 ns 1.11
batchnorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/Metal 285833 ns 289375 ns 0.99
batchnorm(2, act=gelu, affine=true)(4 x 32)/forward/GPU/AMDGPU 210112 ns 208532 ns 1.01
batchnorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/2 thread(s) 139125.5 ns 8083 ns 17.21
batchnorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/4 thread(s) 136708 ns 8084 ns 16.91
batchnorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/8 thread(s) 140166 ns 8916 ns 15.72
batchnorm(2, act=gelu, affine=true)(4 x 32)/zygote/CPU/1 thread(s) 135875 ns 8292 ns 16.39
batchnorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/CUDA 399319 ns 205980.5 ns 1.94
batchnorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/oneAPI 25498054 ns 25622754 ns 1.00
batchnorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/Metal 5342541.5 ns 5187020.5 ns 1.03
batchnorm(2, act=gelu, affine=true)(4 x 32)/zygote/GPU/AMDGPU 816958 ns 699842 ns 1.17
batchedmm(128, Bsize=32)/forward/CPU/2 thread(s) 832125 ns 831083.5 ns 1.00
batchedmm(128, Bsize=32)/forward/CPU/4 thread(s) 616208 ns 618375 ns 1.00
batchedmm(128, Bsize=32)/forward/CPU/8 thread(s) 621417 ns 620896 ns 1.00
batchedmm(128, Bsize=32)/forward/CPU/1 thread(s) 1542583 ns 1545833 ns 1.00
batchedmm(128, Bsize=32)/forward/GPU/CUDA 132827 ns 129305 ns 1.03
batchedmm(128, Bsize=32)/forward/GPU/AMDGPU 170262 ns 168801 ns 1.01
batchedmm(128, Bsize=32)/zygote/CPU/2 thread(s) 2690792 ns 2685437.5 ns 1.00
batchedmm(128, Bsize=32)/zygote/CPU/4 thread(s) 1998875.5 ns 1995875.5 ns 1.00
batchedmm(128, Bsize=32)/zygote/CPU/8 thread(s) 2004042 ns 2004125 ns 1.00
batchedmm(128, Bsize=32)/zygote/CPU/1 thread(s) 4930500 ns 4920541.5 ns 1.00
batchedmm(128, Bsize=32)/zygote/GPU/CUDA 239311 ns 242604 ns 0.99
batchedmm(128, Bsize=32)/zygote/GPU/AMDGPU 865799 ns 856188.5 ns 1.01
batchnorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/2 thread(s) 292 ns 291 ns 1.00
batchnorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/4 thread(s) 334 ns 292 ns 1.14
batchnorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/8 thread(s) 375 ns 375 ns 1
batchnorm(2, act=relu, affine=false)(4 x 32)/forward/CPU/1 thread(s) 291 ns 292 ns 1.00
batchnorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/CUDA 31389 ns 32780 ns 0.96
batchnorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/oneAPI 1262909 ns 1223978 ns 1.03
batchnorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/Metal 299167 ns 272646 ns 1.10
batchnorm(2, act=relu, affine=false)(4 x 32)/forward/GPU/AMDGPU 49080.5 ns 48940 ns 1.00
batchnorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/2 thread(s) 137834 ns 7042 ns 19.57
batchnorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/4 thread(s) 134834 ns 6959 ns 19.38
batchnorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/8 thread(s) 138834 ns 7750 ns 17.91
batchnorm(2, act=relu, affine=false)(4 x 32)/zygote/CPU/1 thread(s) 135000 ns 7000 ns 19.29
batchnorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/CUDA 434137 ns 221584 ns 1.96
batchnorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/oneAPI 21203255 ns 21530894 ns 0.98
batchnorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/Metal 4873458 ns 4625916 ns 1.05
batchnorm(2, act=relu, affine=false)(4 x 32)/zygote/GPU/AMDGPU 507435 ns 370743 ns 1.37
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 2432250 ns 2406167 ns 1.01
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 2409167 ns 2385438 ns 1.01
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 2387833 ns 2416791.5 ns 0.99
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 2396292 ns 2423834 ns 0.99
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/CUDA 189831 ns 195765 ns 0.97
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 8179796 ns 7988442 ns 1.02
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/Metal 1483000 ns 1576520.5 ns 0.94
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 362144 ns 358903 ns 1.01
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 4633541.5 ns 4642937.5 ns 1.00
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 4653062.5 ns 4645584 ns 1.00
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 4675166.5 ns 4665146 ns 1.00
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 4555521 ns 4650250 ns 0.98
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 886700 ns 908920 ns 0.98
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 48864880.5 ns 47384300 ns 1.03
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/Metal 6240563 ns 6194875 ns 1.01
layernorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1267073 ns 1416134 ns 0.89
bias_activation(512, act=relu)(512 x 128)/forward/CPU/2 thread(s) 7208 ns 6812.5 ns 1.06
bias_activation(512, act=relu)(512 x 128)/forward/CPU/4 thread(s) 14083.5 ns 7187 ns 1.96
bias_activation(512, act=relu)(512 x 128)/forward/CPU/8 thread(s) 7500 ns 8125 ns 0.92
bias_activation(512, act=relu)(512 x 128)/forward/CPU/1 thread(s) 6916 ns 6979 ns 0.99
bias_activation(512, act=relu)(512 x 128)/forward/GPU/CUDA 23218 ns 23336 ns 0.99
bias_activation(512, act=relu)(512 x 128)/forward/GPU/oneAPI 1159818.5 ns 1143574.5 ns 1.01
bias_activation(512, act=relu)(512 x 128)/forward/GPU/Metal 262937.5 ns 254646 ns 1.03
bias_activation(512, act=relu)(512 x 128)/forward/GPU/AMDGPU 40390 ns 33330 ns 1.21
bias_activation(512, act=relu)(512 x 128)/zygote/CPU/2 thread(s) 51479 ns 50833.5 ns 1.01
bias_activation(512, act=relu)(512 x 128)/zygote/CPU/4 thread(s) 48791.5 ns 48375 ns 1.01
bias_activation(512, act=relu)(512 x 128)/zygote/CPU/8 thread(s) 50416.5 ns 64916.5 ns 0.78
bias_activation(512, act=relu)(512 x 128)/zygote/CPU/1 thread(s) 47000 ns 63979.5 ns 0.73
bias_activation(512, act=relu)(512 x 128)/zygote/GPU/CUDA 214489 ns 218640 ns 0.98
bias_activation(512, act=relu)(512 x 128)/zygote/GPU/oneAPI 10649119.5 ns 10406518.5 ns 1.02
bias_activation(512, act=relu)(512 x 128)/zygote/GPU/Metal 2003958.5 ns 2003333 ns 1.00
bias_activation(512, act=relu)(512 x 128)/zygote/GPU/AMDGPU 223847 ns 237612 ns 0.94
batchedmm(2, Bsize=512)/forward/CPU/2 thread(s) 21459 ns 21500 ns 1.00
batchedmm(2, Bsize=512)/forward/CPU/4 thread(s) 25062.5 ns 26146 ns 0.96
batchedmm(2, Bsize=512)/forward/CPU/8 thread(s) 24375 ns 25583 ns 0.95
batchedmm(2, Bsize=512)/forward/CPU/1 thread(s) 6000 ns 5250 ns 1.14
batchedmm(2, Bsize=512)/forward/GPU/CUDA 16873 ns 16818.5 ns 1.00
batchedmm(2, Bsize=512)/forward/GPU/AMDGPU 86560 ns 85311 ns 1.01
batchedmm(2, Bsize=512)/zygote/CPU/2 thread(s) 11917 ns 12167 ns 0.98
batchedmm(2, Bsize=512)/zygote/CPU/4 thread(s) 10416 ns 10250 ns 1.02
batchedmm(2, Bsize=512)/zygote/CPU/8 thread(s) 10417 ns 10792 ns 0.97
batchedmm(2, Bsize=512)/zygote/CPU/1 thread(s) 17750 ns 17916.5 ns 0.99
batchedmm(2, Bsize=512)/zygote/GPU/CUDA 225550 ns 229003 ns 0.98
batchedmm(2, Bsize=512)/zygote/GPU/AMDGPU 374944 ns 374873 ns 1.00
dense(512, bias=true, act=relu)(512 x 128)/forward/CPU/2 thread(s) 406166 ns 406333 ns 1.00
dense(512, bias=true, act=relu)(512 x 128)/forward/CPU/4 thread(s) 297084 ns 297083 ns 1.00
dense(512, bias=true, act=relu)(512 x 128)/forward/CPU/8 thread(s) 296875 ns 297291 ns 1.00
dense(512, bias=true, act=relu)(512 x 128)/forward/CPU/1 thread(s) 762250 ns 762375 ns 1.00
dense(512, bias=true, act=relu)(512 x 128)/forward/GPU/CUDA 46316 ns 46714 ns 0.99
dense(512, bias=true, act=relu)(512 x 128)/forward/GPU/oneAPI 1373094 ns 1399660 ns 0.98
dense(512, bias=true, act=relu)(512 x 128)/forward/GPU/Metal 410792 ns 487958 ns 0.84
dense(512, bias=true, act=relu)(512 x 128)/forward/GPU/AMDGPU 89721 ns 91141 ns 0.98
dense(512, bias=true, act=relu)(512 x 128)/zygote/CPU/2 thread(s) 1485562.5 ns 1491333 ns 1.00
dense(512, bias=true, act=relu)(512 x 128)/zygote/CPU/4 thread(s) 1165333 ns 1167145.5 ns 1.00
dense(512, bias=true, act=relu)(512 x 128)/zygote/CPU/8 thread(s) 1166208 ns 1166417 ns 1.00
dense(512, bias=true, act=relu)(512 x 128)/zygote/CPU/1 thread(s) 2514500 ns 2472271 ns 1.02
dense(512, bias=true, act=relu)(512 x 128)/zygote/GPU/CUDA 287030.5 ns 283154 ns 1.01
dense(512, bias=true, act=relu)(512 x 128)/zygote/GPU/oneAPI 11586937 ns 13584197 ns 0.85
dense(512, bias=true, act=relu)(512 x 128)/zygote/GPU/Metal 2073208 ns 2090083.5 ns 0.99
dense(512, bias=true, act=relu)(512 x 128)/zygote/GPU/AMDGPU 379943 ns 380044 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 434375 ns 434000 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 436958 ns 437208 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 436708 ns 436875 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 447167 ns 446459 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/CUDA 53910.5 ns 55157 ns 0.98
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 1015096 ns 1013588 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/Metal 1130542 ns 1079041.5 ns 1.05
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 237292 ns 236567.5 ns 1.00
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 4070791.5 ns 3895271 ns 1.05
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 4204896 ns 3933625.5 ns 1.07
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 4209333 ns 4028229.5 ns 1.04
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 3891208 ns 3807292 ns 1.02
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 512126 ns 259800 ns 1.97
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 31246857.5 ns 36848551 ns 0.85
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/Metal 10831209 ns 10417062.5 ns 1.04
batchnorm(4, act=gelu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 1294643 ns 1238382.5 ns 1.05
dense(32, bias=true, act=gelu)(32 x 128)/forward/CPU/2 thread(s) 8791 ns 8708 ns 1.01
dense(32, bias=true, act=gelu)(32 x 128)/forward/CPU/4 thread(s) 7709 ns 7625 ns 1.01
dense(32, bias=true, act=gelu)(32 x 128)/forward/CPU/8 thread(s) 7667 ns 7666 ns 1.00
dense(32, bias=true, act=gelu)(32 x 128)/forward/CPU/1 thread(s) 12375 ns 12375 ns 1
dense(32, bias=true, act=gelu)(32 x 128)/forward/GPU/CUDA 23822 ns 24111 ns 0.99
dense(32, bias=true, act=gelu)(32 x 128)/forward/GPU/oneAPI 2157287 ns 2073642 ns 1.04
dense(32, bias=true, act=gelu)(32 x 128)/forward/GPU/Metal 224666.5 ns 222645.5 ns 1.01
dense(32, bias=true, act=gelu)(32 x 128)/forward/GPU/AMDGPU 217992 ns 216692 ns 1.01
dense(32, bias=true, act=gelu)(32 x 128)/zygote/CPU/2 thread(s) 45375 ns 45375 ns 1
dense(32, bias=true, act=gelu)(32 x 128)/zygote/CPU/4 thread(s) 45334 ns 45125 ns 1.00
dense(32, bias=true, act=gelu)(32 x 128)/zygote/CPU/8 thread(s) 45250 ns 45250 ns 1
dense(32, bias=true, act=gelu)(32 x 128)/zygote/CPU/1 thread(s) 45209 ns 45292 ns 1.00
dense(32, bias=true, act=gelu)(32 x 128)/zygote/GPU/CUDA 342231 ns 347040 ns 0.99
dense(32, bias=true, act=gelu)(32 x 128)/zygote/GPU/oneAPI 13553832.5 ns 13931322.5 ns 0.97
dense(32, bias=true, act=gelu)(32 x 128)/zygote/GPU/Metal 1724458.5 ns 1692667 ns 1.02
dense(32, bias=true, act=gelu)(32 x 128)/zygote/GPU/AMDGPU 676147 ns 670407 ns 1.01
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/2 thread(s) 123500 ns 89750 ns 1.38
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/4 thread(s) 90167 ns 147041 ns 0.61
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/8 thread(s) 89500 ns 124792 ns 0.72
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/CPU/1 thread(s) 80833 ns 126979.5 ns 0.64
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/CUDA 190984 ns 189741 ns 1.01
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/oneAPI 5773034.5 ns 5754517.5 ns 1.00
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/Metal 1967042 ns 1948042 ns 1.01
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/forward/GPU/AMDGPU 186732 ns 184172 ns 1.01
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/2 thread(s) 2013292 ns 2022000 ns 1.00
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/4 thread(s) 2014792 ns 2017708 ns 1.00
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/8 thread(s) 2023604.5 ns 2021875 ns 1.00
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/CPU/1 thread(s) 1964083 ns 2017979.5 ns 0.97
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/CUDA 532292 ns 539246.5 ns 0.99
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/oneAPI 31086820 ns 28808272 ns 1.08
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/Metal 9598208.5 ns 9259396 ns 1.04
groupnorm(4, act=relu, affine=false)(16 x 16 x 32 x 32)/zygote/GPU/AMDGPU 978310 ns 1110411 ns 0.88

This comment was automatically generated by workflow using github-action-benchmark.

@avik-pal avik-pal merged commit b9686e4 into main Aug 29, 2024
66 of 72 checks passed
@avik-pal avik-pal deleted the ap/warn_no_train branch August 29, 2024 23:55
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@avik-pal