Skip to content

Commit

Permalink
docs: update docstring dropout (#770)
Browse files Browse the repository at this point in the history
  • Loading branch information
@dmetivie
dmetivie authored Jul 16, 2024
1 parent 087faa4 commit 29e466f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/layers/dropout.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ Dropout layer.
## Keyword Arguments
- To apply dropout along certain dimension(s), specify the `dims` keyword. e.g.
`Dropout(p; dims = 3)` will randomly zero out entire channels on WHCN input
`Dropout(p; dims = (3,4))` will randomly zero out entire channels on WHCN input
(also called 2D dropout).
## Inputs
Expand Down

1 comment on commit 29e466f

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Results

Benchmark suite Current: 29e466f Previous: 9a5cc21 Ratio
Dense(2 => 2)/cpu/reverse/ReverseDiff (compiled)/(2, 128) 3708.125 ns 3689.375 ns 1.01
Dense(2 => 2)/cpu/reverse/Zygote/(2, 128) 7202.583333333333 ns 7108.25 ns 1.01
Dense(2 => 2)/cpu/reverse/Tracker/(2, 128) 20769 ns 20729 ns 1.00
Dense(2 => 2)/cpu/reverse/ReverseDiff/(2, 128) 9814.2 ns 9788.2 ns 1.00
Dense(2 => 2)/cpu/reverse/Flux/(2, 128) 8940.8 ns 8803 ns 1.02
Dense(2 => 2)/cpu/reverse/SimpleChains/(2, 128) 4521 ns 4445.75 ns 1.02
Dense(2 => 2)/cpu/reverse/Enzyme/(2, 128) 1162.1666666666667 ns 1157.5942028985507 ns 1.00
Dense(2 => 2)/cpu/forward/NamedTuple/(2, 128) 1108.0264900662253 ns 1115.3624161073826 ns 0.99
Dense(2 => 2)/cpu/forward/ComponentArray/(2, 128) 1193.234375 ns 1179.5149253731342 ns 1.01
Dense(2 => 2)/cpu/forward/Flux/(2, 128) 1782.7857142857142 ns 1789.469387755102 ns 1.00
Dense(2 => 2)/cpu/forward/SimpleChains/(2, 128) 179.94575799721835 ns 180.56022408963585 ns 1.00
Dense(20 => 20)/cpu/reverse/ReverseDiff (compiled)/(20, 128) 17322 ns 17263 ns 1.00
Dense(20 => 20)/cpu/reverse/Zygote/(20, 128) 16781 ns 16761 ns 1.00
Dense(20 => 20)/cpu/reverse/Tracker/(20, 128) 37254.5 ns 37309 ns 1.00
Dense(20 => 20)/cpu/reverse/ReverseDiff/(20, 128) 29044 ns 29264 ns 0.99
Dense(20 => 20)/cpu/reverse/Flux/(20, 128) 21280 ns 19877 ns 1.07
Dense(20 => 20)/cpu/reverse/SimpleChains/(20, 128) 17282 ns 17122 ns 1.01
Dense(20 => 20)/cpu/reverse/Enzyme/(20, 128) 4315.142857142857 ns 4326.571428571428 ns 1.00
Dense(20 => 20)/cpu/forward/NamedTuple/(20, 128) 3829.625 ns 3858.375 ns 0.99
Dense(20 => 20)/cpu/forward/ComponentArray/(20, 128) 3934.875 ns 3932.25 ns 1.00
Dense(20 => 20)/cpu/forward/Flux/(20, 128) 4817.571428571428 ns 4983.571428571428 ns 0.97
Dense(20 => 20)/cpu/forward/SimpleChains/(20, 128) 1654.1 ns 1657 ns 1.00
Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 3, 128) 41551661 ns 49278699 ns 0.84
Conv((3, 3), 3 => 3)/cpu/reverse/Zygote/(64, 64, 3, 128) 58251044.5 ns 57576751.5 ns 1.01
Conv((3, 3), 3 => 3)/cpu/reverse/Tracker/(64, 64, 3, 128) 79896817.5 ns 109160869.5 ns 0.73
Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff/(64, 64, 3, 128) 85832471 ns 106447718.5 ns 0.81
Conv((3, 3), 3 => 3)/cpu/reverse/Flux/(64, 64, 3, 128) 76140849 ns 91530370 ns 0.83
Conv((3, 3), 3 => 3)/cpu/reverse/SimpleChains/(64, 64, 3, 128) 11969223 ns 11675389.5 ns 1.03
Conv((3, 3), 3 => 3)/cpu/reverse/Enzyme/(64, 64, 3, 128) 7080191 ns 6931919 ns 1.02
Conv((3, 3), 3 => 3)/cpu/forward/NamedTuple/(64, 64, 3, 128) 7193590 ns 7098291 ns 1.01
Conv((3, 3), 3 => 3)/cpu/forward/ComponentArray/(64, 64, 3, 128) 7073414 ns 7022403 ns 1.01
Conv((3, 3), 3 => 3)/cpu/forward/Flux/(64, 64, 3, 128) 12338787.5 ns 18360298 ns 0.67
Conv((3, 3), 3 => 3)/cpu/forward/SimpleChains/(64, 64, 3, 128) 6431957 ns 6385781 ns 1.01
vgg16/cpu/reverse/Zygote/(32, 32, 3, 16) 703246914 ns 705394593 ns 1.00
vgg16/cpu/reverse/Zygote/(32, 32, 3, 64) 2570700164 ns 2616824850 ns 0.98
vgg16/cpu/reverse/Zygote/(32, 32, 3, 2) 146137798 ns 144841156.5 ns 1.01
vgg16/cpu/reverse/Tracker/(32, 32, 3, 16) 915110115.5 ns 804574120 ns 1.14
vgg16/cpu/reverse/Tracker/(32, 32, 3, 64) 3187561892 ns 3328481757 ns 0.96
vgg16/cpu/reverse/Tracker/(32, 32, 3, 2) 208837724.5 ns 209149562 ns 1.00
vgg16/cpu/reverse/Flux/(32, 32, 3, 16) 690645735.5 ns 773576256 ns 0.89
vgg16/cpu/reverse/Flux/(32, 32, 3, 64) 2812418266 ns 2806385145 ns 1.00
vgg16/cpu/reverse/Flux/(32, 32, 3, 2) 147830644.5 ns 147363592 ns 1.00
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 16) 174776833 ns 174721051 ns 1.00
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 64) 658430709 ns 654809349.5 ns 1.01
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 2) 34866148 ns 34172290.5 ns 1.02
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 16) 165524659 ns 165141726 ns 1.00
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 64) 641598171.5 ns 643839250 ns 1.00
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 2) 30417511 ns 30005067.5 ns 1.01
vgg16/cpu/forward/Flux/(32, 32, 3, 16) 188258028 ns 231201192 ns 0.81
vgg16/cpu/forward/Flux/(32, 32, 3, 64) 847204472 ns 829336756 ns 1.02
vgg16/cpu/forward/Flux/(32, 32, 3, 2) 38306001.5 ns 37372900 ns 1.02
Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 64, 128) 1222361111.5 ns 1249505098.5 ns 0.98
Conv((3, 3), 64 => 64)/cpu/reverse/Zygote/(64, 64, 64, 128) 1876739603.5 ns 1858491430 ns 1.01
Conv((3, 3), 64 => 64)/cpu/reverse/Tracker/(64, 64, 64, 128) 2318950176 ns 2371943763 ns 0.98
Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff/(64, 64, 64, 128) 2586064020 ns 2553087977 ns 1.01
Conv((3, 3), 64 => 64)/cpu/reverse/Flux/(64, 64, 64, 128) 1910576873.5 ns 1962392226 ns 0.97
Conv((3, 3), 64 => 64)/cpu/reverse/Enzyme/(64, 64, 64, 128) 328082151.5 ns 322990127.5 ns 1.02
Conv((3, 3), 64 => 64)/cpu/forward/NamedTuple/(64, 64, 64, 128) 326373060 ns 319576885 ns 1.02
Conv((3, 3), 64 => 64)/cpu/forward/ComponentArray/(64, 64, 64, 128) 318939236 ns 317940427 ns 1.00
Conv((3, 3), 64 => 64)/cpu/forward/Flux/(64, 64, 64, 128) 364450794.5 ns 389407449 ns 0.94
Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 1, 128) 12110116.5 ns 11696482 ns 1.04
Conv((3, 3), 1 => 1)/cpu/reverse/Zygote/(64, 64, 1, 128) 18065994.5 ns 17828357 ns 1.01
Conv((3, 3), 1 => 1)/cpu/reverse/Tracker/(64, 64, 1, 128) 19378161 ns 19008271 ns 1.02
Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff/(64, 64, 1, 128) 24003824 ns 23738072.5 ns 1.01
Conv((3, 3), 1 => 1)/cpu/reverse/Flux/(64, 64, 1, 128) 18052727 ns 17857580.5 ns 1.01
Conv((3, 3), 1 => 1)/cpu/reverse/SimpleChains/(64, 64, 1, 128) 1157534 ns 1163588 ns 0.99
Conv((3, 3), 1 => 1)/cpu/reverse/Enzyme/(64, 64, 1, 128) 2066505 ns 2064722 ns 1.00
Conv((3, 3), 1 => 1)/cpu/forward/NamedTuple/(64, 64, 1, 128) 2078777.5 ns 2070598 ns 1.00
Conv((3, 3), 1 => 1)/cpu/forward/ComponentArray/(64, 64, 1, 128) 2081573 ns 2074535 ns 1.00
Conv((3, 3), 1 => 1)/cpu/forward/Flux/(64, 64, 1, 128) 2073578 ns 2063896 ns 1.00
Conv((3, 3), 1 => 1)/cpu/forward/SimpleChains/(64, 64, 1, 128) 199862.5 ns 198094 ns 1.01
Dense(200 => 200)/cpu/reverse/ReverseDiff (compiled)/(200, 128) 293267 ns 292194 ns 1.00
Dense(200 => 200)/cpu/reverse/Zygote/(200, 128) 265485 ns 264052 ns 1.01
Dense(200 => 200)/cpu/reverse/Tracker/(200, 128) 369329 ns 364308 ns 1.01
Dense(200 => 200)/cpu/reverse/ReverseDiff/(200, 128) 408983 ns 407148 ns 1.00
Dense(200 => 200)/cpu/reverse/Flux/(200, 128) 275033 ns 273069 ns 1.01
Dense(200 => 200)/cpu/reverse/SimpleChains/(200, 128) 414623.5 ns 411968 ns 1.01
Dense(200 => 200)/cpu/reverse/Enzyme/(200, 128) 83776 ns 83345 ns 1.01
Dense(200 => 200)/cpu/forward/NamedTuple/(200, 128) 81302 ns 81101 ns 1.00
Dense(200 => 200)/cpu/forward/ComponentArray/(200, 128) 81362 ns 81492 ns 1.00
Dense(200 => 200)/cpu/forward/Flux/(200, 128) 87232 ns 86582 ns 1.01
Dense(200 => 200)/cpu/forward/SimpleChains/(200, 128) 104334.5 ns 104545 ns 1.00
Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 16, 128) 196580814.5 ns 196142686.5 ns 1.00
Conv((3, 3), 16 => 16)/cpu/reverse/Zygote/(64, 64, 16, 128) 325601095 ns 325153569 ns 1.00
Conv((3, 3), 16 => 16)/cpu/reverse/Tracker/(64, 64, 16, 128) 422107461.5 ns 446916300.5 ns 0.94
Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff/(64, 64, 16, 128) 460786759.5 ns 496223370 ns 0.93
Conv((3, 3), 16 => 16)/cpu/reverse/Flux/(64, 64, 16, 128) 376576919 ns 421390779 ns 0.89
Conv((3, 3), 16 => 16)/cpu/reverse/SimpleChains/(64, 64, 16, 128) 330132293 ns 322878155 ns 1.02
Conv((3, 3), 16 => 16)/cpu/reverse/Enzyme/(64, 64, 16, 128) 44296074 ns 44140967 ns 1.00
Conv((3, 3), 16 => 16)/cpu/forward/NamedTuple/(64, 64, 16, 128) 44477840 ns 44144811 ns 1.01
Conv((3, 3), 16 => 16)/cpu/forward/ComponentArray/(64, 64, 16, 128) 44123475 ns 43902323 ns 1.01
Conv((3, 3), 16 => 16)/cpu/forward/Flux/(64, 64, 16, 128) 57384877 ns 59027307 ns 0.97
Conv((3, 3), 16 => 16)/cpu/forward/SimpleChains/(64, 64, 16, 128) 28060856.5 ns 28155959 ns 1.00
Dense(2000 => 2000)/cpu/reverse/ReverseDiff (compiled)/(2000, 128) 18956391 ns 18833149 ns 1.01
Dense(2000 => 2000)/cpu/reverse/Zygote/(2000, 128) 19593936 ns 19487768.5 ns 1.01
Dense(2000 => 2000)/cpu/reverse/Tracker/(2000, 128) 23259183.5 ns 23275244 ns 1.00
Dense(2000 => 2000)/cpu/reverse/ReverseDiff/(2000, 128) 24171564 ns 24092658 ns 1.00
Dense(2000 => 2000)/cpu/reverse/Flux/(2000, 128) 19844960.5 ns 19580559 ns 1.01
Dense(2000 => 2000)/cpu/reverse/Enzyme/(2000, 128) 6539285 ns 6501763.5 ns 1.01
Dense(2000 => 2000)/cpu/forward/NamedTuple/(2000, 128) 6578089 ns 6519016 ns 1.01
Dense(2000 => 2000)/cpu/forward/ComponentArray/(2000, 128) 6592691 ns 6495883 ns 1.01
Dense(2000 => 2000)/cpu/forward/Flux/(2000, 128) 6544315 ns 6478856.5 ns 1.01

This comment was automatically generated by workflow using github-action-benchmark.

Please sign in to comment.