Reproducing pytorch's nn.MultiheadAttention #430
Hawar-Dzaee
started this conversation in
Show and tell
Replies: 1 comment
-
|
Looks awesome, thanks for sharing this! |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
This notebook investigates and explains PyTorch's out-of-the-box nn.MultiheadAttention, and its arguments in order to reproduce @rasbt's MultiheadAttention class from Chapter 3.
https://github.com/Hawar-Dzaee/LLMS-FROM-SCRATCH/blob/main/Reproducing_pytorchs_functionality/pytorchs_mha.ipynb
Beta Was this translation helpful? Give feedback.
All reactions