Allow linear to take a >2D weight and a >1D bias. #3073
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
!build |
Priya2698
reviewed
Oct 1, 2024
wujingyue
force-pushed
the
wjy/prefer
branch
2 times, most recently
from
aa7b681 to
ad951e0
Compare
|
!build |
|
!build |
wujingyue
force-pushed
the
wjy/three
branch
3 times, most recently
from
ee5b81d to
96ed5e8
Compare
|
!build |
wujingyue
force-pushed
the
wjy/three
branch
2 times, most recently
from
3d060ab to
c61cdc1
Compare
wujingyue
requested review from
Priya2698 and
cowanmeg
and removed request for
cowanmeg
|
!build |
|
!build |
cowanmeg
approved these changes
Oct 4, 2024
As long as the extra dimensions are DID-parallel.
|
!build |
|
!build |
This was referenced Oct 4, 2024
wujingyue
added a commit
that referenced
this pull request
Oct 5, 2024
This PR fixes a bug introduced in #3073. This bug causes `nvFuser.Tensor` to have a different rank than the corresponding `TensorView`. This didn't trigger any test failure until I wrote a more complicated test that `slice`s the output of a linear. Question for @rdspring1 and/or @kevinstephano: shouldn't this bug be caught earlier? I guess when the Python frontend finalizes the definition it should have checked the output `nvFuser.Tensor`s are consistent with the output `TensorView`s. Wdyt?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As long as the extra dimensions are DID-parallel.
This allows a distributed transformer layer to use linear (instead of matmul+add) for speed and will simplify the pending #3045.
To avoid ambiguity, this PR also removes the support for 1D weight and 0D bias; otherwise, it's unclear whether a 2D weight is one device dimension plus a non-device or two non-devices. This support can be added back by changing the thunder-to-nvFuser bridge to convert a 1D/0D linear to unsqueeze followed by a 2D/1D linear followed by a squeeze.