Multiple matmul fusion example #2458
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is still failing but it makes it more clear what the problem is.
jacobhinkle
commented
Jun 27, 2024
This is a helper class that I'm using temporarily. I think it might be a good intermediate step to refactor the existing scheduler to use a class like this, before taking on the multiple mma stuff.
jacobhinkle
changed the base branch from
main
to
multi_matmul_schedule_prologue
jacobhinkle
changed the title
jacobhinkle
changed the base branch from
multi_matmul_schedule_prologue
to
multi_matmul_schedule_epilogue
|
!build --matmul-bench |
jacobhinkle
added a commit
that referenced
this pull request
Sep 24, 2024
This is the next PR in a series leading up to #2458. The previous PR 2913 scheduled the smem store and added the test framework. This PR schedules the smem load and the prologue, i.e. everything up to the mma instruction. The next PRs in the series will do the following: - Schedule the mma op result - Schedule the epilogue including the split-K tensor - Add test with horizontal fusion of matmuls (#2458) - Switch to multiple matmul scheduler, i.e. remove old scheduleMatmul() - Update matmul heuristic to account for horizontal fusion - Update canSchedule checks to accept horizontally fused matmul segments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stacked on #2994
This adds a test matching the Llama FFN (forward), scheduling it all into one kernel.