-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Over estimation of persistent buffer size when project persistent buffer to its persistent producers #1386
Comments
liqiangxl
changed the title
Fix over estimation of persistent buffer size when project persistent buffer to its persistent producers
Over estimation of persistent buffer size when project persistent buffer to its persistent producers
Nov 27, 2023
liqiangxl
added a commit
that referenced
this issue
Nov 30, 2023
…ers (#1395) The buffer size may be over estimated in some cases. This overestimation occurs because the potential to project persistent tvs onto their immediate persistent producers isn't factored into the calculations. This PR fixed this overestimation by taking out the size of buffers which can be projected to its producers. See #1386
jacobhinkle
pushed a commit
that referenced
this issue
Dec 6, 2023
…ers (#1395) The buffer size may be over estimated in some cases. This overestimation occurs because the potential to project persistent tvs onto their immediate persistent producers isn't factored into the calculations. This PR fixed this overestimation by taking out the size of buffers which can be projected to its producers. See #1386
jacobhinkle
pushed a commit
that referenced
this issue
Dec 6, 2023
…ers (#1395) The buffer size may be over estimated in some cases. This overestimation occurs because the potential to project persistent tvs onto their immediate persistent producers isn't factored into the calculations. This PR fixed this overestimation by taking out the size of buffers which can be projected to its producers. See #1386
fixed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In normalization scheduler, if
project_persistent_buffers = True
,PersistentBufferProjector
will project persistent tvs to inputs, otherwise it trys to project persistent tvs to its immediate persistent producers. See #419 and #343However, the buffer size was over estimated in
scheduler_utils::persistentBufferSize
. This overestimation occurs because the potential to project persistent tvs onto their immediate persistent producers isn't factored into the calculations.For example, in the following fusion, T7 and T11 are persistent, both of them can be projected to inputs. The current calculated
persistent_buffer_size
is based on the reduction dim of T7 and T11. Theprojected_persistent_buffer_size
is based on the correponding dim of inputs T0, T1, and T2. If the inputs are fp16, theprojected_persistent_buffer_size
is smaller thanpersistent_buffer_size
andproject_persistent_buffers = True
.However, since T11 can be projected back to T7, the correct value of
persistent_buffer_size
should only include the reduction dim of T7. Then, theprojected_persistent_buffer_size
is larger thanpersistent_buffer_size
and the correct value ofproject_persistent_buffers
should beFalse
.The text was updated successfully, but these errors were encountered: