Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Over estimation of persistent buffer size when project persistent buffer to its persistent producers #1386

Closed
liqiangxl opened this issue Nov 27, 2023 · 1 comment

Comments

@liqiangxl
Copy link
Collaborator

In normalization scheduler, if project_persistent_buffers = True, PersistentBufferProjector will project persistent tvs to inputs, otherwise it trys to project persistent tvs to its immediate persistent producers. See #419 and #343

However, the buffer size was over estimated in scheduler_utils::persistentBufferSize. This overestimation occurs because the potential to project persistent tvs onto their immediate persistent producers isn't factored into the calculations.

For example, in the following fusion, T7 and T11 are persistent, both of them can be projected to inputs. The current calculated persistent_buffer_size is based on the reduction dim of T7 and T11. The projected_persistent_buffer_size is based on the correponding dim of inputs T0, T1, and T2. If the inputs are fp16, the projected_persistent_buffer_size is smaller than persistent_buffer_size and project_persistent_buffers = True.
However, since T11 can be projected back to T7, the correct value of persistent_buffer_size should only include the reduction dim of T7. Then, the projected_persistent_buffer_size is larger than persistent_buffer_size and the correct value of project_persistent_buffers should be False.
image

@liqiangxl liqiangxl changed the title Fix over estimation of persistent buffer size when project persistent buffer to its persistent producers Over estimation of persistent buffer size when project persistent buffer to its persistent producers Nov 27, 2023
liqiangxl added a commit that referenced this issue Nov 30, 2023
…ers (#1395)

The buffer size may be over estimated in some cases. This overestimation occurs because the potential to project persistent tvs onto their immediate persistent producers isn't factored into the calculations. This PR fixed this overestimation by taking out the size of buffers which can be projected to its producers. See #1386
jacobhinkle pushed a commit that referenced this issue Dec 6, 2023
…ers (#1395)

The buffer size may be over estimated in some cases. This overestimation occurs because the potential to project persistent tvs onto their immediate persistent producers isn't factored into the calculations. This PR fixed this overestimation by taking out the size of buffers which can be projected to its producers. See #1386
jacobhinkle pushed a commit that referenced this issue Dec 6, 2023
…ers (#1395)

The buffer size may be over estimated in some cases. This overestimation occurs because the potential to project persistent tvs onto their immediate persistent producers isn't factored into the calculations. This PR fixed this overestimation by taking out the size of buffers which can be projected to its producers. See #1386
@liqiangxl
Copy link
Collaborator Author

fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant