Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Access Matrix #19

Open
wants to merge 286 commits into
base: main
Choose a base branch
from
Open

Access Matrix #19

wants to merge 286 commits into from

Conversation

liaopeiyuan
Copy link
Collaborator

@liaopeiyuan liaopeiyuan commented Aug 9, 2021

Access matrix on ComputeDAG for all tensors, following a Tiramisu-style construction and flattened, then applied to each TIR-buffer during prediction.

Results on a 1000-subset in e5:

Model: xgb
RMSE: 0.0793
R^2: 0.7567
pairwise comparision accuracy: 0.8504
mape: 10638767510.5327
average peak score@1: 0.8312
average peak score@5: 0.9035

Model: lgbm
RMSE: 0.0793
R^2: 0.7565
pairwise comparision accuracy: 0.8510
mape: 11206471241.5266
average peak score@1: 0.8373
average peak score@5: 0.9073

e2e Runs:

LGBM:

Estimated total latency: 17.081 ms Trials: 102 Used time : 229 s Next ID: -1
Mean inference time (std dev): 14.80 ms (0.00 ms)

XGB:

Estimated total latency: 16.960 ms Trials: 102 Used time : 221 s Next ID: -1
Mean inference time (std dev): 14.60 ms (0.02 ms

Ablation studies wrt naive features:

Model: xgb
RMSE: 0.0790
R^2: 0.7572
pairwise comparision accuracy: 0.8511
mape: 10192227945.6599
average peak score@1: 0.8331
average peak score@5: 0.9058

Model: lgbm
RMSE: 0.0682
R^2: 0.8189
pairwise comparision accuracy: 0.8729
mape: 10005960995.9234
average peak score@1: 0.8736
average peak score@5: 0.9355

Full-scale test results:

Model: xgb
RMSE: 0.0801
R^2: 0.7496
pairwise comparision accuracy: 0.8494
mape: 10025578232.0939
average peak score@1: 0.8284
average peak score@5: 0.9036

Model: lgbm
RMSE: 0.0770
R^2: 0.7686
pairwise comparision accuracy: 0.8564
mape: 10181567914.0740
average peak score@1: 0.8415
average peak score@5: 0.9137

@liaopeiyuan liaopeiyuan requested a review from merrymercy August 24, 2021 20:52
@liaopeiyuan
Copy link
Collaborator Author

This PR is ready for review.

@merrymercy merrymercy force-pushed the main branch 4 times, most recently from b4070bf to a3f54d7 Compare October 25, 2022 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant