-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[opt](mtmv) support rewrite when mv has date_trunc but query doesn't have #44948
base: master
Are you sure you want to change the base?
Conversation
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
run buildall |
087b8f9
to
67befe3
Compare
run buildall |
2 similar comments
run buildall |
run buildall |
PR approved by anyone and no changes requested. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1.Data insertion and query take boundary values into account, with the minimum granularity down to the millisecond level.
2."date_trunc" can be replaced with other units for testing.
3.It is compatible with the old functions. If "date_trunc" appears in a query, it can still be rewritten successfully.
4.The simplest materialized view without "group by" can be added.
5.When "date_trunc" changes the data of MTMV (Materialized Table Maintained by Views), resulting in the actual data of MTMV being inconsistent with that of the base table, whether an SQL query hits the target should be judged according to the actual situation.
eg:
mtmv:
date_trunc(, minute)
mv:
2024-12-01 00:00:00
2024-12-01 00:01:00
2024-12-01 00:02:00
2024-12-01 00:00:00
2024-12-01 00:01:00
2024-12-01 00:02:00
2024-12-01 00:03:00
tb:
2024-12-01 00:00:00
2024-12-01 00:01:00
2024-12-01 00:02:00
2024-12-01 00:00:02
2024-12-01 00:01:03
2024-12-01 00:02:59
2024-12-01 00:03:15
sql:
second >= 00:00:00 < 00:03:05
do not support rewrite
minute >= 00:00:00 < 00:03:00
support rewrite
day >= 00:00:00 < 00:00:00
day >= 00:00:00 < 00:00:00 + 1 day
support rewrite
6.When changes in the base table data cause some partitions of MTMV to become invalid, SQL can be rewritten normally through "union all".
run buildall |
PR approved by at least one committer and no changes requested. |
TPC-H: Total hot run time: 32673 ms
|
TPC-DS: Total hot run time: 197753 ms
|
ClickBench: Total hot run time: 31.72 s
|
c2a1d86
to
56d78e8
Compare
run buildall |
TPC-H: Total hot run time: 32718 ms
|
TPC-DS: Total hot run time: 190343 ms
|
ClickBench: Total hot run time: 31.24 s
|
run buildall |
TPC-H: Total hot run time: 32764 ms
|
TPC-DS: Total hot run time: 197079 ms
|
ClickBench: Total hot run time: 31.05 s
|
What problem does this PR solve?
Such as mv def is as following:
query is as following, if query filter is like
l_commitdate >= '2023-10-17' and l_commitdate < '2023-10-18'
can be rewritten successfully by mv
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
support rewrite when mv has date_trunc but query doesn't have
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)