Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](runtime filter) Fix runtime filter producers #44293

Merged
merged 1 commit into from
Nov 20, 2024

Conversation

Gabriel39
Copy link
Contributor

@Gabriel39 Gabriel39 commented Nov 19, 2024

What problem does this PR solve?

Problem Summary:

A runtime filter producer may have multiple targets some of which are managed in global mgr and others are managed in local mgr. To process it, producer will be shared by both of global mgr and local mgr. In this PR, a producer will be always created by a local mgr and we can always find it by a queryCtx's RF mgr.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Gabriel39
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@Gabriel39
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 19, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@Gabriel39
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40548 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 450b659fe330a05f312b0c33cfa1451ce845ef93, data reload: false

------ Round 1 ----------------------------------
q1	17589	7508	7306	7306
q2	2055	162	169	162
q3	10657	1069	1138	1069
q4	10553	749	762	749
q5	7730	2840	2730	2730
q6	241	148	146	146
q7	953	607	609	607
q8	9566	1924	1960	1924
q9	6667	6403	6442	6403
q10	6975	2282	2288	2282
q11	448	264	255	255
q12	404	222	217	217
q13	17809	2963	2950	2950
q14	241	209	218	209
q15	566	514	515	514
q16	678	600	612	600
q17	984	544	532	532
q18	7253	6568	6578	6568
q19	3163	1058	1096	1058
q20	486	197	198	197
q21	3958	3294	3084	3084
q22	1058	990	986	986
Total cold run time: 110034 ms
Total hot run time: 40548 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7394	7220	7204	7204
q2	321	222	221	221
q3	2951	2842	2880	2842
q4	2070	1791	1729	1729
q5	5660	5758	5721	5721
q6	231	139	143	139
q7	2195	1757	1759	1757
q8	3274	3572	3458	3458
q9	8856	8891	8868	8868
q10	3549	3545	3512	3512
q11	588	486	492	486
q12	804	587	607	587
q13	16556	3160	3127	3127
q14	312	267	274	267
q15	570	511	509	509
q16	718	658	681	658
q17	1850	1621	1568	1568
q18	8118	7950	7350	7350
q19	4702	1605	1638	1605
q20	2089	1872	1802	1802
q21	5342	5348	5224	5224
q22	1145	1043	1029	1029
Total cold run time: 79295 ms
Total hot run time: 59663 ms

@dataroaring dataroaring merged commit 843f7b5 into apache:branch-3.0 Nov 20, 2024
21 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants