Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

branch-3.0: [fix](profile) only printed for non-sink nodes in the merge profile. #44040 #44251

Merged
merged 1 commit into from
Dec 3, 2024
Merged

branch-3.0: [fix](profile) only printed for non-sink nodes in the merge profile. #44040 #44251

merged 1 commit into from
Dec 3, 2024

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #44040

@Mryange
[fix](profile) only printed for non-sink nodes in the merge profile. (#…
650316c 
…44040)

### What problem does this PR solve?
In the past, the result sink did not have a plan node ID, which led to
incorrect plan info being output.
 Now, the plan info for sinks is not printed.

before
```
MergedProfile  
          Fragments:
              Fragment  0:
                  Pipeline  :  0(instance_num=1):
                        -  WaitWorkerTime:  avg  30.984us,  max  30.984us,  min  30.984us
                      RESULT_SINK_OPERATOR  (id=0):
                            -  PlanInfo
                                  -  TABLE:  test.big_string_table(big_string_table),  PREAGGREGATION:  ON
                                  -  partitions=1/1  (big_string_table)
                                  -  tablets=3/3,  tabletList=113264,113266,113268
                                  -  cardinality=131072,  avgRowSize=13.638229,  numNodes=1
                                  -  pushAggOp=COUNT
                                  -  projections:  1
                                  -  project  output  tuple  id:  1
                          OLAP_SCAN_OPERATOR  (id=0.  nereids_id=156.  table  name  =  big_string_table(big_string_table)):
```
now
```
MergedProfile  
          Fragments:
              Fragment  0:
                  Pipeline  :  0(instance_num=1):
                        -  WaitWorkerTime:  avg  32.576us,  max  32.576us,  min  32.576us
                      RESULT_SINK_OPERATOR  (id=0):
                          OLAP_SCAN_OPERATOR  (id=0.  nereids_id=156.  table  name  =  big_string_table(big_string_table)):
                                -  PlanInfo
                                      -  TABLE:  test.big_string_table(big_string_table),  PREAGGREGATION:  ON
                                      -  partitions=1/1  (big_string_table)
                                      -  tablets=3/3,  tabletList=113264,113266,113268
                                      -  cardinality=131072,  avgRowSize=0.0,  numNodes=1
                                      -  pushAggOp=COUNT
                                      -  projections:  1
                                      -  project  output  tuple  id:  1
```
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Nov 19, 2024
@doris-robot
Copy link

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 42759 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 650316c75d64b091abe6fb8c9f7f6b728571d488, data reload: false

------ Round 1 ----------------------------------
q1	17595	7518	7335	7335
q2	2034	171	168	168
q3	10696	1068	1118	1068
q4	10556	744	808	744
q5	7748	2783	2759	2759
q6	232	146	148	146
q7	958	601	603	601
q8	9577	1889	1917	1889
q9	7741	6402	6357	6357
q10	6993	2304	2337	2304
q11	461	255	253	253
q12	402	212	205	205
q13	17764	2971	2972	2971
q14	235	204	200	200
q15	557	505	516	505
q16	693	598	604	598
q17	962	574	523	523
q18	7192	6613	6520	6520
q19	1374	1019	992	992
q20	2867	2682	2666	2666
q21	3884	2978	3183	2978
q22	1059	981	977	977
Total cold run time: 111580 ms
Total hot run time: 42759 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7303	7192	7347	7192
q2	351	235	232	232
q3	3011	2855	2952	2855
q4	2034	1706	1729	1706
q5	5675	5714	5796	5714
q6	222	144	141	141
q7	2142	1777	1754	1754
q8	3228	3807	3483	3483
q9	8847	8814	8795	8795
q10	3519	3533	3504	3504
q11	595	508	502	502
q12	801	625	627	625
q13	16857	3181	3149	3149
q14	291	274	274	274
q15	553	533	514	514
q16	719	655	669	655
q17	1808	1678	1610	1610
q18	8116	7840	7477	7477
q19	5256	1626	1552	1552
q20	2073	1866	1831	1831
q21	5389	5267	5290	5267
q22	1097	1022	1004	1004
Total cold run time: 79887 ms
Total hot run time: 59836 ms

@Mryange
Copy link
Contributor

Mryange commented Nov 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40438 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 650316c75d64b091abe6fb8c9f7f6b728571d488, data reload: false

------ Round 1 ----------------------------------
q1	17564	7338	7279	7279
q2	2059	154	149	149
q3	10817	1072	1185	1072
q4	10552	755	716	716
q5	7743	2846	2809	2809
q6	235	147	142	142
q7	988	607	605	605
q8	9342	1916	1905	1905
q9	6583	6391	6372	6372
q10	6981	2282	2300	2282
q11	458	263	259	259
q12	399	215	211	211
q13	17799	2963	2978	2963
q14	236	203	212	203
q15	569	515	525	515
q16	663	597	595	595
q17	957	500	563	500
q18	7123	6680	6578	6578
q19	1845	1041	1011	1011
q20	467	196	193	193
q21	3893	3210	3107	3107
q22	1106	1008	972	972
Total cold run time: 108379 ms
Total hot run time: 40438 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7313	7203	7220	7203
q2	317	232	226	226
q3	2865	2861	2888	2861
q4	2010	1742	1809	1742
q5	5636	5668	5694	5668
q6	225	143	139	139
q7	2240	1783	1806	1783
q8	3372	3572	3563	3563
q9	8738	8884	8834	8834
q10	3507	3516	3490	3490
q11	584	514	487	487
q12	788	595	606	595
q13	16405	3166	3162	3162
q14	311	278	276	276
q15	556	513	508	508
q16	706	692	668	668
q17	1860	1648	1629	1629
q18	8140	7650	7688	7650
q19	5403	1596	1484	1484
q20	2078	1857	1860	1857
q21	5445	5368	5264	5264
q22	1140	1038	1037	1037
Total cold run time: 79639 ms
Total hot run time: 60126 ms

@Mryange
Copy link
Contributor

Mryange commented Dec 2, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40652 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 650316c75d64b091abe6fb8c9f7f6b728571d488, data reload: false

------ Round 1 ----------------------------------
q1	17558	7916	7337	7337
q2	2068	169	159	159
q3	10685	1069	1113	1069
q4	10558	760	739	739
q5	7755	2832	2799	2799
q6	230	147	147	147
q7	993	615	613	613
q8	9571	1917	1985	1917
q9	7766	6407	6366	6366
q10	6960	2296	2293	2293
q11	452	270	264	264
q12	412	226	222	222
q13	17804	2947	2957	2947
q14	236	212	211	211
q15	552	515	522	515
q16	675	600	608	600
q17	970	539	606	539
q18	7264	6531	6553	6531
q19	3162	1017	1069	1017
q20	478	200	195	195
q21	3936	3246	3219	3219
q22	1036	960	953	953
Total cold run time: 111121 ms
Total hot run time: 40652 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7390	7254	7262	7254
q2	317	240	228	228
q3	3055	2938	2919	2919
q4	2070	1767	1750	1750
q5	5642	5774	5750	5750
q6	219	143	140	140
q7	2209	1781	1771	1771
q8	3381	3438	3501	3438
q9	8948	8831	8808	8808
q10	3554	3516	3518	3516
q11	594	506	504	504
q12	831	676	590	590
q13	16751	3194	3172	3172
q14	303	263	265	263
q15	563	524	523	523
q16	690	661	666	661
q17	1830	1602	1614	1602
q18	8034	7716	7601	7601
q19	1728	1447	1583	1447
q20	2107	1804	1857	1804
q21	5474	5355	5379	5355
q22	1088	1001	1003	1001
Total cold run time: 76778 ms
Total hot run time: 60097 ms

@dataroaring dataroaring merged commit 3db632d into branch-3.0 Dec 3, 2024
22 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-44040-branch-3.0 branch December 3, 2024 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@doris-robot @Mryange @dataroaring