Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement] (nereids)implement showDynamicPartitionCommand in nereids #44295

Merged
merged 3 commits into from
Nov 21, 2024
Merged

[Enhancement] (nereids)implement showDynamicPartitionCommand in nereids #44295

merged 3 commits into from
Nov 21, 2024

Conversation

Vallishp
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #42746

Related PR: #xxx

Problem Summary:
[Enhancement] (nereids)implement showDynamicPartitionCommand in nereids

Release note

None

Check List (For Author)

  • Test

    • Regression test // updated
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Vallishp
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 44708 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e7948ea10a3031c80fa9a2f0763c85a4d19a1f4, data reload: false

------ Round 1 ----------------------------------
q1	17558	7451	7254	7254
q2	2229	1163	1180	1163
q3	9987	1139	1165	1139
q4	10222	709	673	673
q5	7597	2719	2736	2719
q6	238	149	147	147
q7	995	615	609	609
q8	9340	2357	2355	2355
q9	6635	6381	6386	6381
q10	7039	2249	2336	2249
q11	464	256	251	251
q12	426	216	211	211
q13	17764	3076	3041	3041
q14	249	216	209	209
q15	597	546	514	514
q16	675	604	582	582
q17	976	505	529	505
q18	7222	6574	6843	6574
q19	1345	972	1032	972
q20	2925	2684	2668	2668
q21	4121	3247	3174	3174
q22	1399	1342	1318	1318
Total cold run time: 110003 ms
Total hot run time: 44708 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7303	7247	7807	7247
q2	330	236	235	235
q3	3024	2878	3003	2878
q4	2043	1777	1828	1777
q5	5554	5662	5741	5662
q6	227	144	139	139
q7	2232	1806	1838	1806
q8	3309	3475	3467	3467
q9	8886	8903	8928	8903
q10	3558	3556	3615	3556
q11	585	489	498	489
q12	816	631	592	592
q13	13461	3245	3294	3245
q14	304	282	264	264
q15	559	517	537	517
q16	688	642	641	641
q17	1900	1642	1585	1585
q18	8159	7761	7813	7761
q19	1670	1518	1588	1518
q20	2114	1883	1937	1883
q21	5668	5489	5509	5489
q22	647	579	565	565
Total cold run time: 73037 ms
Total hot run time: 60219 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.68 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e7948ea10a3031c80fa9a2f0763c85a4d19a1f4, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.62	0.10	0.11
query5	0.41	0.44	0.39
query6	1.18	0.64	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.49	0.50
query10	0.55	0.55	0.56
query11	0.14	0.10	0.11
query12	0.15	0.11	0.11
query13	0.60	0.60	0.59
query14	2.72	2.86	2.83
query15	0.88	0.83	0.83
query16	0.39	0.38	0.39
query17	1.04	1.08	1.05
query18	0.20	0.20	0.20
query19	1.98	1.82	1.95
query20	0.01	0.01	0.01
query21	15.36	0.57	0.57
query22	2.79	2.48	2.15
query23	17.17	1.09	0.85
query24	3.37	1.64	0.64
query25	0.38	0.10	0.15
query26	0.32	0.15	0.14
query27	0.04	0.05	0.05
query28	10.27	1.12	1.08
query29	12.51	3.22	3.22
query30	0.25	0.06	0.06
query31	2.87	0.38	0.37
query32	3.27	0.45	0.46
query33	2.94	3.07	3.05
query34	16.93	4.43	4.47
query35	4.49	4.57	4.47
query36	0.66	0.48	0.48
query37	0.09	0.07	0.06
query38	0.04	0.04	0.03
query39	0.03	0.03	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.95 s
Total hot run time: 32.68 s

@Vallishp
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 45063 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e7948ea10a3031c80fa9a2f0763c85a4d19a1f4, data reload: false

------ Round 1 ----------------------------------
q1	17601	7467	7295	7295
q2	2236	1160	1169	1160
q3	10057	1161	1178	1161
q4	10350	752	753	752
q5	7579	2696	2737	2696
q6	242	153	149	149
q7	988	628	608	608
q8	9355	2357	2355	2355
q9	6519	6407	6404	6404
q10	7058	2282	2320	2282
q11	472	257	259	257
q12	498	224	215	215
q13	17792	3024	3010	3010
q14	238	214	205	205
q15	574	547	528	528
q16	676	601	600	600
q17	978	568	557	557
q18	7157	6664	6745	6664
q19	1343	1013	1010	1010
q20	2920	2692	2714	2692
q21	3954	3127	3209	3127
q22	1399	1336	1345	1336
Total cold run time: 109986 ms
Total hot run time: 45063 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7321	7248	7482	7248
q2	339	236	242	236
q3	3019	2996	2917	2917
q4	2063	1771	1772	1771
q5	5617	5678	5734	5678
q6	226	148	150	148
q7	2218	1829	1866	1829
q8	3326	3579	3481	3481
q9	8890	8907	9002	8907
q10	3594	3541	3552	3541
q11	594	516	506	506
q12	818	629	627	627
q13	13487	3236	3345	3236
q14	295	289	283	283
q15	568	514	560	514
q16	679	630	661	630
q17	1884	1650	1589	1589
q18	8166	7637	7857	7637
q19	1716	1591	1526	1526
q20	2108	1880	1912	1880
q21	5541	5574	5438	5438
q22	628	579	559	559
Total cold run time: 73097 ms
Total hot run time: 60181 ms

@Vallishp
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

ClickBench: Total hot run time: 32.75 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e7948ea10a3031c80fa9a2f0763c85a4d19a1f4, data reload: false

query1	0.03	0.04	0.03
query2	0.06	0.04	0.02
query3	0.23	0.06	0.07
query4	1.62	0.10	0.10
query5	0.43	0.41	0.39
query6	1.15	0.66	0.65
query7	0.02	0.02	0.02
query8	0.03	0.04	0.03
query9	0.58	0.49	0.50
query10	0.54	0.56	0.55
query11	0.14	0.11	0.12
query12	0.13	0.12	0.11
query13	0.60	0.61	0.60
query14	2.69	2.74	2.83
query15	0.90	0.84	0.82
query16	0.38	0.39	0.40
query17	0.98	1.03	1.01
query18	0.20	0.20	0.20
query19	1.85	1.88	2.00
query20	0.01	0.01	0.01
query21	15.36	0.58	0.59
query22	2.42	2.48	1.87
query23	17.13	0.87	0.86
query24	2.74	1.09	1.46
query25	0.27	0.09	0.09
query26	0.43	0.14	0.15
query27	0.04	0.04	0.04
query28	10.63	1.10	1.08
query29	12.52	3.20	3.23
query30	0.24	0.06	0.06
query31	2.87	0.39	0.38
query32	3.25	0.46	0.47
query33	2.95	2.98	3.03
query34	17.09	4.42	4.44
query35	4.57	4.50	4.51
query36	0.67	0.52	0.49
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.03	0.02
query40	0.17	0.12	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.04
Total cold run time: 106.22 s
Total hot run time: 32.75 s

@doris-robot
Copy link

TPC-H: Total hot run time: 45056 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e7948ea10a3031c80fa9a2f0763c85a4d19a1f4, data reload: false

------ Round 1 ----------------------------------
q1	17577	7440	7308	7308
q2	2238	1162	1166	1162
q3	9989	1150	1134	1134
q4	10228	752	764	752
q5	7616	2746	2747	2746
q6	243	153	149	149
q7	991	630	629	629
q8	9364	2373	2359	2359
q9	6647	6352	6452	6352
q10	6987	2273	2291	2273
q11	461	264	269	264
q12	419	228	214	214
q13	17764	2993	3028	2993
q14	235	210	209	209
q15	565	541	533	533
q16	648	575	585	575
q17	959	583	586	583
q18	7305	6736	6736	6736
q19	1351	1032	973	973
q20	2894	2686	2702	2686
q21	3938	3163	3107	3107
q22	1395	1345	1319	1319
Total cold run time: 109814 ms
Total hot run time: 45056 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7285	7226	7330	7226
q2	329	231	233	231
q3	3076	2923	2930	2923
q4	2063	1804	1807	1804
q5	5627	5723	5383	5383
q6	222	139	140	139
q7	2097	1746	1676	1676
q8	3278	3432	3389	3389
q9	8562	8537	8540	8537
q10	3493	3462	3423	3423
q11	590	472	482	472
q12	771	554	614	554
q13	9248	3033	3059	3033
q14	284	260	259	259
q15	567	515	511	511
q16	648	635	627	627
q17	1823	1564	1551	1551
q18	7759	7464	7429	7429
q19	1693	1553	1532	1532
q20	2034	1845	1827	1827
q21	5384	5289	5157	5157
q22	613	545	565	545
Total cold run time: 67446 ms
Total hot run time: 58228 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e7948ea10a3031c80fa9a2f0763c85a4d19a1f4, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.05	0.04
query3	0.23	0.07	0.06
query4	1.63	0.10	0.11
query5	0.44	0.40	0.40
query6	1.16	0.66	0.66
query7	0.02	0.02	0.01
query8	0.03	0.04	0.03
query9	0.56	0.50	0.50
query10	0.54	0.55	0.55
query11	0.14	0.10	0.10
query12	0.15	0.12	0.11
query13	0.61	0.60	0.61
query14	2.70	2.77	2.75
query15	0.90	0.83	0.83
query16	0.38	0.38	0.37
query17	1.01	0.95	1.06
query18	0.23	0.22	0.22
query19	1.92	1.89	2.00
query20	0.01	0.01	0.01
query21	15.37	0.58	0.58
query22	2.67	2.30	2.08
query23	17.25	1.02	0.85
query24	2.90	1.87	0.47
query25	0.18	0.11	0.14
query26	0.46	0.14	0.13
query27	0.04	0.05	0.03
query28	10.46	1.09	1.08
query29	12.57	3.22	3.22
query30	0.25	0.06	0.06
query31	2.86	0.39	0.38
query32	3.25	0.46	0.46
query33	2.95	3.04	2.99
query34	16.76	4.53	4.50
query35	4.52	4.52	4.48
query36	0.64	0.51	0.49
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.04	0.02	0.03
query40	0.16	0.13	0.12
query41	0.07	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.4 s
Total hot run time: 32.42 s

@Vallishp
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 44928 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e7948ea10a3031c80fa9a2f0763c85a4d19a1f4, data reload: false

------ Round 1 ----------------------------------
q1	17584	7442	7328	7328
q2	2230	1172	1169	1169
q3	9957	1162	1145	1145
q4	10229	763	740	740
q5	7603	2683	2769	2683
q6	238	147	145	145
q7	957	641	600	600
q8	9378	2355	2366	2355
q9	6624	6396	6432	6396
q10	7061	2286	2341	2286
q11	476	266	263	263
q12	432	219	222	219
q13	17785	3026	3057	3026
q14	245	208	210	208
q15	579	536	518	518
q16	671	592	583	583
q17	976	518	491	491
q18	7207	6692	6746	6692
q19	1345	943	967	943
q20	2938	2724	2674	2674
q21	3992	3363	3123	3123
q22	1391	1370	1341	1341
Total cold run time: 109898 ms
Total hot run time: 44928 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7277	7262	7424	7262
q2	345	233	246	233
q3	3046	2963	3062	2963
q4	2063	1794	1776	1776
q5	5550	5697	5801	5697
q6	226	138	137	137
q7	2248	1830	1879	1830
q8	3351	3513	3530	3513
q9	8922	8958	8944	8944
q10	3590	3581	3591	3581
q11	589	516	510	510
q12	818	626	657	626
q13	11224	3254	3243	3243
q14	313	271	273	271
q15	586	524	532	524
q16	671	647	655	647
q17	1903	1658	1625	1625
q18	8271	7799	7830	7799
q19	1719	1440	1589	1440
q20	2132	1919	1884	1884
q21	5575	5455	5493	5455
q22	621	530	595	530
Total cold run time: 71040 ms
Total hot run time: 60490 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.28 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e7948ea10a3031c80fa9a2f0763c85a4d19a1f4, data reload: false

query1	0.04	0.05	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.41	0.44	0.41
query6	1.16	0.65	0.66
query7	0.02	0.02	0.02
query8	0.04	0.04	0.02
query9	0.58	0.49	0.50
query10	0.54	0.56	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.75	2.83	2.85
query15	0.92	0.85	0.84
query16	0.39	0.38	0.36
query17	1.06	1.06	1.06
query18	0.20	0.20	0.21
query19	1.94	1.88	2.00
query20	0.01	0.01	0.01
query21	15.36	0.58	0.59
query22	3.00	2.24	2.20
query23	16.99	0.92	0.77
query24	2.99	1.20	2.55
query25	0.32	0.22	0.05
query26	0.59	0.13	0.13
query27	0.04	0.05	0.04
query28	9.21	1.12	1.08
query29	12.55	3.23	3.22
query30	0.24	0.06	0.06
query31	2.87	0.38	0.37
query32	3.26	0.46	0.46
query33	2.95	3.01	3.04
query34	17.24	4.45	4.53
query35	4.57	4.52	4.49
query36	0.68	0.49	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.22 s
Total hot run time: 33.28 s

@Vallishp Vallishp force-pushed the dynamic_show_partition branch from 7e7948e to 39a3119 Compare November 20, 2024 06:05
@Vallishp
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 20, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@Vallishp
Copy link
Contributor Author

run performance

@Vallishp
Copy link
Contributor Author

run buildall

1 similar comment
@Vallishp
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39873 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7da76bbc7121f085c4a013917eb4bcac91e69d8b, data reload: false

------ Round 1 ----------------------------------
q1	17586	7477	7310	7310
q2	2036	171	159	159
q3	10591	1105	1252	1105
q4	10533	715	723	715
q5	7564	2711	2690	2690
q6	237	148	144	144
q7	971	623	588	588
q8	9236	1829	1888	1829
q9	6576	6377	6343	6343
q10	6995	2283	2338	2283
q11	465	257	259	257
q12	412	225	212	212
q13	17755	3033	3034	3033
q14	256	218	207	207
q15	570	530	531	530
q16	658	581	583	581
q17	982	647	468	468
q18	7269	6668	6706	6668
q19	1347	1082	987	987
q20	453	188	183	183
q21	3963	3288	3266	3266
q22	386	318	315	315
Total cold run time: 106841 ms
Total hot run time: 39873 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7251	7294	7278	7278
q2	337	236	230	230
q3	2883	2805	2955	2805
q4	2071	1812	1744	1744
q5	5754	5677	5634	5634
q6	228	148	141	141
q7	2255	1824	1838	1824
q8	3436	3548	3540	3540
q9	8743	8848	8870	8848
q10	3618	3547	3530	3530
q11	610	517	509	509
q12	851	613	636	613
q13	13638	3277	3213	3213
q14	307	278	271	271
q15	576	534	524	524
q16	689	672	652	652
q17	1834	1616	1612	1612
q18	8323	7829	7364	7364
q19	1749	1671	1481	1481
q20	2139	1872	1876	1872
q21	5633	5497	5476	5476
q22	658	590	575	575
Total cold run time: 73583 ms
Total hot run time: 59736 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196647 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7da76bbc7121f085c4a013917eb4bcac91e69d8b, data reload: false

query1	1236	963	971	963
query2	6240	2212	2098	2098
query3	10817	3928	3941	3928
query4	67443	28850	23663	23663
query5	4974	458	429	429
query6	405	185	182	182
query7	5539	300	290	290
query8	322	227	224	224
query9	8654	2687	2672	2672
query10	454	239	241	239
query11	17462	15474	15834	15474
query12	156	101	100	100
query13	1488	444	403	403
query14	10566	7651	7614	7614
query15	208	189	185	185
query16	7207	449	454	449
query17	1048	566	558	558
query18	1819	295	278	278
query19	195	145	144	144
query20	114	114	113	113
query21	198	96	103	96
query22	4583	4446	4472	4446
query23	34637	34413	33986	33986
query24	5348	2438	2466	2438
query25	505	415	407	407
query26	650	150	157	150
query27	1688	284	284	284
query28	4129	2453	2405	2405
query29	710	430	427	427
query30	225	150	148	148
query31	1008	800	819	800
query32	95	52	57	52
query33	434	273	274	273
query34	919	517	508	508
query35	802	705	698	698
query36	1103	947	967	947
query37	115	70	79	70
query38	4310	4188	4227	4188
query39	1475	1422	1412	1412
query40	198	99	98	98
query41	45	43	45	43
query42	107	99	100	99
query43	560	519	516	516
query44	1136	820	811	811
query45	183	172	168	168
query46	1117	666	683	666
query47	1950	1847	1814	1814
query48	430	312	312	312
query49	710	372	384	372
query50	786	388	416	388
query51	7401	7193	7062	7062
query52	98	88	91	88
query53	257	176	180	176
query54	522	398	385	385
query55	78	79	77	77
query56	263	237	231	231
query57	1312	1173	1140	1140
query58	223	210	206	206
query59	3420	3225	3203	3203
query60	267	247	246	246
query61	110	118	102	102
query62	801	673	692	673
query63	220	198	192	192
query64	1349	667	626	626
query65	3293	3268	3232	3232
query66	763	308	319	308
query67	15973	15772	15660	15660
query68	3658	569	549	549
query69	428	253	247	247
query70	1211	1274	1165	1165
query71	359	247	249	247
query72	6391	4102	3991	3991
query73	752	358	366	358
query74	10190	8969	8995	8969
query75	3406	2669	2690	2669
query76	1946	1001	1070	1001
query77	510	274	290	274
query78	10609	9498	9447	9447
query79	1325	610	600	600
query80	871	433	517	433
query81	498	235	235	235
query82	1337	117	114	114
query83	187	154	160	154
query84	286	69	71	69
query85	850	294	297	294
query86	328	299	310	299
query87	4717	4639	4504	4504
query88	3136	2209	2194	2194
query89	416	352	297	297
query90	2033	187	180	180
query91	134	104	104	104
query92	65	56	50	50
query93	1646	530	538	530
query94	774	286	290	286
query95	342	254	251	251
query96	621	281	281	281
query97	2851	2710	2720	2710
query98	215	209	198	198
query99	1932	1312	1318	1312
Total cold run time: 316877 ms
Total hot run time: 196647 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.34 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7da76bbc7121f085c4a013917eb4bcac91e69d8b, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.04	0.03
query3	0.24	0.08	0.07
query4	1.62	0.10	0.10
query5	0.43	0.41	0.41
query6	1.16	0.65	0.67
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.50	0.49
query10	0.57	0.54	0.56
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.60	0.60	0.59
query14	2.82	2.74	2.72
query15	0.90	0.83	0.82
query16	0.38	0.37	0.37
query17	0.99	1.02	0.99
query18	0.20	0.19	0.20
query19	1.95	1.86	1.97
query20	0.02	0.02	0.01
query21	15.38	0.58	0.58
query22	2.78	1.95	1.79
query23	17.13	0.90	0.74
query24	3.27	1.78	0.95
query25	0.36	0.14	0.10
query26	0.43	0.14	0.13
query27	0.05	0.04	0.04
query28	9.81	1.10	1.08
query29	12.56	3.21	3.20
query30	0.25	0.06	0.05
query31	2.88	0.38	0.38
query32	3.27	0.48	0.47
query33	2.97	3.01	3.05
query34	17.09	4.42	4.42
query35	4.49	4.51	4.50
query36	0.68	0.49	0.48
query37	0.09	0.06	0.05
query38	0.05	0.03	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.81 s
Total hot run time: 32.34 s

@Vallishp
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40100 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6434dcfb5ad23d7463dd4fbb9caa21cd0bfb4076, data reload: false

------ Round 1 ----------------------------------
q1	17591	7520	7348	7348
q2	2042	177	181	177
q3	10616	1137	1138	1137
q4	10257	738	726	726
q5	7605	2728	2727	2727
q6	241	156	148	148
q7	1014	633	609	609
q8	9223	1813	1927	1813
q9	6500	6480	6365	6365
q10	6964	2275	2312	2275
q11	464	265	257	257
q12	425	222	212	212
q13	17758	3032	3066	3032
q14	236	211	210	210
q15	570	522	525	522
q16	639	579	580	579
q17	980	626	566	566
q18	7352	6778	6704	6704
q19	1339	1059	963	963
q20	475	177	178	177
q21	3921	3236	3362	3236
q22	386	317	320	317
Total cold run time: 106598 ms
Total hot run time: 40100 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7275	7239	7274	7239
q2	332	235	232	232
q3	2948	2806	2776	2776
q4	2002	1693	1718	1693
q5	5384	5418	5425	5418
q6	217	139	137	137
q7	2121	1734	1708	1708
q8	3254	3432	3437	3432
q9	8545	8519	8561	8519
q10	3513	3460	3436	3436
q11	599	504	497	497
q12	823	634	586	586
q13	12395	3046	3050	3046
q14	307	267	270	267
q15	560	512	526	512
q16	684	647	665	647
q17	1794	1564	1569	1564
q18	7874	7369	7449	7369
q19	1656	1533	1583	1533
q20	2073	1840	1830	1830
q21	5256	5201	5177	5177
q22	650	562	566	562
Total cold run time: 70262 ms
Total hot run time: 58180 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191642 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6434dcfb5ad23d7463dd4fbb9caa21cd0bfb4076, data reload: false

query1	987	384	378	378
query2	6525	2275	2196	2196
query3	6697	214	219	214
query4	34232	23496	23441	23441
query5	4320	456	447	447
query6	279	196	187	187
query7	4637	302	296	296
query8	298	231	226	226
query9	9490	2668	2647	2647
query10	501	245	260	245
query11	18008	15324	15194	15194
query12	146	101	102	101
query13	1648	404	391	391
query14	10106	7094	6944	6944
query15	254	179	188	179
query16	8130	475	492	475
query17	1642	597	556	556
query18	2150	296	294	294
query19	347	158	146	146
query20	129	110	113	110
query21	209	107	104	104
query22	4793	4640	4496	4496
query23	35075	34072	34214	34072
query24	11337	2456	2534	2456
query25	689	403	470	403
query26	1788	146	151	146
query27	2898	295	273	273
query28	8163	2461	2433	2433
query29	1004	396	411	396
query30	307	149	151	149
query31	1014	793	817	793
query32	96	59	60	59
query33	769	292	340	292
query34	929	501	514	501
query35	852	717	721	717
query36	1071	972	940	940
query37	186	75	78	75
query38	4287	4249	4271	4249
query39	1466	1432	1426	1426
query40	284	103	97	97
query41	51	44	43	43
query42	109	95	94	94
query43	555	519	503	503
query44	1275	809	799	799
query45	182	165	165	165
query46	1149	704	704	704
query47	1954	1872	1847	1847
query48	417	326	312	312
query49	1302	379	379	379
query50	796	398	387	387
query51	7372	7144	7121	7121
query52	97	90	88	88
query53	266	179	176	176
query54	1275	406	405	405
query55	84	74	78	74
query56	262	259	232	232
query57	1291	1187	1161	1161
query58	238	209	218	209
query59	3309	3248	3300	3248
query60	287	243	239	239
query61	110	106	104	104
query62	877	661	681	661
query63	213	195	180	180
query64	5156	660	633	633
query65	3314	3221	3230	3221
query66	1446	317	310	310
query67	16066	15964	15616	15616
query68	5054	549	543	543
query69	446	304	269	269
query70	1225	1146	1122	1122
query71	333	252	242	242
query72	6520	4029	4049	4029
query73	760	357	355	355
query74	10504	9027	9039	9027
query75	3420	2681	2630	2630
query76	3006	1198	1030	1030
query77	419	271	284	271
query78	10510	9457	9416	9416
query79	2383	581	615	581
query80	1062	428	427	427
query81	549	242	230	230
query82	646	116	114	114
query83	248	145	148	145
query84	231	78	77	77
query85	1872	291	291	291
query86	499	297	266	266
query87	4820	4577	4652	4577
query88	4171	2174	2147	2147
query89	407	298	306	298
query90	2088	202	184	184
query91	135	104	101	101
query92	57	50	50	50
query93	1983	543	545	543
query94	936	288	303	288
query95	351	248	246	246
query96	634	280	274	274
query97	2903	2728	2713	2713
query98	209	208	195	195
query99	1532	1331	1306	1306
Total cold run time: 307109 ms
Total hot run time: 191642 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.23 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6434dcfb5ad23d7463dd4fbb9caa21cd0bfb4076, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.11
query5	0.43	0.42	0.41
query6	1.16	0.66	0.66
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.51
query10	0.55	0.55	0.57
query11	0.14	0.11	0.10
query12	0.14	0.11	0.12
query13	0.62	0.60	0.61
query14	2.72	2.73	2.71
query15	0.89	0.82	0.83
query16	0.38	0.36	0.38
query17	0.97	1.02	1.01
query18	0.23	0.22	0.21
query19	1.94	1.84	1.98
query20	0.01	0.01	0.01
query21	15.35	0.57	0.59
query22	2.43	2.35	2.28
query23	16.94	1.01	0.79
query24	2.50	1.60	1.24
query25	0.16	0.19	0.20
query26	0.55	0.13	0.13
query27	0.05	0.04	0.03
query28	10.43	1.12	1.07
query29	12.52	3.19	3.17
query30	0.25	0.07	0.07
query31	2.87	0.38	0.38
query32	3.26	0.47	0.47
query33	3.02	3.12	3.00
query34	17.17	4.47	4.44
query35	4.48	4.45	4.49
query36	0.66	0.48	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.02	0.02
Total cold run time: 105.88 s
Total hot run time: 33.23 s

@starocean999 starocean999 merged commit ede62c6 into apache:master Nov 21, 2024
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LiBinfeng-01 LiBinfeng-01 LiBinfeng-01 approved these changes

@starocean999 starocean999 starocean999 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Enhancement] (nereids)implement showDynamicPartitionCommand in nereids
4 participants
@Vallishp @doris-robot @starocean999 @LiBinfeng-01