Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] (nereids)implement showColumnStatsCommand in nereids #45439

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

msridhar78
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #42717

Related PR: #xxx

Problem Summary:
[Enhancement] (nereids)implement showColumnStatsCommand in nereids

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@msridhar78
Copy link
Contributor Author

run buildall

1 similar comment
@msridhar78
Copy link
Contributor Author

run buildall

return new ShowColumnStatsCommand(new TableNameInfo(tableNameParts), colNames, partitionNames, isCached);
}

private PartitionNames parsePartitionSpec(PartitionSpecContext ctx) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a visitPartitionSpec method already, please use it instead of creating a new one. Take bellow as example:

Pair<Boolean, List<String>> partitionSpec = visitPartitionSpec(ctx.partitions);
PartitionNamesInfo partitionNames = new PartitionNamesInfo(partitionSpec.first, partitionSpec.second);


private final TableNameInfo tableName;
private final List<String> columnNames;
private final PartitionNames partitionNames;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private final PartitionNames partitionNames;
private final PartitionNamesInfo partitionNames;

use PartitionNamesInfo instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done - pls review

analyze();
}

public void analyze() throws AnalysisException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use PartitionNamesInfo, PartitionNames don't need to be changed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done - please review

@msridhar78 msridhar78 force-pushed the show_column_stats branch 2 times, most recently from c52a5c5 to 9367eb9 Compare December 20, 2024 09:38
@msridhar78
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39896 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9367eb9f269a4b534dba988e9fcfb639429ba3ef, data reload: false

------ Round 1 ----------------------------------
q1	17611	7464	7311	7311
q2	2041	186	178	178
q3	10616	1153	1151	1151
q4	10412	765	716	716
q5	7613	2771	2704	2704
q6	245	150	151	150
q7	985	622	633	622
q8	9248	1924	1948	1924
q9	6626	6441	6458	6441
q10	7021	2327	2349	2327
q11	484	253	257	253
q12	419	226	222	222
q13	17810	2896	2934	2896
q14	241	208	229	208
q15	549	508	507	507
q16	685	594	577	577
q17	976	537	594	537
q18	7281	6718	6589	6589
q19	1378	964	918	918
q20	475	185	180	180
q21	4065	3179	3277	3179
q22	380	306	306	306
Total cold run time: 107161 ms
Total hot run time: 39896 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7240	7218	7233	7218
q2	330	231	226	226
q3	2901	2780	3019	2780
q4	2133	1902	1801	1801
q5	5566	5696	5640	5640
q6	222	141	143	141
q7	2280	1767	1808	1767
q8	3382	3578	3502	3502
q9	9076	9099	9017	9017
q10	3610	3521	3529	3521
q11	598	497	501	497
q12	819	631	585	585
q13	12932	3101	3033	3033
q14	313	263	269	263
q15	562	508	504	504
q16	697	659	680	659
q17	1876	1592	1557	1557
q18	7888	7487	7478	7478
q19	1713	1580	1551	1551
q20	2122	1792	1784	1784
q21	5447	5269	5414	5269
q22	640	609	593	593
Total cold run time: 72347 ms
Total hot run time: 59386 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189738 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9367eb9f269a4b534dba988e9fcfb639429ba3ef, data reload: false

query1	971	387	377	377
query2	6542	2312	2415	2312
query3	6710	220	210	210
query4	33977	23421	23509	23421
query5	4327	476	451	451
query6	291	183	195	183
query7	4612	312	317	312
query8	312	239	238	238
query9	9371	2724	2719	2719
query10	478	257	257	257
query11	17973	15069	15071	15069
query12	161	104	106	104
query13	1684	438	452	438
query14	11203	7694	6983	6983
query15	319	188	188	188
query16	8258	447	453	447
query17	2207	593	577	577
query18	2182	294	298	294
query19	370	158	155	155
query20	123	128	112	112
query21	213	101	104	101
query22	4373	4362	4168	4168
query23	37476	33729	33442	33442
query24	11281	2530	2462	2462
query25	562	377	385	377
query26	1200	152	151	151
query27	2773	333	324	324
query28	7804	2404	2439	2404
query29	666	405	412	405
query30	300	145	150	145
query31	1046	806	802	802
query32	101	58	65	58
query33	765	336	290	290
query34	950	523	510	510
query35	889	739	749	739
query36	1121	962	941	941
query37	216	74	79	74
query38	4276	4149	4180	4149
query39	1517	1454	1421	1421
query40	282	109	100	100
query41	47	46	43	43
query42	111	105	102	102
query43	519	485	502	485
query44	1233	802	790	790
query45	180	165	170	165
query46	1153	675	700	675
query47	1970	1827	1804	1804
query48	418	314	326	314
query49	1278	377	392	377
query50	820	391	396	391
query51	7124	6955	6990	6955
query52	103	94	93	93
query53	264	189	183	183
query54	1224	405	421	405
query55	81	78	81	78
query56	267	253	253	253
query57	1248	1112	1144	1112
query58	243	211	232	211
query59	3254	2994	2824	2824
query60	265	246	236	236
query61	106	111	144	111
query62	879	678	666	666
query63	227	200	200	200
query64	4973	672	625	625
query65	3291	3257	3219	3219
query66	1392	316	329	316
query67	15801	15623	15540	15540
query68	5884	554	553	553
query69	413	246	260	246
query70	1139	1150	1140	1140
query71	445	264	250	250
query72	6301	4093	4094	4093
query73	790	362	362	362
query74	10396	8807	8883	8807
query75	3456	2649	2671	2649
query76	3557	1079	1174	1079
query77	553	273	288	273
query78	10219	9297	9388	9297
query79	1773	600	597	597
query80	1082	435	444	435
query81	534	237	233	233
query82	938	118	126	118
query83	243	149	145	145
query84	234	71	74	71
query85	1396	313	312	312
query86	371	300	310	300
query87	4821	4342	4428	4342
query88	3711	2225	2225	2225
query89	420	286	304	286
query90	2117	189	189	189
query91	144	104	105	104
query92	64	53	53	53
query93	1500	539	544	539
query94	1102	293	304	293
query95	365	251	255	251
query96	622	279	286	279
query97	2856	2627	2711	2627
query98	225	202	198	198
query99	1571	1309	1337	1309
Total cold run time: 307595 ms
Total hot run time: 189738 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.98 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9367eb9f269a4b534dba988e9fcfb639429ba3ef, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.11	0.11
query5	0.41	0.39	0.40
query6	1.17	0.66	0.67
query7	0.02	0.02	0.02
query8	0.04	0.02	0.03
query9	0.58	0.50	0.50
query10	0.55	0.60	0.57
query11	0.15	0.12	0.11
query12	0.13	0.11	0.11
query13	0.61	0.62	0.60
query14	2.88	2.84	2.89
query15	0.92	0.83	0.83
query16	0.38	0.38	0.38
query17	1.06	1.06	1.01
query18	0.22	0.21	0.22
query19	1.99	1.78	1.97
query20	0.01	0.01	0.02
query21	15.36	0.61	0.57
query22	2.44	2.31	1.74
query23	16.84	1.00	0.96
query24	3.00	1.05	1.20
query25	0.31	0.08	0.07
query26	0.41	0.14	0.15
query27	0.06	0.04	0.05
query28	10.50	1.12	1.07
query29	12.57	3.22	3.21
query30	0.27	0.08	0.08
query31	2.83	0.41	0.39
query32	3.23	0.46	0.46
query33	3.12	3.14	3.11
query34	17.13	4.47	4.48
query35	4.56	4.55	4.53
query36	0.67	0.48	0.48
query37	0.10	0.06	0.07
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.18	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.9 s
Total hot run time: 32.98 s

@msridhar78
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39693 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c3e07ca121250fe71eed1b6dfc4fbedf8ed71352, data reload: false

------ Round 1 ----------------------------------
q1	17632	7476	7311	7311
q2	2055	178	167	167
q3	10706	1107	1213	1107
q4	10494	742	745	742
q5	7604	2702	2776	2702
q6	245	149	145	145
q7	986	616	609	609
q8	9246	1867	1980	1867
q9	6662	6468	6496	6468
q10	7058	2303	2287	2287
q11	463	265	268	265
q12	434	222	222	222
q13	17767	2911	2947	2911
q14	248	206	209	206
q15	556	502	489	489
q16	664	583	582	582
q17	1001	501	493	493
q18	7254	6736	6672	6672
q19	1348	1047	887	887
q20	478	180	183	180
q21	3987	3177	3070	3070
q22	392	323	311	311
Total cold run time: 107280 ms
Total hot run time: 39693 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7220	7237	7190	7190
q2	327	233	237	233
q3	2914	2853	2941	2853
q4	2069	1869	1765	1765
q5	5680	5703	5649	5649
q6	227	145	144	144
q7	2213	1815	1805	1805
q8	3399	3564	3490	3490
q9	8925	8980	9061	8980
q10	3603	3556	3560	3556
q11	617	502	519	502
q12	816	606	602	602
q13	12677	3185	3038	3038
q14	325	294	269	269
q15	550	505	507	505
q16	671	641	653	641
q17	1875	1669	1652	1652
q18	8399	7728	7830	7728
q19	1768	1646	1447	1447
q20	2062	1847	1959	1847
q21	5598	5509	5364	5364
q22	669	576	583	576
Total cold run time: 72604 ms
Total hot run time: 59836 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195779 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c3e07ca121250fe71eed1b6dfc4fbedf8ed71352, data reload: false

query1	1331	996	891	891
query2	6219	2396	2351	2351
query3	10989	4662	4780	4662
query4	32939	23470	23316	23316
query5	3740	481	465	465
query6	273	232	214	214
query7	4001	308	296	296
query8	285	236	215	215
query9	9519	2690	2687	2687
query10	456	252	239	239
query11	17739	15071	15088	15071
query12	148	102	101	101
query13	1580	430	433	430
query14	9931	6689	7226	6689
query15	281	197	198	197
query16	8217	468	472	468
query17	1707	602	583	583
query18	2173	316	303	303
query19	372	155	172	155
query20	125	118	124	118
query21	220	104	107	104
query22	4855	4698	4609	4609
query23	34652	33467	33485	33467
query24	10593	2434	2472	2434
query25	609	402	392	392
query26	1247	152	150	150
query27	2350	336	336	336
query28	7515	2506	2494	2494
query29	855	427	431	427
query30	246	149	156	149
query31	1051	831	828	828
query32	117	57	58	57
query33	764	288	289	288
query34	928	525	506	506
query35	919	769	774	769
query36	1102	950	946	946
query37	139	74	71	71
query38	4186	4069	4109	4069
query39	1491	1468	1466	1466
query40	211	105	101	101
query41	45	43	51	43
query42	124	105	106	105
query43	541	537	506	506
query44	1299	841	844	841
query45	188	167	170	167
query46	1184	710	738	710
query47	2017	1942	1962	1942
query48	417	337	336	336
query49	930	393	380	380
query50	830	402	414	402
query51	7312	7144	7166	7144
query52	105	91	92	91
query53	260	185	185	185
query54	1200	432	418	418
query55	86	83	81	81
query56	274	254	237	237
query57	1269	1172	1125	1125
query58	234	222	265	222
query59	3493	3105	3129	3105
query60	289	240	239	239
query61	106	114	106	106
query62	917	711	694	694
query63	221	197	195	195
query64	4110	658	631	631
query65	3241	3184	3184	3184
query66	829	295	306	295
query67	15884	15711	15583	15583
query68	4642	560	556	556
query69	452	254	307	254
query70	1185	1140	1108	1108
query71	405	243	254	243
query72	6405	4087	4135	4087
query73	765	373	371	371
query74	10336	8929	8929	8929
query75	3425	2643	2623	2623
query76	2701	1136	1073	1073
query77	386	269	268	268
query78	10265	9333	9501	9333
query79	1421	610	608	608
query80	1034	428	421	421
query81	543	228	239	228
query82	916	114	119	114
query83	236	146	141	141
query84	231	77	75	75
query85	1443	296	335	296
query86	398	304	297	297
query87	4531	4331	4426	4331
query88	3378	2214	2190	2190
query89	414	286	291	286
query90	1945	186	184	184
query91	148	104	109	104
query92	62	54	52	52
query93	1658	551	541	541
query94	903	281	290	281
query95	352	254	249	249
query96	626	277	279	277
query97	2871	2727	2659	2659
query98	216	199	197	197
query99	1606	1322	1327	1322
Total cold run time: 299379 ms
Total hot run time: 195779 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.17 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c3e07ca121250fe71eed1b6dfc4fbedf8ed71352, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.10
query5	0.43	0.43	0.41
query6	1.15	0.65	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.59	0.51	0.50
query10	0.56	0.57	0.57
query11	0.15	0.10	0.10
query12	0.14	0.11	0.10
query13	0.61	0.59	0.61
query14	2.82	2.73	2.69
query15	0.90	0.82	0.82
query16	0.38	0.37	0.39
query17	1.07	1.07	0.99
query18	0.23	0.20	0.20
query19	1.98	1.89	1.99
query20	0.02	0.01	0.01
query21	15.36	0.61	0.59
query22	2.60	1.88	2.38
query23	16.95	1.08	0.86
query24	2.99	1.70	1.49
query25	0.32	0.16	0.07
query26	0.57	0.15	0.13
query27	0.04	0.05	0.04
query28	9.69	1.11	1.07
query29	12.62	3.22	3.21
query30	0.25	0.07	0.07
query31	2.85	0.39	0.37
query32	3.25	0.47	0.46
query33	3.04	3.11	3.08
query34	17.00	4.45	4.46
query35	4.53	4.44	4.45
query36	0.68	0.49	0.50
query37	0.10	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.03	0.03
query40	0.17	0.12	0.13
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.04
Total cold run time: 106.31 s
Total hot run time: 33.17 s

@msridhar78
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39966 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c3e07ca121250fe71eed1b6dfc4fbedf8ed71352, data reload: false

------ Round 1 ----------------------------------
q1	17640	7456	7240	7240
q2	2043	175	173	173
q3	10644	1082	1155	1082
q4	10561	756	741	741
q5	7605	2647	2683	2647
q6	241	148	146	146
q7	984	614	604	604
q8	9476	1839	1859	1839
q9	6654	6409	6336	6336
q10	6993	2253	2301	2253
q11	469	277	273	273
q12	428	224	217	217
q13	17774	2961	2875	2875
q14	233	211	214	211
q15	567	515	509	509
q16	653	588	586	586
q17	973	517	580	517
q18	7311	6977	6985	6977
q19	1348	996	977	977
q20	459	189	184	184
q21	4015	3269	3301	3269
q22	383	316	310	310
Total cold run time: 107454 ms
Total hot run time: 39966 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7205	7189	7153	7153
q2	324	240	228	228
q3	2872	2795	2919	2795
q4	2016	1860	1766	1766
q5	5639	5620	5656	5620
q6	223	141	148	141
q7	2221	1811	1856	1811
q8	3360	3514	3497	3497
q9	8913	8983	8928	8928
q10	3597	3509	3522	3509
q11	599	515	511	511
q12	811	590	614	590
q13	13324	3169	3185	3169
q14	316	287	278	278
q15	550	511	509	509
q16	703	645	634	634
q17	1817	1639	1611	1611
q18	8199	7901	7688	7688
q19	1707	1540	1361	1361
q20	2139	1916	1927	1916
q21	5493	5330	5447	5330
q22	654	582	591	582
Total cold run time: 72682 ms
Total hot run time: 59627 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197126 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c3e07ca121250fe71eed1b6dfc4fbedf8ed71352, data reload: false

query1	1298	947	907	907
query2	6244	2356	2414	2356
query3	10971	4571	4673	4571
query4	33485	23384	23330	23330
query5	4979	457	465	457
query6	291	208	187	187
query7	4009	313	307	307
query8	287	226	220	220
query9	9298	2727	2734	2727
query10	496	232	245	232
query11	17804	15202	15045	15045
query12	154	102	105	102
query13	1555	414	412	412
query14	9670	7555	7761	7555
query15	300	213	202	202
query16	8178	463	467	463
query17	1772	625	610	610
query18	2201	324	337	324
query19	368	170	158	158
query20	125	129	121	121
query21	211	110	108	108
query22	4938	4341	4423	4341
query23	35083	33797	34002	33797
query24	10559	2529	2491	2491
query25	630	388	391	388
query26	1193	156	164	156
query27	2554	341	347	341
query28	7870	2520	2502	2502
query29	885	416	419	416
query30	233	158	158	158
query31	1007	828	859	828
query32	100	56	56	56
query33	743	300	296	296
query34	939	536	536	536
query35	863	772	750	750
query36	1120	968	979	968
query37	140	71	77	71
query38	4212	4093	4134	4093
query39	1517	1455	1468	1455
query40	207	109	103	103
query41	45	41	45	41
query42	118	108	105	105
query43	527	504	522	504
query44	1293	845	820	820
query45	188	172	173	172
query46	1174	748	752	748
query47	2116	1924	1928	1924
query48	432	338	322	322
query49	952	403	391	391
query50	828	408	407	407
query51	7301	7181	7292	7181
query52	109	91	98	91
query53	264	194	197	194
query54	1107	420	415	415
query55	82	77	81	77
query56	263	256	247	247
query57	1295	1173	1150	1150
query58	265	225	231	225
query59	3514	3156	3189	3156
query60	270	243	251	243
query61	120	109	107	107
query62	857	685	704	685
query63	222	189	196	189
query64	3895	707	646	646
query65	3262	3253	3276	3253
query66	704	311	320	311
query67	16405	15539	15615	15539
query68	6488	560	555	555
query69	496	254	248	248
query70	1218	1130	1148	1130
query71	538	273	249	249
query72	6545	4050	4106	4050
query73	803	363	380	363
query74	9935	8839	9033	8839
query75	4550	2664	2662	2662
query76	4327	1091	1112	1091
query77	796	281	271	271
query78	10295	9418	9428	9418
query79	1975	611	612	611
query80	850	426	430	426
query81	509	225	226	225
query82	712	120	123	120
query83	178	143	147	143
query84	253	72	70	70
query85	1256	301	296	296
query86	395	308	274	274
query87	4522	4425	4551	4425
query88	3696	2250	2237	2237
query89	428	301	286	286
query90	1955	189	188	188
query91	134	140	105	105
query92	61	52	53	52
query93	1982	553	556	553
query94	699	299	289	289
query95	352	250	248	248
query96	610	285	277	277
query97	2832	2700	2652	2652
query98	214	202	198	198
query99	1634	1292	1323	1292
Total cold run time: 307335 ms
Total hot run time: 197126 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.12 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c3e07ca121250fe71eed1b6dfc4fbedf8ed71352, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.60	0.10	0.10
query5	0.42	0.42	0.41
query6	1.15	0.66	0.65
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.49	0.54
query10	0.57	0.59	0.55
query11	0.15	0.10	0.10
query12	0.13	0.11	0.10
query13	0.61	0.61	0.60
query14	2.84	2.81	2.71
query15	0.88	0.82	0.84
query16	0.39	0.39	0.38
query17	1.05	0.96	1.04
query18	0.22	0.21	0.21
query19	1.91	1.84	2.04
query20	0.02	0.01	0.01
query21	15.36	0.58	0.59
query22	2.41	2.47	1.64
query23	17.03	0.92	0.78
query24	2.68	0.71	1.95
query25	0.26	0.17	0.10
query26	0.46	0.13	0.13
query27	0.05	0.04	0.04
query28	10.40	1.10	1.07
query29	12.57	3.30	3.26
query30	0.25	0.07	0.06
query31	2.85	0.39	0.39
query32	3.23	0.47	0.46
query33	3.06	3.11	3.12
query34	16.83	4.47	4.45
query35	4.45	4.45	4.44
query36	0.70	0.48	0.50
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.18	0.13	0.12
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.02 s
Total hot run time: 32.12 s

@msridhar78
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32508 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 855f1c2397fdde32741126821a6e0243b9f9327d, data reload: false

------ Round 1 ----------------------------------
q1	17606	6109	6029	6029
q2	2036	291	162	162
q3	10455	1214	732	732
q4	10201	861	428	428
q5	7525	2171	1977	1977
q6	205	175	144	144
q7	882	748	596	596
q8	9227	1358	1161	1161
q9	5090	4988	4891	4891
q10	6754	2294	1872	1872
q11	478	281	251	251
q12	346	370	227	227
q13	17758	3609	2959	2959
q14	235	230	216	216
q15	556	507	489	489
q16	643	629	578	578
q17	569	851	323	323
q18	6901	6547	6447	6447
q19	2919	1004	563	563
q20	311	328	195	195
q21	2834	2171	1955	1955
q22	369	330	313	313
Total cold run time: 103900 ms
Total hot run time: 32508 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6341	6299	6339	6299
q2	232	336	226	226
q3	2294	2700	2339	2339
q4	1385	1825	1371	1371
q5	4356	4766	4881	4766
q6	194	184	149	149
q7	2072	1996	1833	1833
q8	2673	2808	2730	2730
q9	7328	7278	7245	7245
q10	3113	3381	2964	2964
q11	573	532	514	514
q12	683	803	725	725
q13	3596	4082	3301	3301
q14	299	338	318	318
q15	573	533	524	524
q16	675	732	665	665
q17	1265	1785	1309	1309
q18	8099	7384	7426	7384
q19	1790	1099	1148	1099
q20	1969	2040	1996	1996
q21	5788	5312	5044	5044
q22	597	605	568	568
Total cold run time: 55895 ms
Total hot run time: 53369 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196811 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 855f1c2397fdde32741126821a6e0243b9f9327d, data reload: false

query1	1311	986	907	907
query2	6497	2395	2270	2270
query3	10984	4831	4694	4694
query4	33254	23748	23359	23359
query5	4304	613	466	466
query6	294	191	188	188
query7	3989	480	306	306
query8	303	251	256	251
query9	9513	2718	2724	2718
query10	482	305	241	241
query11	17769	15384	15209	15209
query12	160	105	100	100
query13	1571	523	393	393
query14	10247	7638	7463	7463
query15	241	203	191	191
query16	8157	585	450	450
query17	1542	746	590	590
query18	2119	418	317	317
query19	205	185	158	158
query20	118	119	112	112
query21	217	123	112	112
query22	4654	4582	4338	4338
query23	34532	33899	33587	33587
query24	6519	2335	2279	2279
query25	471	461	411	411
query26	819	271	153	153
query27	2031	467	332	332
query28	5862	2484	2443	2443
query29	620	558	422	422
query30	207	184	149	149
query31	965	913	842	842
query32	72	59	54	54
query33	466	378	308	308
query34	780	878	513	513
query35	808	826	794	794
query36	1026	1090	974	974
query37	115	105	75	75
query38	4158	4220	4257	4220
query39	1517	1490	1458	1458
query40	205	120	104	104
query41	46	44	44	44
query42	116	102	98	98
query43	504	532	493	493
query44	1347	826	818	818
query45	189	179	175	175
query46	894	1073	672	672
query47	1994	2024	1951	1951
query48	406	440	323	323
query49	722	480	396	396
query50	620	685	393	393
query51	7294	7372	7350	7350
query52	102	102	92	92
query53	230	252	182	182
query54	482	496	429	429
query55	87	82	79	79
query56	296	271	255	255
query57	1240	1231	1164	1164
query58	234	223	224	223
query59	3162	3281	3088	3088
query60	273	258	257	257
query61	106	111	109	109
query62	866	823	760	760
query63	219	189	188	188
query64	3176	1055	658	658
query65	3326	3366	3237	3237
query66	777	413	305	305
query67	16464	16040	15557	15557
query68	7834	760	512	512
query69	491	315	263	263
query70	1198	1146	1154	1146
query71	448	294	261	261
query72	6084	3868	3876	3868
query73	650	759	369	369
query74	9809	9173	8865	8865
query75	4024	3159	2646	2646
query76	3558	1194	786	786
query77	761	371	272	272
query78	10153	10154	9403	9403
query79	5011	895	581	581
query80	750	512	528	512
query81	507	278	232	232
query82	643	146	116	116
query83	190	156	145	145
query84	275	90	81	81
query85	783	365	299	299
query86	381	310	318	310
query87	4638	4426	4440	4426
query88	4706	2189	2194	2189
query89	451	331	292	292
query90	1973	181	181	181
query91	129	135	114	114
query92	66	53	51	51
query93	3378	889	515	515
query94	647	396	278	278
query95	331	266	244	244
query96	481	613	285	285
query97	2740	2802	2696	2696
query98	222	204	193	193
query99	1685	1552	1433	1433
Total cold run time: 299273 ms
Total hot run time: 196811 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 855f1c2397fdde32741126821a6e0243b9f9327d, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.03
query3	0.24	0.07	0.07
query4	1.62	0.11	0.11
query5	0.42	0.41	0.41
query6	1.14	0.65	0.64
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.57	0.51	0.50
query10	0.55	0.57	0.55
query11	0.15	0.10	0.10
query12	0.13	0.11	0.12
query13	0.61	0.60	0.59
query14	2.71	2.76	2.90
query15	0.88	0.81	0.82
query16	0.38	0.38	0.38
query17	1.00	1.02	1.04
query18	0.23	0.21	0.20
query19	1.86	1.82	2.04
query20	0.01	0.02	0.01
query21	15.37	0.96	0.59
query22	0.75	0.80	0.65
query23	15.32	1.41	0.56
query24	3.02	0.99	1.37
query25	0.15	0.15	0.14
query26	0.33	0.14	0.14
query27	0.06	0.06	0.04
query28	13.89	1.52	1.05
query29	12.55	4.04	3.31
query30	0.24	0.08	0.06
query31	2.84	0.59	0.39
query32	3.24	0.55	0.46
query33	3.16	3.02	3.06
query34	16.69	5.09	4.45
query35	4.50	4.48	4.46
query36	0.64	0.49	0.47
query37	0.10	0.06	0.05
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.91 s
Total hot run time: 31.28 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement] (nereids)implement showColumnStatsCommand in nereids
4 participants