Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve](ut) add ut for column_array #45827

Open
wants to merge 26 commits into
base: master
Choose a base branch
from

Conversation

amorynan
Copy link
Contributor

@amorynan amorynan commented Dec 23, 2024

What problem does this PR solve?

this pr do :
0/ add ut cases for column_array
1/ delete convert_to_full_column_if_const and add convert_to_full_column_if_const in create action
2/ improve replicate function for column_array according ut test result, eg before we deal with decimal type with replicate_generic is much slower than now we deal with filter_number
before impl:
截屏2024-12-30 10 55 33
now impl:
截屏2024-12-30 10 55 24

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 23, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@amorynan
Copy link
Contributor Author

run buildall

@amorynan
Copy link
Contributor Author

run buildall

@amorynan
Copy link
Contributor Author

run buildall

@amorynan
Copy link
Contributor Author

run compile

@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32344 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c96e646315cb1105a1cbc12f0c528536e2e54860, data reload: false

------ Round 1 ----------------------------------
q1	17830	6171	6024	6024
q2	2046	302	172	172
q3	10426	1305	715	715
q4	10221	873	435	435
q5	7539	2194	1924	1924
q6	203	182	145	145
q7	899	762	614	614
q8	9251	1346	1120	1120
q9	5097	4910	5074	4910
q10	6744	2331	1851	1851
q11	472	269	265	265
q12	346	357	213	213
q13	17768	3523	2942	2942
q14	231	237	214	214
q15	569	496	511	496
q16	625	618	591	591
q17	563	846	320	320
q18	7188	6485	6416	6416
q19	2633	969	537	537
q20	290	309	185	185
q21	2795	2160	1952	1952
q22	379	325	303	303
Total cold run time: 104115 ms
Total hot run time: 32344 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6297	6233	6267	6233
q2	239	335	234	234
q3	2230	2622	2325	2325
q4	1477	1839	1401	1401
q5	4355	4727	4805	4727
q6	187	176	144	144
q7	2150	2000	1839	1839
q8	2586	2763	2686	2686
q9	7309	7165	7174	7165
q10	3021	3342	2771	2771
q11	576	502	510	502
q12	683	748	597	597
q13	3345	3764	3091	3091
q14	311	308	276	276
q15	570	498	506	498
q16	678	686	649	649
q17	1236	1751	1255	1255
q18	7857	7523	6986	6986
q19	789	1072	1073	1072
q20	1932	1972	1853	1853
q21	5468	5066	4861	4861
q22	597	599	578	578
Total cold run time: 53893 ms
Total hot run time: 51743 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191538 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c96e646315cb1105a1cbc12f0c528536e2e54860, data reload: false

query1	973	381	382	381
query2	6521	2523	2372	2372
query3	6705	219	216	216
query4	33529	23754	23706	23706
query5	4333	603	462	462
query6	298	197	180	180
query7	4623	490	300	300
query8	310	260	239	239
query9	9373	2660	2655	2655
query10	446	335	252	252
query11	18456	15461	15191	15191
query12	160	105	113	105
query13	1673	548	412	412
query14	11766	7214	8008	7214
query15	240	192	193	192
query16	7803	625	435	435
query17	1603	756	548	548
query18	1330	352	299	299
query19	207	176	148	148
query20	116	111	110	110
query21	206	119	102	102
query22	4418	4627	4384	4384
query23	35139	33513	33666	33513
query24	6331	2322	2346	2322
query25	508	452	377	377
query26	1202	276	157	157
query27	2044	461	331	331
query28	5236	2440	2411	2411
query29	656	542	410	410
query30	236	182	154	154
query31	997	890	830	830
query32	93	63	61	61
query33	508	357	322	322
query34	766	829	516	516
query35	795	836	752	752
query36	1029	1050	961	961
query37	108	99	78	78
query38	4123	4084	4086	4084
query39	1493	1448	1425	1425
query40	205	114	100	100
query41	46	48	46	46
query42	123	101	101	101
query43	529	532	496	496
query44	1334	821	823	821
query45	179	181	170	170
query46	858	1042	641	641
query47	1972	1956	1875	1875
query48	380	407	325	325
query49	761	473	379	379
query50	625	645	401	401
query51	7193	7105	7097	7097
query52	108	102	95	95
query53	221	248	186	186
query54	478	476	414	414
query55	81	81	79	79
query56	250	282	239	239
query57	1215	1199	1155	1155
query58	230	231	219	219
query59	3142	3212	2895	2895
query60	273	267	280	267
query61	108	111	107	107
query62	874	808	710	710
query63	231	189	192	189
query64	4442	1019	645	645
query65	3291	3193	3231	3193
query66	821	418	316	316
query67	16076	15782	15632	15632
query68	8942	747	531	531
query69	421	306	276	276
query70	1233	1201	1138	1138
query71	409	293	250	250
query72	6109	3929	4001	3929
query73	1079	751	359	359
query74	10566	9153	8924	8924
query75	4294	3181	2643	2643
query76	5149	1194	788	788
query77	936	369	283	283
query78	10032	10314	9844	9844
query79	4043	883	594	594
query80	832	518	425	425
query81	484	264	227	227
query82	279	156	116	116
query83	212	168	146	146
query84	281	94	73	73
query85	764	364	310	310
query86	362	309	312	309
query87	4439	4428	4524	4428
query88	3859	2249	2196	2196
query89	425	344	300	300
query90	2011	191	188	188
query91	139	138	105	105
query92	72	59	54	54
query93	2077	896	525	525
query94	683	441	293	293
query95	334	267	249	249
query96	485	596	279	279
query97	2728	2770	2723	2723
query98	228	211	200	200
query99	1646	1568	1433	1433
Total cold run time: 297952 ms
Total hot run time: 191538 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.26 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c96e646315cb1105a1cbc12f0c528536e2e54860, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.11	0.11
query5	0.44	0.41	0.39
query6	1.15	0.64	0.66
query7	0.02	0.01	0.01
query8	0.03	0.03	0.03
query9	0.57	0.52	0.51
query10	0.54	0.61	0.57
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.61	0.60
query14	2.78	2.83	2.83
query15	0.90	0.82	0.82
query16	0.39	0.39	0.37
query17	1.06	1.08	0.99
query18	0.23	0.21	0.21
query19	1.87	1.85	2.03
query20	0.01	0.02	0.01
query21	15.37	0.96	0.60
query22	0.75	0.73	0.64
query23	15.33	1.42	0.61
query24	3.16	1.40	0.98
query25	0.14	0.31	0.07
query26	0.33	0.15	0.14
query27	0.08	0.04	0.06
query28	13.92	1.51	1.05
query29	12.60	3.85	3.25
query30	0.25	0.10	0.06
query31	2.80	0.61	0.37
query32	3.23	0.55	0.46
query33	3.04	3.13	3.03
query34	16.93	5.15	4.45
query35	4.44	4.45	4.47
query36	0.66	0.51	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.14	0.13
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.37 s
Total hot run time: 31.26 s

be/test/vec/columns/column_array_test.cpp Outdated Show resolved Hide resolved
be/test/vec/columns/column_array_test.cpp Show resolved Hide resolved
be/test/vec/columns/column_array_test.cpp Outdated Show resolved Hide resolved
be/test/vec/columns/column_array_test.cpp Outdated Show resolved Hide resolved
be/test/vec/columns/column_array_test.cpp Outdated Show resolved Hide resolved
be/test/vec/columns/column_array_test.cpp Show resolved Hide resolved
be/test/vec/columns/column_array_test.cpp Show resolved Hide resolved
be/test/vec/columns/column_array_test.cpp Outdated Show resolved Hide resolved
@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32693 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7ccb484cf1d22e10cac68826f84e3afe7d49142b, data reload: false

------ Round 1 ----------------------------------
q1	17575	6185	6025	6025
q2	2052	317	176	176
q3	10401	1332	735	735
q4	10224	886	445	445
q5	7506	2183	2022	2022
q6	200	182	150	150
q7	936	778	616	616
q8	9248	1373	1233	1233
q9	5306	5002	4915	4915
q10	6747	2325	1891	1891
q11	468	273	271	271
q12	363	354	222	222
q13	17788	3619	2987	2987
q14	239	238	213	213
q15	566	522	496	496
q16	622	606	582	582
q17	591	899	325	325
q18	7002	6524	6310	6310
q19	1414	977	555	555
q20	306	314	181	181
q21	2945	2215	2027	2027
q22	365	327	316	316
Total cold run time: 102864 ms
Total hot run time: 32693 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6253	6257	6252	6252
q2	241	317	234	234
q3	2302	2621	2327	2327
q4	1463	1826	1383	1383
q5	4368	4765	4834	4765
q6	184	176	141	141
q7	2039	1956	1827	1827
q8	2598	2840	2708	2708
q9	7301	7258	7239	7239
q10	3083	3328	2857	2857
q11	594	528	502	502
q12	691	751	604	604
q13	3365	3726	3099	3099
q14	295	305	288	288
q15	572	512	502	502
q16	632	689	628	628
q17	1224	1731	1247	1247
q18	7775	7622	7070	7070
q19	812	1031	1083	1031
q20	1942	1980	1833	1833
q21	5592	5117	4876	4876
q22	616	572	568	568
Total cold run time: 53942 ms
Total hot run time: 51981 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189655 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7ccb484cf1d22e10cac68826f84e3afe7d49142b, data reload: false

query1	977	379	378	378
query2	6526	2318	2407	2318
query3	6698	210	214	210
query4	33825	23660	23669	23660
query5	4357	625	466	466
query6	277	217	191	191
query7	4632	495	318	318
query8	313	258	242	242
query9	9637	2746	2724	2724
query10	455	307	254	254
query11	18069	15434	15300	15300
query12	173	107	107	107
query13	1673	535	402	402
query14	11066	6734	6816	6734
query15	248	193	185	185
query16	8279	602	459	459
query17	1574	737	569	569
query18	2137	411	307	307
query19	233	183	160	160
query20	120	150	113	113
query21	205	120	106	106
query22	4470	4480	4317	4317
query23	34382	33226	32956	32956
query24	6373	2228	2347	2228
query25	501	461	389	389
query26	1213	266	147	147
query27	2039	466	334	334
query28	5029	2459	2434	2434
query29	742	544	412	412
query30	224	181	156	156
query31	965	902	807	807
query32	116	62	57	57
query33	500	346	289	289
query34	746	828	509	509
query35	811	827	750	750
query36	1024	1077	960	960
query37	115	106	75	75
query38	4149	4077	4139	4077
query39	1493	1440	1469	1440
query40	200	109	96	96
query41	46	44	46	44
query42	120	100	102	100
query43	491	529	495	495
query44	1294	803	794	794
query45	176	167	166	166
query46	845	1024	639	639
query47	1901	1933	1879	1879
query48	391	405	332	332
query49	771	477	380	380
query50	617	646	385	385
query51	7121	7106	7002	7002
query52	100	106	91	91
query53	226	251	185	185
query54	472	488	392	392
query55	80	79	77	77
query56	268	284	242	242
query57	1227	1182	1139	1139
query58	252	225	230	225
query59	3144	3349	3010	3010
query60	277	257	232	232
query61	115	107	113	107
query62	904	803	750	750
query63	221	190	185	185
query64	4545	986	674	674
query65	3287	3157	3222	3157
query66	1040	431	303	303
query67	15881	15765	15585	15585
query68	8730	741	504	504
query69	459	290	245	245
query70	1188	1141	1153	1141
query71	420	317	243	243
query72	5811	3764	3835	3764
query73	1133	743	363	363
query74	10358	9003	8920	8920
query75	4378	3182	2675	2675
query76	5103	1176	759	759
query77	944	365	272	272
query78	10058	10102	9463	9463
query79	2851	875	644	644
query80	692	516	431	431
query81	464	279	235	235
query82	335	148	124	124
query83	202	161	144	144
query84	279	87	72	72
query85	743	345	319	319
query86	348	303	303	303
query87	4454	4506	4385	4385
query88	3690	2221	2173	2173
query89	403	327	303	303
query90	2029	188	185	185
query91	133	132	106	106
query92	63	55	53	53
query93	1424	880	530	530
query94	670	403	286	286
query95	337	263	248	248
query96	501	603	274	274
query97	2748	2849	2667	2667
query98	225	201	200	200
query99	1642	1561	1418	1418
Total cold run time: 295193 ms
Total hot run time: 189655 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.68 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7ccb484cf1d22e10cac68826f84e3afe7d49142b, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.43	0.41	0.40
query6	1.16	0.65	0.64
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.50	0.51
query10	0.56	0.57	0.56
query11	0.15	0.11	0.11
query12	0.13	0.10	0.11
query13	0.62	0.60	0.59
query14	2.73	2.72	2.74
query15	0.88	0.82	0.82
query16	0.40	0.40	0.39
query17	1.06	1.09	1.05
query18	0.23	0.22	0.21
query19	1.90	1.74	1.98
query20	0.02	0.01	0.01
query21	15.35	0.99	0.58
query22	0.74	0.77	0.65
query23	15.33	1.46	0.56
query24	2.92	1.54	1.42
query25	0.17	0.09	0.16
query26	0.32	0.18	0.13
query27	0.06	0.04	0.05
query28	14.23	1.53	1.05
query29	12.58	3.96	3.26
query30	0.24	0.09	0.06
query31	2.84	0.58	0.39
query32	3.23	0.56	0.46
query33	3.07	3.24	3.11
query34	16.80	5.16	4.50
query35	4.55	4.53	4.47
query36	0.66	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.04	0.03	0.02
query40	0.17	0.14	0.13
query41	0.09	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.48 s
Total hot run time: 31.68 s

@amorynan
Copy link
Contributor Author

run buildall

@amorynan
Copy link
Contributor Author

run buildall

@amorynan
Copy link
Contributor Author

run buildall

@amorynan
Copy link
Contributor Author

amorynan commented Jan 1, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32635 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cd28ffbbf3c5ff8a1548887489306b2457afc7f6, data reload: false

------ Round 1 ----------------------------------
q1	17590	6104	6064	6064
q2	2060	290	164	164
q3	10432	1213	720	720
q4	10219	863	436	436
q5	7731	2164	1975	1975
q6	207	176	153	153
q7	892	759	608	608
q8	9233	1322	1188	1188
q9	5136	4858	4834	4834
q10	6749	2299	1869	1869
q11	480	279	259	259
q12	353	372	223	223
q13	17799	3577	3020	3020
q14	232	250	224	224
q15	580	505	498	498
q16	654	617	587	587
q17	562	839	320	320
q18	6792	6507	6473	6473
q19	3161	963	542	542
q20	295	310	187	187
q21	2784	2105	1982	1982
q22	363	331	309	309
Total cold run time: 104304 ms
Total hot run time: 32635 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6391	6198	6158	6158
q2	237	330	235	235
q3	2246	2645	2330	2330
q4	1395	1822	1326	1326
q5	4299	4731	4807	4731
q6	185	174	145	145
q7	2059	1988	1791	1791
q8	2620	2815	2696	2696
q9	7265	7178	7283	7178
q10	3074	3356	2870	2870
q11	567	500	496	496
q12	682	754	618	618
q13	3395	3787	3064	3064
q14	303	300	279	279
q15	581	512	506	506
q16	640	693	645	645
q17	1249	1765	1271	1271
q18	7727	7381	7249	7249
q19	806	1170	1042	1042
q20	1972	2092	1911	1911
q21	5641	5447	4931	4931
q22	602	579	572	572
Total cold run time: 53936 ms
Total hot run time: 52044 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198635 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cd28ffbbf3c5ff8a1548887489306b2457afc7f6, data reload: false

query1	1307	952	1013	952
query2	6491	2425	2405	2405
query3	10947	4809	4990	4809
query4	33160	23824	23817	23817
query5	3927	613	460	460
query6	261	227	195	195
query7	3988	482	310	310
query8	326	249	235	235
query9	9441	2734	2741	2734
query10	475	315	254	254
query11	18086	15364	15271	15271
query12	156	109	108	108
query13	1589	548	427	427
query14	10534	7451	7783	7451
query15	235	218	191	191
query16	7964	603	460	460
query17	1545	762	591	591
query18	2117	406	305	305
query19	197	176	156	156
query20	121	112	140	112
query21	204	125	102	102
query22	4737	4635	4560	4560
query23	34592	33496	33911	33496
query24	6446	2400	2318	2318
query25	484	456	385	385
query26	874	278	162	162
query27	2045	473	333	333
query28	5643	2503	2459	2459
query29	586	557	425	425
query30	217	177	146	146
query31	986	925	841	841
query32	69	58	57	57
query33	485	359	287	287
query34	781	864	548	548
query35	787	853	783	783
query36	1039	1055	981	981
query37	116	101	81	81
query38	4419	4279	4337	4279
query39	1571	1445	1507	1445
query40	207	113	101	101
query41	46	43	47	43
query42	123	112	106	106
query43	547	560	497	497
query44	1371	837	859	837
query45	182	174	181	174
query46	892	1063	665	665
query47	2011	2031	1993	1993
query48	410	401	331	331
query49	734	485	386	386
query50	689	680	400	400
query51	7356	7176	7153	7153
query52	104	104	91	91
query53	222	256	188	188
query54	480	502	411	411
query55	84	82	84	82
query56	268	262	245	245
query57	1271	1228	1154	1154
query58	244	228	230	228
query59	3205	3516	3255	3255
query60	282	298	259	259
query61	126	103	119	103
query62	866	805	754	754
query63	234	190	198	190
query64	3109	1089	661	661
query65	3376	3271	3219	3219
query66	807	407	312	312
query67	16448	15775	15571	15571
query68	9625	774	528	528
query69	487	288	263	263
query70	1200	1139	1146	1139
query71	436	282	253	253
query72	5922	3864	3979	3864
query73	917	778	374	374
query74	10499	9137	9176	9137
query75	4687	3166	2672	2672
query76	5612	1175	762	762
query77	1040	371	285	285
query78	10149	10256	9911	9911
query79	2687	896	611	611
query80	720	512	444	444
query81	499	261	221	221
query82	531	150	118	118
query83	193	181	141	141
query84	278	89	70	70
query85	778	354	338	338
query86	359	304	303	303
query87	4482	4601	4338	4338
query88	3375	2254	2183	2183
query89	417	327	304	304
query90	1963	190	190	190
query91	132	131	106	106
query92	65	57	52	52
query93	1187	911	542	542
query94	660	382	293	293
query95	342	277	260	260
query96	490	619	276	276
query97	2761	2840	2668	2668
query98	229	207	193	193
query99	1667	1560	1456	1456
Total cold run time: 299031 ms
Total hot run time: 198635 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.81 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cd28ffbbf3c5ff8a1548887489306b2457afc7f6, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.04	0.03
query3	0.24	0.07	0.07
query4	1.60	0.10	0.11
query5	0.40	0.41	0.41
query6	1.16	0.66	0.65
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.59	0.49	0.50
query10	0.55	0.55	0.55
query11	0.14	0.10	0.11
query12	0.14	0.11	0.10
query13	0.61	0.62	0.60
query14	2.72	2.72	2.74
query15	0.88	0.82	0.82
query16	0.38	0.39	0.40
query17	1.06	1.07	0.98
query18	0.22	0.21	0.21
query19	1.97	1.85	1.96
query20	0.02	0.01	0.01
query21	15.36	0.89	0.57
query22	0.76	0.92	0.66
query23	15.20	1.39	0.53
query24	3.02	1.59	1.96
query25	0.19	0.08	0.09
query26	0.30	0.14	0.13
query27	0.06	0.06	0.04
query28	14.54	1.47	1.05
query29	12.58	3.97	3.32
query30	0.25	0.09	0.06
query31	2.82	0.59	0.37
query32	3.23	0.52	0.46
query33	3.17	3.13	3.08
query34	16.59	5.07	4.49
query35	4.50	4.44	4.43
query36	0.64	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.13
query41	0.07	0.03	0.02
query42	0.03	0.03	0.02
query43	0.03	0.03	0.04
Total cold run time: 106.52 s
Total hot run time: 31.81 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 39.27% (10217/26015)
Line Coverage: 30.26% (86612/286203)
Region Coverage: 29.44% (44370/150708)
Branch Coverage: 26.03% (22733/87334)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cd28ffbbf3c5ff8a1548887489306b2457afc7f6_cd28ffbbf3c5ff8a1548887489306b2457afc7f6/report/index.html

@amorynan
Copy link
Contributor Author

amorynan commented Jan 7, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32847 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 13ac07e3a677ee63e7444f334701bb2d51165e9f, data reload: false

------ Round 1 ----------------------------------
q1	17643	6197	6080	6080
q2	2050	307	178	178
q3	10490	1321	733	733
q4	10228	888	455	455
q5	7615	2260	1995	1995
q6	204	185	149	149
q7	922	769	619	619
q8	9215	1375	1209	1209
q9	5399	4881	4893	4881
q10	6770	2329	1851	1851
q11	464	278	261	261
q12	343	361	226	226
q13	17776	3752	3125	3125
q14	243	234	205	205
q15	570	510	509	509
q16	636	603	585	585
q17	591	855	319	319
q18	7218	6487	6404	6404
q19	1624	981	568	568
q20	317	321	188	188
q21	2854	2273	2002	2002
q22	365	329	305	305
Total cold run time: 103537 ms
Total hot run time: 32847 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6279	6216	6277	6216
q2	239	323	229	229
q3	2233	2695	2265	2265
q4	1404	1884	1359	1359
q5	4403	4774	4955	4774
q6	188	176	141	141
q7	2120	2029	1837	1837
q8	2638	2812	2676	2676
q9	7298	7297	7238	7238
q10	3027	3357	2828	2828
q11	606	529	503	503
q12	658	753	602	602
q13	3465	3804	3188	3188
q14	277	326	278	278
q15	577	511	503	503
q16	646	656	641	641
q17	1224	1745	1250	1250
q18	7919	7408	7427	7408
q19	826	1143	1043	1043
q20	1912	1977	1848	1848
q21	5466	4970	4910	4910
q22	596	624	570	570
Total cold run time: 54001 ms
Total hot run time: 52307 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191387 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 13ac07e3a677ee63e7444f334701bb2d51165e9f, data reload: false

query1	967	377	385	377
query2	6518	2519	2313	2313
query3	6705	215	214	214
query4	33913	23374	23448	23374
query5	4364	656	468	468
query6	293	202	192	192
query7	4617	491	301	301
query8	299	277	239	239
query9	9750	2748	2729	2729
query10	482	314	253	253
query11	18359	15416	15389	15389
query12	161	113	110	110
query13	1674	536	406	406
query14	10091	7764	7367	7367
query15	246	194	196	194
query16	8306	619	432	432
query17	1618	766	579	579
query18	2151	404	300	300
query19	238	227	161	161
query20	117	113	114	113
query21	209	125	110	110
query22	4332	4498	4199	4199
query23	35044	34002	33537	33537
query24	6636	2317	2284	2284
query25	495	440	393	393
query26	1202	270	152	152
query27	2077	457	325	325
query28	5389	2464	2444	2444
query29	752	532	414	414
query30	229	184	152	152
query31	963	930	811	811
query32	84	62	60	60
query33	508	344	293	293
query34	745	862	518	518
query35	779	827	743	743
query36	1033	1058	953	953
query37	117	96	80	80
query38	4153	4075	4178	4075
query39	1476	1424	1470	1424
query40	200	112	102	102
query41	49	47	45	45
query42	121	104	100	100
query43	520	523	496	496
query44	1293	813	817	813
query45	176	173	169	169
query46	863	1038	645	645
query47	1903	1935	1852	1852
query48	385	411	347	347
query49	751	474	405	405
query50	631	668	376	376
query51	7020	7034	6915	6915
query52	104	100	91	91
query53	221	266	183	183
query54	466	471	415	415
query55	79	82	77	77
query56	266	246	256	246
query57	1180	1204	1117	1117
query58	235	221	228	221
query59	3041	3088	2918	2918
query60	274	271	251	251
query61	114	106	108	106
query62	848	810	747	747
query63	228	199	189	189
query64	4479	983	640	640
query65	3340	3181	3238	3181
query66	844	423	323	323
query67	16018	15847	15585	15585
query68	7678	696	520	520
query69	450	289	253	253
query70	1230	1125	1119	1119
query71	420	305	269	269
query72	6491	3898	3884	3884
query73	647	751	354	354
query74	10590	9325	9379	9325
query75	4182	3172	2772	2772
query76	3800	1211	772	772
query77	1117	393	300	300
query78	10293	10085	9405	9405
query79	2837	812	604	604
query80	669	527	428	428
query81	495	275	231	231
query82	692	149	120	120
query83	240	165	140	140
query84	232	90	67	67
query85	793	380	299	299
query86	408	308	303	303
query87	4398	4564	4532	4532
query88	4692	2202	2168	2168
query89	408	337	303	303
query90	1840	194	193	193
query91	131	139	105	105
query92	69	56	54	54
query93	1685	899	536	536
query94	649	397	286	286
query95	332	265	253	253
query96	488	603	283	283
query97	2957	3018	2835	2835
query98	233	210	195	195
query99	1622	1586	1446	1446
Total cold run time: 295509 ms
Total hot run time: 191387 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.45 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 13ac07e3a677ee63e7444f334701bb2d51165e9f, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.60	0.11	0.11
query5	0.44	0.40	0.41
query6	1.18	0.67	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.49	0.50
query10	0.56	0.57	0.55
query11	0.14	0.11	0.10
query12	0.14	0.10	0.11
query13	0.59	0.62	0.59
query14	2.74	2.86	2.75
query15	0.89	0.82	0.85
query16	0.39	0.39	0.38
query17	1.04	1.08	1.06
query18	0.22	0.21	0.21
query19	1.88	1.90	1.99
query20	0.01	0.02	0.01
query21	15.40	0.91	0.60
query22	0.74	0.73	0.84
query23	15.22	1.42	0.53
query24	3.61	0.67	1.62
query25	0.20	0.24	0.23
query26	0.18	0.15	0.13
query27	0.08	0.05	0.05
query28	14.41	1.50	1.05
query29	12.58	4.04	3.36
query30	0.25	0.09	0.06
query31	2.81	0.62	0.39
query32	3.23	0.55	0.47
query33	3.09	3.10	3.16
query34	16.83	5.09	4.53
query35	4.50	4.48	4.47
query36	0.82	0.48	0.48
query37	0.09	0.06	0.05
query38	0.05	0.03	0.04
query39	0.04	0.02	0.02
query40	0.18	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 107.26 s
Total hot run time: 31.45 s

@amorynan
Copy link
Contributor Author

amorynan commented Jan 8, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32705 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8df94c46dbd256698f9944adafc5338f59c0411c, data reload: false

------ Round 1 ----------------------------------
q1	17567	6197	6094	6094
q2	2049	315	168	168
q3	10417	1263	737	737
q4	10228	874	446	446
q5	7947	2191	1954	1954
q6	234	182	150	150
q7	878	773	611	611
q8	9239	1358	1123	1123
q9	5278	4853	4900	4853
q10	6772	2324	1878	1878
q11	477	264	258	258
q12	358	371	229	229
q13	17843	3710	3145	3145
q14	256	240	229	229
q15	579	516	500	500
q16	632	619	596	596
q17	589	861	339	339
q18	7003	6518	6413	6413
q19	1735	952	540	540
q20	305	325	188	188
q21	2913	2165	1961	1961
q22	370	352	293	293
Total cold run time: 103669 ms
Total hot run time: 32705 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6298	6274	6264	6264
q2	235	322	233	233
q3	2274	2668	2301	2301
q4	1417	1833	1324	1324
q5	4395	4761	4839	4761
q6	187	179	142	142
q7	2028	1995	1820	1820
q8	2678	2817	2701	2701
q9	7319	7225	7276	7225
q10	3084	3351	2882	2882
q11	580	510	503	503
q12	633	748	605	605
q13	3540	3884	3249	3249
q14	304	334	289	289
q15	553	523	509	509
q16	644	693	641	641
q17	1251	1718	1281	1281
q18	7740	7610	7450	7450
q19	831	982	1147	982
q20	1986	2083	1953	1953
q21	5775	5164	4764	4764
q22	611	610	595	595
Total cold run time: 54363 ms
Total hot run time: 52474 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196582 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8df94c46dbd256698f9944adafc5338f59c0411c, data reload: false

query1	1283	958	946	946
query2	6444	2371	2370	2370
query3	10962	4780	4693	4693
query4	33182	23648	23876	23648
query5	4328	614	488	488
query6	292	196	182	182
query7	3989	490	301	301
query8	301	247	237	237
query9	9423	2736	2718	2718
query10	469	314	253	253
query11	18095	15435	15206	15206
query12	166	109	105	105
query13	1665	541	404	404
query14	10979	6885	7805	6885
query15	278	203	194	194
query16	8417	608	480	480
query17	1604	737	583	583
query18	2148	422	308	308
query19	204	177	154	154
query20	122	110	111	110
query21	204	120	112	112
query22	4528	4615	4484	4484
query23	34677	33569	33912	33569
query24	6365	2376	2320	2320
query25	458	462	400	400
query26	922	275	157	157
query27	2289	460	329	329
query28	5491	2498	2469	2469
query29	590	569	417	417
query30	216	201	152	152
query31	974	929	866	866
query32	71	62	57	57
query33	484	359	295	295
query34	791	839	525	525
query35	790	862	761	761
query36	1051	1084	992	992
query37	118	104	78	78
query38	4147	4243	4104	4104
query39	1527	1469	1469	1469
query40	214	121	98	98
query41	43	42	44	42
query42	126	105	109	105
query43	525	542	497	497
query44	1358	838	866	838
query45	178	173	183	173
query46	884	1068	653	653
query47	1998	2014	1936	1936
query48	390	423	312	312
query49	726	488	407	407
query50	664	668	403	403
query51	7175	7206	6984	6984
query52	102	100	88	88
query53	240	259	186	186
query54	481	502	406	406
query55	104	85	92	85
query56	270	263	244	244
query57	1255	1234	1175	1175
query58	244	230	231	230
query59	3397	3259	3242	3242
query60	271	268	244	244
query61	120	125	114	114
query62	884	841	764	764
query63	231	190	195	190
query64	3172	1016	673	673
query65	3359	3275	3240	3240
query66	893	404	308	308
query67	16507	15939	15562	15562
query68	8333	705	526	526
query69	501	299	256	256
query70	1206	1129	1108	1108
query71	437	297	257	257
query72	6407	3931	3856	3856
query73	663	770	357	357
query74	9792	8916	8856	8856
query75	4482	3175	2672	2672
query76	3936	1197	803	803
query77	772	377	278	278
query78	10173	10254	9383	9383
query79	3541	820	605	605
query80	591	527	442	442
query81	491	275	233	233
query82	666	153	119	119
query83	239	161	151	151
query84	239	91	67	67
query85	794	371	299	299
query86	389	319	302	302
query87	4381	4600	4307	4307
query88	4666	2192	2197	2192
query89	426	337	293	293
query90	1771	190	186	186
query91	129	134	104	104
query92	70	56	53	53
query93	2249	880	539	539
query94	649	392	294	294
query95	338	264	249	249
query96	487	626	278	278
query97	2929	3028	2863	2863
query98	224	209	192	192
query99	1587	1548	1442	1442
Total cold run time: 299413 ms
Total hot run time: 196582 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.71 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8df94c46dbd256698f9944adafc5338f59c0411c, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.11
query5	0.41	0.42	0.42
query6	1.13	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.51	0.50
query10	0.54	0.57	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.60	0.61
query14	2.80	2.72	2.72
query15	0.90	0.83	0.82
query16	0.38	0.39	0.38
query17	1.03	1.09	1.06
query18	0.23	0.20	0.21
query19	1.84	1.75	2.01
query20	0.01	0.01	0.01
query21	15.39	0.95	0.61
query22	0.76	0.70	0.72
query23	15.32	1.47	0.62
query24	3.36	1.18	2.19
query25	0.12	0.34	0.11
query26	0.22	0.14	0.14
query27	0.05	0.06	0.04
query28	14.13	1.45	1.05
query29	12.57	4.02	3.32
query30	0.25	0.08	0.06
query31	2.83	0.60	0.39
query32	3.22	0.53	0.46
query33	3.21	3.10	3.15
query34	16.86	5.11	4.53
query35	4.44	4.48	4.47
query36	0.83	0.48	0.50
query37	0.08	0.06	0.05
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.8 s
Total hot run time: 31.71 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 39.45% (10279/26053)
Line Coverage: 30.44% (87237/286572)
Region Coverage: 29.58% (44602/150792)
Branch Coverage: 26.17% (22869/87402)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8df94c46dbd256698f9944adafc5338f59c0411c_8df94c46dbd256698f9944adafc5338f59c0411c/report/index.html

eldenmoon
eldenmoon previously approved these changes Jan 8, 2025
Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@amorynan
Copy link
Contributor Author

amorynan commented Jan 9, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32729 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 01a2d741e62c02c34fb5bdd76382ac1dca9cd5f4, data reload: false

------ Round 1 ----------------------------------
q1	17582	6288	6038	6038
q2	2056	307	175	175
q3	10480	1239	753	753
q4	10205	870	429	429
q5	7552	2204	1962	1962
q6	209	179	144	144
q7	915	751	605	605
q8	9255	1376	1141	1141
q9	5303	4961	4953	4953
q10	6779	2320	1841	1841
q11	507	280	259	259
q12	357	366	215	215
q13	17785	3696	3115	3115
q14	240	241	211	211
q15	562	489	496	489
q16	623	606	583	583
q17	570	854	328	328
q18	7228	6566	6488	6488
q19	2186	967	571	571
q20	310	312	185	185
q21	2871	2147	1921	1921
q22	363	343	323	323
Total cold run time: 103938 ms
Total hot run time: 32729 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6305	6191	6200	6191
q2	236	328	230	230
q3	2258	2710	2307	2307
q4	1438	1829	1380	1380
q5	4349	4722	4813	4722
q6	188	188	142	142
q7	2065	1964	1811	1811
q8	2621	2835	2712	2712
q9	7391	7261	7291	7261
q10	3096	3244	2750	2750
q11	594	528	506	506
q12	672	794	638	638
q13	3426	3878	3160	3160
q14	295	307	266	266
q15	561	522	518	518
q16	671	699	646	646
q17	1235	1734	1275	1275
q18	7830	7580	7388	7388
q19	840	1171	1039	1039
q20	1969	2181	2020	2020
q21	5759	5135	5142	5135
q22	636	622	585	585
Total cold run time: 54435 ms
Total hot run time: 52682 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194616 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 01a2d741e62c02c34fb5bdd76382ac1dca9cd5f4, data reload: false

query1	1294	955	898	898
query2	6379	2305	2367	2305
query3	11145	4778	4986	4778
query4	32992	24298	23164	23164
query5	5005	588	446	446
query6	312	206	177	177
query7	3978	483	316	316
query8	302	245	256	245
query9	9586	2677	2647	2647
query10	484	316	263	263
query11	18250	15174	15045	15045
query12	157	111	101	101
query13	1551	518	401	401
query14	9113	7752	7230	7230
query15	236	214	191	191
query16	7666	613	473	473
query17	1536	786	597	597
query18	1535	392	331	331
query19	222	217	161	161
query20	121	117	120	117
query21	207	129	107	107
query22	4791	4571	4601	4571
query23	34078	33285	33180	33180
query24	6385	2260	2317	2260
query25	474	449	388	388
query26	817	270	149	149
query27	2101	471	333	333
query28	5515	2477	2477	2477
query29	581	553	432	432
query30	231	179	151	151
query31	982	922	798	798
query32	67	59	57	57
query33	491	337	301	301
query34	746	859	517	517
query35	800	822	770	770
query36	1020	1043	977	977
query37	140	105	75	75
query38	4079	4058	4041	4041
query39	1519	1462	1449	1449
query40	200	112	101	101
query41	53	49	49	49
query42	122	102	108	102
query43	517	523	484	484
query44	1369	868	835	835
query45	187	172	171	171
query46	858	1039	643	643
query47	1927	1958	1901	1901
query48	376	401	306	306
query49	698	486	405	405
query50	647	651	394	394
query51	7206	6999	7024	6999
query52	103	102	91	91
query53	223	248	179	179
query54	475	512	417	417
query55	78	86	77	77
query56	256	270	236	236
query57	1223	1178	1148	1148
query58	236	229	238	229
query59	3151	3266	3049	3049
query60	278	268	243	243
query61	107	106	131	106
query62	843	787	746	746
query63	221	189	193	189
query64	3425	1087	653	653
query65	3278	3241	3160	3160
query66	766	402	308	308
query67	16025	15848	15439	15439
query68	8518	686	498	498
query69	490	291	257	257
query70	1161	1154	1097	1097
query71	423	272	249	249
query72	5126	3785	3877	3785
query73	647	741	346	346
query74	10264	9083	8758	8758
query75	4595	3139	2628	2628
query76	4519	1173	746	746
query77	786	359	267	267
query78	10157	10216	9460	9460
query79	2468	743	608	608
query80	591	516	425	425
query81	492	265	232	232
query82	226	154	120	120
query83	240	161	136	136
query84	228	92	68	68
query85	735	352	292	292
query86	356	307	305	305
query87	4311	4571	4303	4303
query88	4288	2148	2154	2148
query89	390	317	288	288
query90	1904	184	180	180
query91	133	129	104	104
query92	67	55	50	50
query93	1676	858	530	530
query94	586	394	286	286
query95	337	256	239	239
query96	475	626	280	280
query97	2782	2960	2762	2762
query98	212	206	197	197
query99	1397	1492	1393	1393
Total cold run time: 292316 ms
Total hot run time: 194616 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.77 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 01a2d741e62c02c34fb5bdd76382ac1dca9cd5f4, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.04
query3	0.24	0.07	0.07
query4	1.61	0.11	0.10
query5	0.44	0.42	0.42
query6	1.17	0.65	0.66
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.52	0.50
query10	0.56	0.55	0.53
query11	0.15	0.11	0.10
query12	0.13	0.12	0.10
query13	0.61	0.59	0.60
query14	2.85	2.81	2.77
query15	0.87	0.84	0.82
query16	0.39	0.39	0.37
query17	1.07	1.05	1.08
query18	0.23	0.21	0.21
query19	1.86	1.86	1.97
query20	0.01	0.02	0.01
query21	15.35	0.94	0.60
query22	0.77	0.85	0.74
query23	15.14	1.33	0.54
query24	2.94	1.92	0.36
query25	0.20	0.19	0.08
query26	0.28	0.15	0.14
query27	0.05	0.05	0.05
query28	13.51	1.54	1.04
query29	12.58	3.97	3.22
query30	0.25	0.09	0.06
query31	2.83	0.59	0.37
query32	3.22	0.55	0.46
query33	3.08	3.06	3.22
query34	16.83	5.16	4.51
query35	4.51	4.50	4.50
query36	0.85	0.48	0.49
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.8 s
Total hot run time: 30.77 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 39.46% (10278/26047)
Line Coverage: 30.46% (87235/286370)
Region Coverage: 29.59% (44587/150671)
Branch Coverage: 26.18% (22861/87322)
Coverage Report: http://coverage.selectdb-in.cc/coverage/01a2d741e62c02c34fb5bdd76382ac1dca9cd5f4_01a2d741e62c02c34fb5bdd76382ac1dca9cd5f4/report/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants