Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](cloud-mow) Fix the issue of compaction holding mow table lock for a long time #40431

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

hust-hhb
Copy link
Contributor

@hust-hhb hust-hhb commented Sep 5, 2024

Now compaction use the same lock id (i.e. -1) when getting mow table lock from ms, if there are a large number of continuous compaction tasks to be executed, compaction will hold the lock for a long time, which will make loading task hang on waiting mow table lock when commiting txn, my test show it may lead to commit txn fail.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@hust-hhb
Copy link
Contributor Author

hust-hhb commented Sep 5, 2024

run buildall

Copy link
Contributor

github-actions bot commented Sep 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

@hust-hhb
Copy link
Contributor Author

hust-hhb commented Sep 5, 2024

run buildall

Copy link
Contributor

github-actions bot commented Sep 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38443 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9fdfda03d0d5fe0d853cd116d08eb591bd2d66a6, data reload: false

------ Round 1 ----------------------------------
q1	17620	4476	4391	4391
q2	2013	191	184	184
q3	11782	981	1115	981
q4	10509	713	755	713
q5	7728	2866	2835	2835
q6	229	142	140	140
q7	958	606	601	601
q8	9531	2089	2081	2081
q9	7002	6571	6581	6571
q10	7013	2252	2263	2252
q11	459	246	243	243
q12	403	224	225	224
q13	17833	3131	3053	3053
q14	269	249	232	232
q15	536	493	474	474
q16	603	531	504	504
q17	987	702	729	702
q18	7410	6858	6873	6858
q19	1390	1040	1116	1040
q20	700	326	339	326
q21	3878	3362	3032	3032
q22	1097	1006	1027	1006
Total cold run time: 109950 ms
Total hot run time: 38443 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4368	4328	4292	4292
q2	381	270	277	270
q3	2923	2662	2678	2662
q4	1996	1745	1711	1711
q5	5703	5680	5863	5680
q6	235	137	141	137
q7	2275	1891	1860	1860
q8	3309	3419	3530	3419
q9	8882	8882	8853	8853
q10	3690	3454	3336	3336
q11	603	519	532	519
q12	838	684	713	684
q13	16270	3319	3305	3305
q14	323	288	282	282
q15	535	490	512	490
q16	628	586	596	586
q17	1862	1577	1515	1515
q18	8247	7925	7895	7895
q19	1756	1735	1529	1529
q20	2166	1933	1963	1933
q21	5756	5563	5292	5292
q22	1144	1083	1059	1059
Total cold run time: 73890 ms
Total hot run time: 57309 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193593 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9fdfda03d0d5fe0d853cd116d08eb591bd2d66a6, data reload: false

query1	1267	901	924	901
query2	6465	1968	1898	1898
query3	10633	3987	3882	3882
query4	59534	23853	23271	23271
query5	5440	507	516	507
query6	410	160	164	160
query7	5820	298	297	297
query8	330	229	225	225
query9	8825	2522	2512	2512
query10	485	281	272	272
query11	18119	14932	15191	14932
query12	154	113	105	105
query13	1602	428	420	420
query14	11133	7890	7440	7440
query15	238	166	179	166
query16	7024	472	490	472
query17	1142	591	647	591
query18	1228	309	303	303
query19	301	162	166	162
query20	125	113	113	113
query21	206	111	108	108
query22	4613	4483	4534	4483
query23	35453	34049	33633	33633
query24	5931	2874	2790	2790
query25	482	393	389	389
query26	672	156	150	150
query27	1760	277	278	277
query28	3949	2063	2045	2045
query29	627	414	406	406
query30	236	156	155	155
query31	945	809	787	787
query32	80	54	57	54
query33	467	286	272	272
query34	882	484	461	461
query35	863	738	720	720
query36	1068	947	943	943
query37	152	94	89	89
query38	4071	3872	3878	3872
query39	1432	1395	1380	1380
query40	193	113	116	113
query41	48	46	43	43
query42	113	94	95	94
query43	507	456	473	456
query44	1110	759	744	744
query45	199	165	162	162
query46	1086	786	731	731
query47	1940	1802	1853	1802
query48	382	296	291	291
query49	803	437	451	437
query50	842	411	409	409
query51	7117	6901	6887	6887
query52	97	87	85	85
query53	249	180	181	180
query54	564	456	468	456
query55	78	73	73	73
query56	276	248	265	248
query57	1176	1095	1071	1071
query58	219	228	233	228
query59	3099	2849	2868	2849
query60	291	264	270	264
query61	104	99	102	99
query62	737	643	669	643
query63	216	192	185	185
query64	2897	684	663	663
query65	3209	3141	3157	3141
query66	626	342	335	335
query67	15511	15379	15390	15379
query68	2947	590	589	589
query69	404	283	285	283
query70	1164	1129	1144	1129
query71	338	284	269	269
query72	5812	4228	3976	3976
query73	758	331	322	322
query74	9289	8807	8953	8807
query75	3373	2680	2710	2680
query76	1361	961	1013	961
query77	562	324	332	324
query78	10788	9360	9192	9192
query79	1047	533	591	533
query80	688	506	521	506
query81	468	246	231	231
query82	250	147	149	147
query83	169	146	148	146
query84	258	78	76	76
query85	669	278	282	278
query86	300	294	306	294
query87	4336	4426	4281	4281
query88	3003	2379	2392	2379
query89	396	296	287	287
query90	2001	188	187	187
query91	124	99	99	99
query92	62	52	50	50
query93	1060	532	544	532
query94	690	280	293	280
query95	353	252	251	251
query96	586	275	266	266
query97	3234	3077	3038	3038
query98	228	208	211	208
query99	1544	1283	1281	1281
Total cold run time: 306115 ms
Total hot run time: 193593 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.23 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9fdfda03d0d5fe0d853cd116d08eb591bd2d66a6, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.04	0.04
query3	0.22	0.04	0.05
query4	1.68	0.07	0.08
query5	0.52	0.51	0.51
query6	1.14	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.55	0.50	0.49
query10	0.55	0.57	0.55
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	2.06	2.08	2.06
query15	0.85	0.83	0.83
query16	0.37	0.36	0.38
query17	1.06	1.04	1.05
query18	0.21	0.20	0.21
query19	1.88	1.88	1.75
query20	0.02	0.01	0.01
query21	15.39	0.67	0.67
query22	4.66	8.26	1.17
query23	18.27	1.25	1.23
query24	2.14	0.22	0.22
query25	0.15	0.09	0.08
query26	0.26	0.18	0.17
query27	0.07	0.08	0.08
query28	13.28	1.03	0.99
query29	12.64	3.35	3.34
query30	0.25	0.05	0.06
query31	2.88	0.39	0.40
query32	3.25	0.48	0.48
query33	2.97	2.96	3.02
query34	17.04	4.32	4.39
query35	4.44	4.48	4.39
query36	0.66	0.49	0.51
query37	0.18	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.17	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 111.29 s
Total hot run time: 31.23 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.84% (9395/25503)
Line Coverage: 28.28% (77465/273951)
Region Coverage: 27.67% (39977/144464)
Branch Coverage: 24.31% (20341/83668)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9fdfda03d0d5fe0d853cd116d08eb591bd2d66a6_9fdfda03d0d5fe0d853cd116d08eb591bd2d66a6/report/index.html

@hust-hhb
Copy link
Contributor Author

hust-hhb commented Sep 5, 2024

run buildall

Copy link
Contributor

github-actions bot commented Sep 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.84% (9395/25503)
Line Coverage: 28.27% (77438/273951)
Region Coverage: 27.67% (39975/144464)
Branch Coverage: 24.31% (20343/83668)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3f3d16eac8a58c8bff61c1bb53994f19f2e2662b_3f3d16eac8a58c8bff61c1bb53994f19f2e2662b/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 37891 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3f3d16eac8a58c8bff61c1bb53994f19f2e2662b, data reload: false

------ Round 1 ----------------------------------
q1	17782	4412	4309	4309
q2	2029	191	179	179
q3	11789	946	1141	946
q4	10512	727	726	726
q5	7777	2858	2823	2823
q6	227	138	140	138
q7	957	604	609	604
q8	9522	2067	2066	2066
q9	7297	6526	6539	6526
q10	7001	2168	2227	2168
q11	458	247	242	242
q12	394	217	218	217
q13	17754	3087	3073	3073
q14	282	238	239	238
q15	533	499	491	491
q16	613	525	510	510
q17	981	635	747	635
q18	7310	6773	6871	6773
q19	1383	1011	1065	1011
q20	680	339	351	339
q21	3866	2920	2866	2866
q22	1097	1034	1011	1011
Total cold run time: 110244 ms
Total hot run time: 37891 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4372	4300	4254	4254
q2	385	281	265	265
q3	2932	2643	2662	2643
q4	1964	1711	1679	1679
q5	5623	5690	5730	5690
q6	232	140	134	134
q7	2269	1848	1840	1840
q8	3349	3439	3485	3439
q9	8811	8923	8874	8874
q10	3571	3704	3389	3389
q11	613	530	519	519
q12	838	650	668	650
q13	14091	3240	3286	3240
q14	313	315	288	288
q15	518	497	489	489
q16	631	581	576	576
q17	1846	1553	1514	1514
q18	8172	7903	8024	7903
q19	1730	1680	1445	1445
q20	2180	1911	2004	1911
q21	5840	5451	5575	5451
q22	1125	1037	1046	1037
Total cold run time: 71405 ms
Total hot run time: 57230 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192484 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3f3d16eac8a58c8bff61c1bb53994f19f2e2662b, data reload: false

query1	1250	867	864	864
query2	6203	1869	1861	1861
query3	10610	4091	4004	4004
query4	59771	25395	23338	23338
query5	5241	511	493	493
query6	402	183	161	161
query7	5763	307	294	294
query8	309	222	211	211
query9	8944	2494	2489	2489
query10	507	275	262	262
query11	17948	15019	15308	15019
query12	151	102	102	102
query13	1589	420	400	400
query14	10962	7374	7497	7374
query15	223	175	193	175
query16	7107	436	467	436
query17	1115	574	579	574
query18	1972	299	299	299
query19	283	152	152	152
query20	121	111	116	111
query21	205	104	105	104
query22	4559	4359	4399	4359
query23	34292	33673	33009	33009
query24	5965	2937	2850	2850
query25	540	404	398	398
query26	653	152	151	151
query27	1733	273	276	273
query28	3886	2075	2054	2054
query29	646	403	406	403
query30	232	156	155	155
query31	926	765	760	760
query32	75	52	57	52
query33	441	287	290	287
query34	881	476	464	464
query35	839	729	730	729
query36	1063	933	928	928
query37	144	90	87	87
query38	4045	3872	3910	3872
query39	1461	1383	1383	1383
query40	200	119	115	115
query41	50	49	47	47
query42	116	98	99	98
query43	506	479	469	469
query44	1074	753	741	741
query45	201	172	167	167
query46	1098	746	739	739
query47	1912	1810	1808	1808
query48	391	306	305	305
query49	779	464	467	464
query50	849	413	425	413
query51	7060	6857	6960	6857
query52	101	89	89	89
query53	253	189	187	187
query54	577	461	472	461
query55	79	78	79	78
query56	287	270	297	270
query57	1200	1060	1069	1060
query58	233	238	244	238
query59	3014	2779	2687	2687
query60	304	280	280	280
query61	121	118	117	117
query62	751	669	650	650
query63	218	189	189	189
query64	2456	753	749	749
query65	3223	3133	3224	3133
query66	682	348	370	348
query67	15288	15111	15030	15030
query68	3332	606	575	575
query69	402	279	288	279
query70	1137	1148	1089	1089
query71	355	290	281	281
query72	6452	4223	4146	4146
query73	753	324	331	324
query74	9296	8778	8824	8778
query75	3381	2645	2662	2645
query76	1571	974	1011	974
query77	592	316	330	316
query78	9801	9004	9061	9004
query79	2755	541	550	541
query80	1150	508	496	496
query81	569	242	236	236
query82	605	144	152	144
query83	178	148	150	148
query84	261	78	75	75
query85	912	312	282	282
query86	423	289	302	289
query87	4545	4308	4296	4296
query88	3917	2375	2363	2363
query89	383	288	289	288
query90	1847	191	187	187
query91	120	96	97	96
query92	65	48	48	48
query93	2510	545	535	535
query94	842	303	311	303
query95	353	255	259	255
query96	607	269	276	269
query97	3163	3029	3116	3029
query98	213	201	197	197
query99	1571	1270	1275	1270
Total cold run time: 310141 ms
Total hot run time: 192484 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.38 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3f3d16eac8a58c8bff61c1bb53994f19f2e2662b, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.65	0.08	0.08
query5	0.50	0.50	0.51
query6	1.18	0.74	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.50	0.50
query10	0.55	0.58	0.55
query11	0.15	0.12	0.11
query12	0.16	0.13	0.12
query13	0.60	0.59	0.59
query14	2.07	2.14	2.05
query15	0.85	0.82	0.84
query16	0.36	0.38	0.37
query17	1.06	1.04	1.07
query18	0.20	0.20	0.20
query19	1.97	1.83	1.81
query20	0.02	0.01	0.01
query21	15.38	0.67	0.67
query22	4.24	7.25	2.13
query23	18.32	1.36	1.27
query24	2.15	0.22	0.22
query25	0.14	0.08	0.08
query26	0.28	0.17	0.18
query27	0.08	0.08	0.08
query28	13.25	1.01	1.00
query29	12.57	3.34	3.33
query30	0.24	0.06	0.05
query31	2.87	0.40	0.40
query32	3.27	0.47	0.48
query33	2.94	3.03	2.98
query34	17.18	4.39	4.40
query35	4.48	4.44	4.46
query36	0.65	0.48	0.46
query37	0.19	0.17	0.15
query38	0.16	0.14	0.15
query39	0.05	0.04	0.04
query40	0.16	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 111.08 s
Total hot run time: 32.38 s

@hust-hhb
Copy link
Contributor Author

hust-hhb commented Sep 5, 2024

run buildall

Copy link
Contributor

github-actions bot commented Sep 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38844 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8492f6425acadaceae6ca5be9c88dcc914b9fb8b, data reload: false

------ Round 1 ----------------------------------
q1	18185	4610	4433	4433
q2	2621	190	186	186
q3	11009	1158	1141	1141
q4	10497	774	726	726
q5	7777	2983	2876	2876
q6	233	143	145	143
q7	982	641	651	641
q8	9557	2106	2086	2086
q9	7122	6568	6559	6559
q10	6991	2210	2234	2210
q11	451	242	240	240
q12	394	230	228	228
q13	17755	3062	3066	3062
q14	284	246	247	246
q15	538	509	482	482
q16	518	453	440	440
q17	971	749	802	749
q18	7427	7012	6934	6934
q19	1398	1080	1133	1080
q20	676	342	341	341
q21	4066	3204	3053	3053
q22	1088	988	1007	988
Total cold run time: 110540 ms
Total hot run time: 38844 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4311	4291	4296	4291
q2	372	281	262	262
q3	2911	2679	2660	2660
q4	1925	1667	1664	1664
q5	5403	5411	5387	5387
q6	229	133	133	133
q7	2175	1721	1768	1721
q8	3230	3406	3363	3363
q9	8492	8499	8461	8461
q10	3450	3187	3217	3187
q11	605	516	512	512
q12	789	611	618	611
q13	8106	3058	3043	3043
q14	305	270	274	270
q15	531	476	492	476
q16	519	497	474	474
q17	1816	1496	1485	1485
q18	7690	7396	7358	7358
q19	1699	1530	1644	1530
q20	2091	1826	1814	1814
q21	5427	5338	5156	5156
q22	1108	1042	1026	1026
Total cold run time: 63184 ms
Total hot run time: 54884 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187247 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8492f6425acadaceae6ca5be9c88dcc914b9fb8b, data reload: false

query1	919	377	375	375
query2	6484	1914	1857	1857
query3	6652	207	222	207
query4	27838	23091	23136	23091
query5	4160	508	505	505
query6	252	165	168	165
query7	4575	315	304	304
query8	274	217	231	217
query9	8599	2474	2477	2474
query10	449	290	278	278
query11	17885	15016	15108	15016
query12	150	101	106	101
query13	1629	382	369	369
query14	9852	7514	7061	7061
query15	263	166	185	166
query16	8120	460	472	460
query17	1593	574	565	565
query18	2150	316	280	280
query19	183	141	145	141
query20	117	112	106	106
query21	211	107	98	98
query22	4363	4085	4274	4085
query23	33772	33253	33207	33207
query24	11222	2872	2886	2872
query25	624	371	382	371
query26	1301	155	158	155
query27	2901	274	274	274
query28	7727	2054	2058	2054
query29	875	408	402	402
query30	310	159	148	148
query31	975	754	793	754
query32	99	55	54	54
query33	758	288	287	287
query34	989	480	476	476
query35	842	733	717	717
query36	1071	922	896	896
query37	154	84	77	77
query38	3929	3954	3837	3837
query39	1452	1397	1383	1383
query40	273	112	115	112
query41	48	47	47	47
query42	137	96	99	96
query43	501	465	460	460
query44	1285	763	747	747
query45	199	163	169	163
query46	1093	718	735	718
query47	1870	1754	1781	1754
query48	375	305	299	299
query49	1088	430	432	430
query50	820	408	422	408
query51	7061	6875	6889	6875
query52	96	84	86	84
query53	253	182	183	182
query54	905	453	452	452
query55	75	77	76	76
query56	286	250	255	250
query57	1211	1064	1059	1059
query58	237	229	232	229
query59	2928	2768	2746	2746
query60	330	262	264	262
query61	100	97	99	97
query62	829	658	646	646
query63	222	182	185	182
query64	5245	683	653	653
query65	3217	3140	3153	3140
query66	1436	354	387	354
query67	15479	15348	15410	15348
query68	3571	588	581	581
query69	385	274	291	274
query70	1165	1079	1127	1079
query71	341	281	276	276
query72	6267	4161	3976	3976
query73	752	333	329	329
query74	9131	8867	8837	8837
query75	3367	2679	2635	2635
query76	1944	1006	962	962
query77	456	304	320	304
query78	9962	9027	8976	8976
query79	1041	530	522	522
query80	695	498	499	498
query81	449	234	231	231
query82	236	141	142	141
query83	170	154	147	147
query84	223	79	74	74
query85	730	287	269	269
query86	302	297	294	294
query87	4371	4253	4487	4253
query88	2945	2354	2349	2349
query89	391	291	283	283
query90	1965	195	197	195
query91	132	103	103	103
query92	63	51	49	49
query93	1031	538	533	533
query94	687	305	296	296
query95	344	265	253	253
query96	585	274	265	265
query97	3156	3065	3108	3065
query98	233	196	205	196
query99	1475	1248	1269	1248
Total cold run time: 281883 ms
Total hot run time: 187247 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8492f6425acadaceae6ca5be9c88dcc914b9fb8b, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.05
query3	0.23	0.04	0.04
query4	1.68	0.08	0.08
query5	0.53	0.50	0.51
query6	1.14	0.74	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.54	0.50	0.49
query10	0.55	0.57	0.52
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.60	0.59	0.58
query14	1.39	1.42	1.44
query15	0.85	0.80	0.83
query16	0.35	0.38	0.38
query17	1.03	1.08	0.97
query18	0.21	0.20	0.22
query19	1.91	1.82	1.87
query20	0.02	0.01	0.01
query21	15.43	0.66	0.66
query22	4.65	6.50	2.03
query23	18.25	1.38	1.33
query24	2.07	0.24	0.22
query25	0.15	0.08	0.08
query26	0.26	0.18	0.17
query27	0.07	0.07	0.08
query28	13.23	1.01	1.01
query29	12.63	3.34	3.28
query30	0.24	0.05	0.06
query31	2.88	0.40	0.39
query32	3.25	0.48	0.48
query33	2.98	3.01	2.95
query34	17.03	4.40	4.43
query35	4.44	4.41	4.44
query36	0.66	0.48	0.48
query37	0.19	0.15	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.16	0.13	0.13
query41	0.09	0.04	0.04
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 110.5 s
Total hot run time: 31.5 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.78% (9381/25507)
Line Coverage: 28.21% (77350/274190)
Region Coverage: 27.62% (39939/144585)
Branch Coverage: 24.27% (20328/83754)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8492f6425acadaceae6ca5be9c88dcc914b9fb8b_8492f6425acadaceae6ca5be9c88dcc914b9fb8b/report/index.html

@hust-hhb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.90% (9456/25629)
Line Coverage: 28.26% (77808/275306)
Region Coverage: 27.66% (40156/145195)
Branch Coverage: 24.27% (20405/84084)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bcc07001b56b1730fa62848bc9c748933768348a_bcc07001b56b1730fa62848bc9c748933768348a/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 39815 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bcc07001b56b1730fa62848bc9c748933768348a, data reload: false

------ Round 1 ----------------------------------
q1	17640	4289	4295	4289
q2	2039	186	181	181
q3	10480	1289	1380	1289
q4	10351	980	1020	980
q5	7729	3137	3139	3137
q6	227	145	140	140
q7	1057	636	634	634
q8	9454	1998	2014	1998
q9	6823	6313	6316	6313
q10	7009	2488	2460	2460
q11	431	252	251	251
q12	408	240	238	238
q13	17763	3033	3105	3033
q14	286	245	251	245
q15	542	505	481	481
q16	512	427	418	418
q17	969	940	941	940
q18	7394	6695	6941	6695
q19	1381	1231	1239	1231
q20	596	343	333	333
q21	3916	3549	3551	3549
q22	1115	1041	980	980
Total cold run time: 108122 ms
Total hot run time: 39815 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4218	4257	4183	4183
q2	340	240	239	239
q3	2926	2897	2886	2886
q4	1989	1975	2028	1975
q5	5762	5734	5782	5734
q6	232	139	141	139
q7	2155	1781	1779	1779
q8	3365	3374	3404	3374
q9	8844	8840	8842	8840
q10	3536	3595	3652	3595
q11	596	499	486	486
q12	818	614	633	614
q13	10810	3279	3100	3100
q14	337	283	266	266
q15	553	493	505	493
q16	501	487	461	461
q17	1831	1753	1751	1751
q18	8580	7867	8145	7867
q19	1766	1750	1754	1750
q20	2164	1913	1905	1905
q21	5865	5680	5659	5659
q22	1156	1024	1026	1024
Total cold run time: 68344 ms
Total hot run time: 58120 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199326 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bcc07001b56b1730fa62848bc9c748933768348a, data reload: false

query1	1251	886	905	886
query2	6238	1749	1760	1749
query3	10807	3906	3999	3906
query4	55858	25315	24020	24020
query5	5120	562	541	541
query6	352	173	171	171
query7	5675	320	305	305
query8	292	234	223	223
query9	6100	2452	2463	2452
query10	416	292	275	275
query11	15861	15488	15479	15479
query12	157	112	102	102
query13	1445	392	375	375
query14	10241	6880	7374	6880
query15	204	178	184	178
query16	7009	527	481	481
query17	1172	590	602	590
query18	1882	313	299	299
query19	202	155	156	155
query20	136	111	114	111
query21	212	109	106	106
query22	4816	4416	4464	4416
query23	34549	34149	33505	33505
query24	6209	3096	3108	3096
query25	507	388	380	380
query26	608	156	155	155
query27	1580	282	283	282
query28	2789	2017	1998	1998
query29	663	427	421	421
query30	223	150	151	150
query31	943	738	797	738
query32	78	53	57	53
query33	442	308	301	301
query34	899	480	491	480
query35	850	743	737	737
query36	1063	913	939	913
query37	144	93	86	86
query38	4122	3902	3912	3902
query39	1472	1417	1408	1408
query40	203	113	113	113
query41	46	45	46	45
query42	123	98	99	98
query43	463	446	418	418
query44	1193	768	749	749
query45	197	174	172	172
query46	1089	862	832	832
query47	1897	1790	1796	1790
query48	357	292	286	286
query49	727	456	448	448
query50	959	453	446	446
query51	6986	6925	6944	6925
query52	103	90	87	87
query53	254	189	187	187
query54	567	466	473	466
query55	79	79	78	78
query56	293	270	284	270
query57	1208	1061	1086	1061
query58	233	243	248	243
query59	2725	2459	2381	2381
query60	322	285	304	285
query61	105	101	98	98
query62	800	687	673	673
query63	222	193	195	193
query64	1633	679	676	676
query65	3252	3180	3215	3180
query66	678	341	338	338
query67	15892	15447	15505	15447
query68	1873	869	848	848
query69	427	340	340	340
query70	1170	1137	1144	1137
query71	354	353	359	353
query72	4703	3583	3541	3541
query73	589	581	587	581
query74	9106	9241	9062	9062
query75	3113	3017	3055	3017
query76	959	887	873	873
query77	498	431	434	431
query78	9446	9316	9381	9316
query79	936	889	886	886
query80	869	884	857	857
query81	444	269	274	269
query82	273	271	276	271
query83	202	195	312	195
query84	209	151	109	109
query85	649	421	412	412
query86	324	330	316	316
query87	4490	4353	4416	4353
query88	4088	4024	4008	4008
query89	380	369	374	369
query90	1400	317	311	311
query91	125	128	129	128
query92	82	80	79	79
query93	1033	1034	1033	1033
query94	551	364	404	364
query95	453	443	436	436
query96	471	470	473	470
query97	3127	3145	3124	3124
query98	244	231	234	231
query99	1382	1323	1316	1316
Total cold run time: 289163 ms
Total hot run time: 199326 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.14 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bcc07001b56b1730fa62848bc9c748933768348a, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.04	0.04
query3	0.22	0.04	0.05
query4	1.68	0.07	0.06
query5	0.49	0.50	0.50
query6	1.14	0.72	0.72
query7	0.02	0.02	0.02
query8	0.06	0.05	0.04
query9	0.56	0.51	0.50
query10	0.58	0.58	0.55
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.62	0.60	0.60
query14	1.44	1.46	1.46
query15	0.89	0.88	0.87
query16	0.35	0.37	0.37
query17	1.04	1.04	1.00
query18	0.16	0.17	0.17
query19	1.90	1.88	1.86
query20	0.01	0.01	0.01
query21	15.40	0.68	0.68
query22	3.86	7.35	1.35
query23	18.02	1.30	1.31
query24	2.26	0.23	0.22
query25	0.19	0.08	0.07
query26	0.28	0.18	0.18
query27	0.07	0.08	0.08
query28	13.16	1.01	0.99
query29	12.57	3.36	3.36
query30	0.24	0.06	0.06
query31	2.89	0.42	0.42
query32	3.24	0.49	0.50
query33	3.01	3.07	3.06
query34	15.46	4.32	4.33
query35	4.37	4.37	4.35
query36	0.69	0.49	0.49
query37	0.20	0.17	0.16
query38	0.16	0.15	0.16
query39	0.04	0.05	0.04
query40	0.17	0.14	0.14
query41	0.09	0.05	0.05
query42	0.06	0.05	0.04
query43	0.05	0.05	0.04
Total cold run time: 108.06 s
Total hot run time: 31.14 s

@hust-hhb hust-hhb changed the title [fix](cloud-mow) compaciton should not use the same lock id when geting mow table lock from ms [fix](cloud-mow) Fix the issue of compaction holding mow table lock for a long time Sep 12, 2024
@hust-hhb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.89% (9462/25647)
Line Coverage: 28.25% (77785/275352)
Region Coverage: 27.66% (40175/145241)
Branch Coverage: 24.28% (20418/84110)
Coverage Report: http://coverage.selectdb-in.cc/coverage/39693808c042a34b6fd2621a5230b0c97dd110ec_39693808c042a34b6fd2621a5230b0c97dd110ec/report/index.html

Copy link
Contributor

github-actions bot commented Nov 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

@hust-hhb
Copy link
Contributor Author

hust-hhb commented Nov 6, 2024

run buildall

Copy link
Contributor

github-actions bot commented Nov 6, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.89% (9848/25989)
Line Coverage: 29.04% (81881/281955)
Region Coverage: 28.28% (42178/149129)
Branch Coverage: 24.85% (21400/86116)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b5b73fb4addf2c0c5bd8d97c44a26ae9fb4ca67e_b5b73fb4addf2c0c5bd8d97c44a26ae9fb4ca67e/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41394 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b5b73fb4addf2c0c5bd8d97c44a26ae9fb4ca67e, data reload: false

------ Round 1 ----------------------------------
q1	17761	7467	7294	7294
q2	2069	179	171	171
q3	10629	1078	1193	1078
q4	10554	882	818	818
q5	7754	3084	3037	3037
q6	236	150	151	150
q7	1027	612	599	599
q8	9376	2029	2038	2029
q9	6562	6422	6465	6422
q10	7084	2410	2434	2410
q11	464	259	256	256
q12	400	214	218	214
q13	17792	3020	3041	3020
q14	240	203	220	203
q15	572	526	529	526
q16	648	599	582	582
q17	993	612	558	558
q18	7270	6711	6680	6680
q19	1332	995	1006	995
q20	479	186	180	180
q21	3999	3174	3182	3174
q22	1122	1033	998	998
Total cold run time: 108363 ms
Total hot run time: 41394 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7269	7342	7266	7266
q2	344	251	258	251
q3	3011	2918	2940	2918
q4	2087	1848	1803	1803
q5	5744	5748	5808	5748
q6	230	144	146	144
q7	2288	1849	1841	1841
q8	3396	3595	3477	3477
q9	8904	8885	8879	8879
q10	3597	3581	3550	3550
q11	597	497	501	497
q12	835	641	646	641
q13	11291	3217	3211	3211
q14	305	277	290	277
q15	602	560	551	551
q16	696	648	653	648
q17	1873	1646	1610	1610
q18	8355	7927	7701	7701
q19	1774	1639	1629	1629
q20	2133	1861	1895	1861
q21	5536	5605	5521	5521
q22	1123	1039	1068	1039
Total cold run time: 71990 ms
Total hot run time: 61063 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194842 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b5b73fb4addf2c0c5bd8d97c44a26ae9fb4ca67e, data reload: false

query1	2357	2085	2221	2085
query2	6263	2135	2067	2067
query3	14827	11301	227	227
query4	32955	23578	23518	23518
query5	3651	459	447	447
query6	293	187	185	185
query7	3973	299	298	298
query8	284	236	230	230
query9	9299	2675	2648	2648
query10	452	254	247	247
query11	17948	15388	15190	15190
query12	152	100	100	100
query13	1602	448	412	412
query14	9551	6689	7734	6689
query15	248	187	196	187
query16	8159	494	521	494
query17	1615	590	594	590
query18	2472	632	643	632
query19	383	190	208	190
query20	127	118	114	114
query21	214	106	106	106
query22	4714	4495	4198	4198
query23	34529	34209	34103	34103
query24	11638	3393	3372	3372
query25	629	409	417	409
query26	1236	181	182	181
query27	2217	301	288	288
query28	7459	2434	2399	2399
query29	875	432	419	419
query30	404	308	307	307
query31	1005	812	810	810
query32	95	55	59	55
query33	788	292	278	278
query34	917	517	511	511
query35	882	784	763	763
query36	1103	972	989	972
query37	139	75	79	75
query38	4416	4223	4429	4223
query39	1466	1413	1439	1413
query40	208	100	99	99
query41	54	45	47	45
query42	107	95	99	95
query43	533	515	510	510
query44	1300	825	821	821
query45	180	168	172	168
query46	1153	694	700	694
query47	1951	1844	1886	1844
query48	432	336	327	327
query49	908	383	403	383
query50	806	411	420	411
query51	7411	7142	7124	7124
query52	97	90	88	88
query53	254	182	182	182
query54	1159	451	438	438
query55	76	74	74	74
query56	257	239	233	233
query57	1261	1153	1138	1138
query58	226	208	199	199
query59	3166	3124	3038	3038
query60	270	253	247	247
query61	106	107	109	107
query62	863	677	668	668
query63	215	196	187	187
query64	4171	653	634	634
query65	3324	3226	3219	3219
query66	855	330	302	302
query67	16174	15873	15807	15807
query68	4656	579	593	579
query69	446	256	250	250
query70	1202	1065	1153	1065
query71	371	253	253	253
query72	6330	3995	4069	3995
query73	783	360	375	360
query74	10215	9053	9138	9053
query75	3440	2692	2691	2691
query76	2315	1205	1111	1111
query77	369	289	291	289
query78	10396	9369	9378	9369
query79	1319	598	597	597
query80	869	422	437	422
query81	522	241	238	238
query82	1330	113	115	113
query83	230	160	174	160
query84	240	79	76	76
query85	1032	302	291	291
query86	315	310	275	275
query87	4746	4745	4776	4745
query88	3073	2233	2202	2202
query89	414	287	289	287
query90	2101	196	191	191
query91	141	105	115	105
query92	61	51	48	48
query93	1425	552	547	547
query94	780	301	297	297
query95	366	254	255	254
query96	628	279	283	279
query97	2937	2689	2703	2689
query98	218	206	199	199
query99	1550	1298	1365	1298
Total cold run time: 303674 ms
Total hot run time: 194842 ms

@hust-hhb
Copy link
Contributor Author

hust-hhb commented Nov 6, 2024

run buildall

Copy link
Contributor

github-actions bot commented Nov 6, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 41995 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2463af298f2c898fea6b94c7c700faa91cf638a, data reload: false

------ Round 1 ----------------------------------
q1	17587	7745	7493	7493
q2	2069	188	177	177
q3	10720	1177	1187	1177
q4	10554	871	856	856
q5	7709	3137	3126	3126
q6	241	148	150	148
q7	1015	615	612	612
q8	9378	2004	2031	2004
q9	6640	6430	6464	6430
q10	7074	2429	2416	2416
q11	455	258	261	258
q12	421	224	220	220
q13	17764	3037	3026	3026
q14	253	221	221	221
q15	577	541	518	518
q16	651	596	583	583
q17	981	550	617	550
q18	7264	6675	6893	6675
q19	1333	976	1049	976
q20	490	189	185	185
q21	4073	3332	3328	3328
q22	1137	1032	1016	1016
Total cold run time: 108386 ms
Total hot run time: 41995 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7502	7405	7384	7384
q2	348	259	257	257
q3	3013	2852	2799	2799
q4	2034	1740	1795	1740
q5	5461	5564	5512	5512
q6	230	144	147	144
q7	2175	1754	1713	1713
q8	3296	3455	3465	3455
q9	8644	8606	8661	8606
q10	3557	3513	3456	3456
q11	606	498	502	498
q12	835	577	614	577
q13	7959	3052	3045	3045
q14	296	257	258	257
q15	582	548	546	546
q16	671	658	643	643
q17	1828	1624	1604	1604
q18	7829	7600	7603	7600
q19	1682	1458	1532	1458
q20	2084	1822	1789	1789
q21	5378	5218	5340	5218
q22	1150	1028	1021	1021
Total cold run time: 67160 ms
Total hot run time: 59322 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.88% (9850/26000)
Line Coverage: 29.04% (81915/282038)
Region Coverage: 28.28% (42183/149168)
Branch Coverage: 24.85% (21405/86132)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a2463af298f2c898fea6b94c7c700faa91cf638a_a2463af298f2c898fea6b94c7c700faa91cf638a/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 193752 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a2463af298f2c898fea6b94c7c700faa91cf638a, data reload: false

query1	980	384	362	362
query2	6541	2164	2135	2135
query3	6789	219	214	214
query4	34096	23694	23654	23654
query5	4450	440	439	439
query6	272	190	185	185
query7	4625	298	293	293
query8	293	235	229	229
query9	9436	2668	2667	2667
query10	491	251	253	251
query11	18231	15380	15226	15226
query12	166	102	101	101
query13	1682	429	406	406
query14	10362	6652	6863	6652
query15	262	172	180	172
query16	7818	466	474	466
query17	1605	578	559	559
query18	2199	614	605	605
query19	334	186	179	179
query20	118	110	110	110
query21	203	105	105	105
query22	4581	4415	4342	4342
query23	35648	34346	34354	34346
query24	11698	3380	3458	3380
query25	596	409	412	409
query26	1200	188	180	180
query27	2789	284	281	281
query28	8035	2458	2464	2458
query29	708	432	431	431
query30	461	318	322	318
query31	1076	811	840	811
query32	105	59	61	59
query33	785	283	289	283
query34	974	519	527	519
query35	918	733	748	733
query36	1110	966	966	966
query37	130	73	78	73
query38	4416	4338	4327	4327
query39	1479	1444	1404	1404
query40	289	107	107	107
query41	50	48	47	47
query42	109	107	98	98
query43	538	495	506	495
query44	1244	831	822	822
query45	199	174	178	174
query46	1168	724	706	706
query47	1965	1828	1822	1822
query48	431	319	343	319
query49	1279	407	394	394
query50	811	395	405	395
query51	7314	7096	7148	7096
query52	105	90	90	90
query53	270	188	185	185
query54	1252	439	422	422
query55	84	77	77	77
query56	274	254	262	254
query57	1291	1150	1166	1150
query58	258	200	205	200
query59	3282	3076	3216	3076
query60	274	251	256	251
query61	105	108	105	105
query62	881	678	678	678
query63	218	191	192	191
query64	5390	649	621	621
query65	3344	3219	3201	3201
query66	1377	312	321	312
query67	16152	15672	15667	15667
query68	5130	588	589	588
query69	441	283	256	256
query70	1204	1153	1140	1140
query71	435	257	249	249
query72	6144	4056	3909	3909
query73	791	357	367	357
query74	10604	9104	9162	9104
query75	3377	2665	2675	2665
query76	2894	1157	1188	1157
query77	409	272	274	272
query78	11231	9836	9453	9453
query79	1729	600	632	600
query80	943	466	441	441
query81	568	246	239	239
query82	778	119	116	116
query83	232	167	161	161
query84	230	76	71	71
query85	804	304	293	293
query86	416	317	307	307
query87	4818	4747	4889	4747
query88	3231	2235	2164	2164
query89	397	286	295	286
query90	2154	190	192	190
query91	141	106	109	106
query92	63	52	50	50
query93	1357	566	559	559
query94	924	293	293	293
query95	351	251	246	246
query96	615	274	274	274
query97	2863	2711	2665	2665
query98	215	204	203	203
query99	1544	1306	1315	1306
Total cold run time: 304649 ms
Total hot run time: 193752 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a2463af298f2c898fea6b94c7c700faa91cf638a, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.02	0.03
query3	0.23	0.07	0.07
query4	1.63	0.10	0.11
query5	0.42	0.39	0.39
query6	1.16	0.64	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.55	0.50	0.50
query10	0.53	0.54	0.58
query11	0.14	0.11	0.10
query12	0.13	0.11	0.12
query13	0.61	0.60	0.60
query14	2.69	2.80	2.83
query15	0.90	0.83	0.83
query16	0.40	0.38	0.40
query17	1.06	1.06	1.05
query18	0.24	0.23	0.22
query19	1.95	1.89	1.99
query20	0.01	0.01	0.01
query21	15.35	0.62	0.59
query22	2.62	2.54	2.00
query23	17.17	0.91	0.67
query24	3.31	1.06	0.38
query25	0.20	0.12	0.04
query26	0.49	0.14	0.14
query27	0.04	0.05	0.05
query28	11.08	1.09	1.07
query29	12.53	3.26	3.24
query30	0.24	0.06	0.06
query31	2.86	0.38	0.38
query32	3.29	0.46	0.47
query33	2.94	3.03	3.02
query34	16.80	4.52	4.57
query35	4.60	4.56	4.58
query36	0.66	0.48	0.48
query37	0.09	0.06	0.05
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.13	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.04
Total cold run time: 107.46 s
Total hot run time: 32.28 s

@hust-hhb
Copy link
Contributor Author

hust-hhb commented Nov 6, 2024

run buildall

Copy link
Contributor

github-actions bot commented Nov 6, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.88% (9848/25998)
Line Coverage: 29.04% (81898/282028)
Region Coverage: 28.27% (42177/149168)
Branch Coverage: 24.84% (21392/86132)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e67f83238049ea7a35fae194b4281136454996d1_e67f83238049ea7a35fae194b4281136454996d1/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41174 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e67f83238049ea7a35fae194b4281136454996d1, data reload: false

------ Round 1 ----------------------------------
q1	17968	8637	7273	7273
q2	2076	171	170	170
q3	10719	1084	1216	1084
q4	10538	845	866	845
q5	7742	3092	3067	3067
q6	234	150	145	145
q7	1012	621	617	617
q8	9372	1990	2064	1990
q9	6557	6408	6459	6408
q10	7083	2391	2475	2391
q11	461	261	263	261
q12	414	215	217	215
q13	17781	2991	3004	2991
q14	238	214	215	214
q15	575	497	517	497
q16	657	568	604	568
q17	977	532	559	532
q18	7401	6689	6627	6627
q19	1321	1000	1040	1000
q20	472	186	177	177
q21	3938	3126	3185	3126
q22	1095	976	993	976
Total cold run time: 108631 ms
Total hot run time: 41174 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7304	7245	7254	7245
q2	355	245	244	244
q3	3031	2973	2922	2922
q4	2059	1798	1760	1760
q5	5773	5786	5793	5786
q6	242	145	149	145
q7	2254	1837	1771	1771
q8	3419	3558	3439	3439
q9	8981	8964	8867	8867
q10	3589	3555	3545	3545
q11	607	517	508	508
q12	809	646	638	638
q13	11486	3248	3175	3175
q14	319	274	269	269
q15	605	534	561	534
q16	683	657	642	642
q17	1883	1638	1598	1598
q18	8453	7786	7767	7767
q19	1697	1587	1546	1546
q20	2134	1860	1870	1860
q21	5642	5435	5413	5413
q22	1165	1047	1056	1047
Total cold run time: 72490 ms
Total hot run time: 60721 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194206 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e67f83238049ea7a35fae194b4281136454996d1, data reload: false

query1	2326	2216	2093	2093
query2	6228	2094	2020	2020
query3	14749	11175	233	233
query4	33283	23554	23528	23528
query5	3288	450	447	447
query6	276	187	188	187
query7	3973	290	304	290
query8	287	230	228	228
query9	9477	2667	2637	2637
query10	453	250	245	245
query11	17985	15160	15362	15160
query12	169	102	102	102
query13	1572	427	411	411
query14	9221	7146	7573	7146
query15	251	182	189	182
query16	8246	469	494	469
query17	1687	603	577	577
query18	2471	621	621	621
query19	400	207	175	175
query20	125	116	116	116
query21	211	99	102	99
query22	4919	4262	4325	4262
query23	34690	33894	34103	33894
query24	11437	3306	3311	3306
query25	604	363	372	363
query26	1538	169	165	165
query27	2526	282	281	281
query28	7586	2420	2391	2391
query29	916	400	397	397
query30	389	303	308	303
query31	999	779	800	779
query32	83	52	51	51
query33	759	263	264	263
query34	1007	499	528	499
query35	873	743	722	722
query36	1105	965	964	964
query37	131	73	71	71
query38	4549	4234	4325	4234
query39	1435	1432	1404	1404
query40	245	96	95	95
query41	44	43	44	43
query42	109	95	95	95
query43	537	513	493	493
query44	1307	807	821	807
query45	178	163	165	163
query46	1121	704	692	692
query47	1922	1814	1897	1814
query48	413	324	320	320
query49	1004	375	362	362
query50	809	381	404	381
query51	7166	7109	7176	7109
query52	97	86	85	85
query53	249	177	176	176
query54	1364	403	413	403
query55	80	75	77	75
query56	257	241	236	236
query57	1283	1155	1158	1155
query58	220	231	206	206
query59	3256	3042	3187	3042
query60	276	246	249	246
query61	109	110	116	110
query62	872	694	664	664
query63	217	197	185	185
query64	5020	759	706	706
query65	3291	3219	3214	3214
query66	1399	315	317	315
query67	16091	15769	15692	15692
query68	4798	599	588	588
query69	439	258	256	256
query70	1201	1141	1068	1068
query71	406	256	249	249
query72	6402	4165	3950	3950
query73	772	351	353	351
query74	10398	9080	8949	8949
query75	3385	2685	2671	2671
query76	2658	1071	1076	1071
query77	390	278	261	261
query78	10238	9420	9348	9348
query79	2355	592	606	592
query80	918	412	435	412
query81	557	235	240	235
query82	1160	116	117	116
query83	237	158	155	155
query84	235	74	76	74
query85	1645	310	299	299
query86	478	301	294	294
query87	4763	4749	4730	4730
query88	4050	2185	2169	2169
query89	420	288	289	288
query90	2059	187	187	187
query91	134	105	103	103
query92	62	48	51	48
query93	2597	550	557	550
query94	916	286	296	286
query95	344	249	242	242
query96	625	281	287	281
query97	2918	2695	2737	2695
query98	228	203	194	194
query99	1525	1301	1295	1295
Total cold run time: 310473 ms
Total hot run time: 194206 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.04 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e67f83238049ea7a35fae194b4281136454996d1, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.05	0.05
query3	0.23	0.05	0.06
query4	1.65	0.10	0.10
query5	0.40	0.40	0.40
query6	1.17	0.66	0.67
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.49
query10	0.54	0.57	0.58
query11	0.17	0.12	0.12
query12	0.15	0.13	0.13
query13	0.61	0.59	0.59
query14	2.83	2.83	2.76
query15	0.92	0.84	0.83
query16	0.39	0.39	0.37
query17	1.07	1.07	1.04
query18	0.19	0.18	0.19
query19	1.91	1.85	2.03
query20	0.02	0.01	0.01
query21	15.36	0.67	0.66
query22	4.64	7.19	1.76
query23	18.23	1.38	1.23
query24	2.07	0.23	0.22
query25	0.14	0.08	0.08
query26	0.25	0.17	0.18
query27	0.08	0.08	0.08
query28	13.28	1.15	1.15
query29	12.66	3.37	3.38
query30	0.25	0.05	0.06
query31	2.87	0.41	0.40
query32	3.23	0.49	0.48
query33	3.04	3.11	3.02
query34	16.88	4.50	4.50
query35	4.54	4.53	4.59
query36	0.66	0.47	0.50
query37	0.20	0.17	0.16
query38	0.16	0.14	0.16
query39	0.05	0.04	0.04
query40	0.16	0.13	0.12
query41	0.09	0.06	0.05
query42	0.06	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 111.93 s
Total hot run time: 33.04 s

@hust-hhb
Copy link
Contributor Author

hust-hhb commented Nov 7, 2024

run buildall

Copy link
Contributor

github-actions bot commented Nov 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.87% (9848/26005)
Line Coverage: 29.05% (81967/282156)
Region Coverage: 28.26% (42197/149312)
Branch Coverage: 24.85% (21414/86188)
Coverage Report: http://coverage.selectdb-in.cc/coverage/93da71bf25e1c4ad122a00410fd0aa4e2c7a9a69_93da71bf25e1c4ad122a00410fd0aa4e2c7a9a69/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41223 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 93da71bf25e1c4ad122a00410fd0aa4e2c7a9a69, data reload: false

------ Round 1 ----------------------------------
q1	17892	7547	7283	7283
q2	2075	176	179	176
q3	10617	1075	1120	1075
q4	10591	827	864	827
q5	7725	3081	3109	3081
q6	237	146	143	143
q7	993	615	614	614
q8	9368	1975	2063	1975
q9	6660	6470	6459	6459
q10	7068	2455	2415	2415
q11	457	265	254	254
q12	401	212	211	211
q13	17792	2993	2986	2986
q14	234	215	218	215
q15	585	520	521	520
q16	660	571	585	571
q17	998	568	552	552
q18	7331	6511	6716	6511
q19	1331	990	927	927
q20	489	178	182	178
q21	3986	3262	3295	3262
q22	1151	994	988	988
Total cold run time: 108641 ms
Total hot run time: 41223 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7264	7308	7247	7247
q2	352	251	253	251
q3	3026	2966	2951	2951
q4	2105	1853	1808	1808
q5	5770	5735	5812	5735
q6	224	146	147	146
q7	2236	1837	1809	1809
q8	3430	3580	3462	3462
q9	8974	8980	8904	8904
q10	3579	3575	3539	3539
q11	599	509	495	495
q12	817	637	654	637
q13	10385	3248	3181	3181
q14	314	268	271	268
q15	586	560	544	544
q16	708	635	649	635
q17	1877	1643	1604	1604
q18	8318	7851	7684	7684
q19	1736	1501	1394	1394
q20	2127	1937	1873	1873
q21	5609	5357	5400	5357
q22	1138	1045	1047	1045
Total cold run time: 71174 ms
Total hot run time: 60569 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195362 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 93da71bf25e1c4ad122a00410fd0aa4e2c7a9a69, data reload: false

query1	2288	2092	2104	2092
query2	6240	2137	2103	2103
query3	15057	11036	222	222
query4	33021	23389	23739	23389
query5	3444	439	449	439
query6	268	169	181	169
query7	3965	287	308	287
query8	304	231	231	231
query9	9553	2734	2727	2727
query10	433	254	238	238
query11	17815	15265	15280	15265
query12	156	103	98	98
query13	1598	434	433	433
query14	8500	7453	7322	7322
query15	251	178	185	178
query16	7893	449	483	449
query17	1570	574	588	574
query18	2383	661	621	621
query19	403	198	190	190
query20	124	108	117	108
query21	206	106	106	106
query22	4898	4563	4424	4424
query23	35481	34237	34312	34237
query24	12050	3398	3351	3351
query25	655	422	413	413
query26	1749	173	170	170
query27	2520	282	277	277
query28	7347	2435	2436	2435
query29	1004	428	414	414
query30	406	310	320	310
query31	1094	784	802	784
query32	96	56	56	56
query33	773	274	272	272
query34	1005	515	521	515
query35	881	728	716	716
query36	1112	941	949	941
query37	153	79	70	70
query38	4431	4358	4234	4234
query39	1468	1446	1418	1418
query40	251	99	100	99
query41	47	50	46	46
query42	104	100	97	97
query43	532	516	492	492
query44	1381	814	806	806
query45	184	170	170	170
query46	1136	714	708	708
query47	1965	1830	1860	1830
query48	414	348	319	319
query49	1015	423	393	393
query50	809	382	394	382
query51	7325	7224	7109	7109
query52	96	89	89	89
query53	253	178	180	178
query54	1118	439	397	397
query55	80	74	73	73
query56	236	245	235	235
query57	1284	1151	1136	1136
query58	216	208	196	196
query59	3205	3181	3161	3161
query60	268	233	232	232
query61	107	114	109	109
query62	822	687	688	687
query63	214	190	183	183
query64	4977	635	593	593
query65	3296	3226	3179	3179
query66	1191	294	307	294
query67	16229	15568	15796	15568
query68	4865	589	560	560
query69	401	247	248	247
query70	1063	1143	1117	1117
query71	324	251	248	248
query72	6183	4052	4132	4052
query73	760	364	361	361
query74	10283	9065	9114	9065
query75	3417	2666	2663	2663
query76	2720	1106	1051	1051
query77	387	278	273	273
query78	10336	9450	9474	9450
query79	1906	599	597	597
query80	1119	409	438	409
query81	582	239	242	239
query82	954	121	122	121
query83	248	160	157	157
query84	232	72	72	72
query85	1318	313	288	288
query86	447	308	302	302
query87	4898	4681	4773	4681
query88	3543	2225	2190	2190
query89	413	297	291	291
query90	1859	187	186	186
query91	143	101	101	101
query92	64	51	49	49
query93	2782	543	538	538
query94	768	287	290	287
query95	345	244	246	244
query96	622	288	293	288
query97	2911	2702	2719	2702
query98	226	198	209	198
query99	1544	1306	1296	1296
Total cold run time: 309013 ms
Total hot run time: 195362 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.19 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 93da71bf25e1c4ad122a00410fd0aa4e2c7a9a69, data reload: false

query1	0.03	0.03	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.06
query4	1.67	0.08	0.07
query5	0.42	0.39	0.40
query6	1.16	0.66	0.65
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.56	0.50	0.52
query10	0.59	0.56	0.56
query11	0.17	0.12	0.12
query12	0.15	0.13	0.13
query13	0.61	0.60	0.60
query14	2.72	2.75	2.82
query15	0.91	0.84	0.84
query16	0.39	0.39	0.38
query17	1.02	1.08	1.01
query18	0.18	0.18	0.19
query19	1.85	1.78	1.88
query20	0.01	0.01	0.01
query21	15.36	0.67	0.66
query22	3.77	7.20	1.95
query23	18.27	1.32	1.31
query24	2.18	0.21	0.21
query25	0.15	0.09	0.08
query26	0.27	0.17	0.18
query27	0.08	0.08	0.07
query28	13.24	1.16	1.13
query29	12.65	3.42	3.40
query30	0.26	0.07	0.05
query31	2.87	0.40	0.40
query32	3.24	0.47	0.48
query33	2.95	3.06	3.12
query34	16.56	4.54	4.49
query35	4.55	4.51	4.57
query36	0.67	0.48	0.49
query37	0.18	0.15	0.15
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.16	0.13	0.13
query41	0.10	0.06	0.05
query42	0.06	0.05	0.06
query43	0.05	0.05	0.04
Total cold run time: 110.65 s
Total hot run time: 33.19 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants