Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix]Remove be special handling of date types and bugs in registration functions #45159

Open
wants to merge 36 commits into
base: master
Choose a base branch
from

Conversation

koarz
Copy link
Contributor

@koarz koarz commented Dec 8, 2024

What problem does this PR solve?

  1. be will do something special with datetype in get_function, now remove that part of the code.
  2. Some functions may have the same parameters but have multiple return types, but in the registration of the function does not deal with this situation, which will lead to this type of function only to keep a copy (be end of only one return type), so in the registration of the function to the key_str plus the return value type to distinguish between
  3. If a function's return value type depends on its parameters, and the parameter types can only be determined at runtime, then the has_return_type_in_signature function should be overloaded to ensure that get_function works.
  4. Some of the date functions are not always nullable on the be, so I've changed the nullable handling of the fe functions.
  5. If a function has explicit return type,and has_return_type_in_signature return true (default is true) then we need override get_return_type(const DataTypes) const.
  6. if the function cant return directly we need override ColumnsWithNameAndType version such as
get_return_type(const DataTypes arguments) override {
  if (arguments[0] is null) {
    return Nullable(Type1)  
  }
  if (...) {
    return ...Type1 
  }
  return Type1
}
the function always return Type1, but may do check for arguments's type
please override it to
get_return_type(const ColumnsWithNameAndType arguments) override {
  if (arguments[0].type is null) {
    return Nullable(Type1)  
  }
  if (...) {
    return ...Type1 
  }
  return Type1
}
get_return_type(const DataTypes arguments) override {
  return Type1
}
  1. Add the DataTypeDate type to the Return Type of the macro ADD_TIME_FUNCTION_IMPL
    Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@koarz
Copy link
Contributor Author

koarz commented Dec 8, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.99% (10174/26092)
Line Coverage: 29.65% (84356/284532)
Region Coverage: 28.71% (43296/150786)
Branch Coverage: 25.21% (21915/86932)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ccad421c5072729e13a277000d4e9236994c607b_ccad421c5072729e13a277000d4e9236994c607b/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 39869 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ccad421c5072729e13a277000d4e9236994c607b, data reload: false

------ Round 1 ----------------------------------
q1	17624	7493	7287	7287
q2	2042	179	162	162
q3	10769	1071	1215	1071
q4	10514	728	756	728
q5	7596	2712	2673	2673
q6	237	147	147	147
q7	984	611	607	607
q8	9232	1880	1924	1880
q9	6592	6501	6455	6455
q10	6988	2310	2325	2310
q11	462	262	241	241
q12	426	221	229	221
q13	17757	3018	3021	3018
q14	247	223	208	208
q15	570	548	529	529
q16	639	579	583	579
q17	970	576	556	556
q18	7143	6694	6579	6579
q19	1337	1066	965	965
q20	458	181	179	179
q21	4041	3157	3186	3157
q22	375	317	318	317
Total cold run time: 107003 ms
Total hot run time: 39869 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7254	7273	7232	7232
q2	327	229	229	229
q3	2862	2792	2916	2792
q4	2047	1805	1826	1805
q5	5659	5646	5642	5642
q6	228	142	144	142
q7	2241	1836	1808	1808
q8	3366	3558	3491	3491
q9	9009	9049	9094	9049
q10	3594	3522	3528	3522
q11	605	511	499	499
q12	817	639	612	612
q13	11381	3218	3213	3213
q14	322	286	275	275
q15	577	530	519	519
q16	693	649	655	649
q17	1861	1607	1598	1598
q18	8259	7663	7603	7603
q19	1685	1571	1523	1523
q20	2093	1867	1838	1838
q21	5578	5447	5386	5386
q22	655	563	567	563
Total cold run time: 71113 ms
Total hot run time: 59990 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197769 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ccad421c5072729e13a277000d4e9236994c607b, data reload: false

query1	1250	964	948	948
query2	6230	2080	2039	2039
query3	10951	4439	4404	4404
query4	67345	28907	23662	23662
query5	4949	470	468	468
query6	423	193	188	188
query7	5473	310	305	305
query8	325	250	247	247
query9	8617	2632	2627	2627
query10	418	249	244	244
query11	17068	15200	15974	15200
query12	156	111	103	103
query13	1455	447	431	431
query14	9929	7433	7393	7393
query15	214	174	187	174
query16	7068	457	477	457
query17	1421	563	569	563
query18	1774	308	296	296
query19	201	148	148	148
query20	121	118	111	111
query21	211	103	97	97
query22	4756	4695	4684	4684
query23	34865	34428	34470	34428
query24	5463	2491	2555	2491
query25	523	382	383	382
query26	652	155	159	155
query27	1837	279	288	279
query28	4539	2484	2453	2453
query29	691	412	426	412
query30	208	157	151	151
query31	1036	864	839	839
query32	65	53	56	53
query33	462	277	281	277
query34	952	506	534	506
query35	907	785	762	762
query36	1102	971	964	964
query37	121	71	76	71
query38	4481	4351	4380	4351
query39	1521	1477	1468	1468
query40	198	103	97	97
query41	42	43	42	42
query42	114	97	102	97
query43	536	495	488	488
query44	1248	826	833	826
query45	184	173	170	170
query46	1171	737	725	725
query47	2066	1949	1916	1916
query48	423	306	323	306
query49	717	416	392	392
query50	834	398	397	397
query51	7343	7062	7076	7062
query52	95	83	87	83
query53	252	178	179	178
query54	509	400	405	400
query55	78	74	73	73
query56	277	244	249	244
query57	1263	1100	1137	1100
query58	224	225	217	217
query59	3165	3087	2970	2970
query60	289	253	261	253
query61	133	123	128	123
query62	785	670	702	670
query63	222	197	193	193
query64	1339	674	669	669
query65	3308	3198	3246	3198
query66	691	301	296	296
query67	15941	15675	15737	15675
query68	3723	592	587	587
query69	425	255	246	246
query70	1176	1118	1048	1048
query71	387	252	245	245
query72	6377	4078	4083	4078
query73	765	350	354	350
query74	10204	9067	8964	8964
query75	3385	2664	2659	2659
query76	1839	1128	1126	1126
query77	461	273	265	265
query78	10531	9602	9509	9509
query79	1612	587	593	587
query80	1020	428	434	428
query81	483	231	245	231
query82	1240	120	116	116
query83	159	148	154	148
query84	282	68	69	68
query85	930	296	298	296
query86	330	292	307	292
query87	4771	4582	4696	4582
query88	3734	2158	2127	2127
query89	423	302	286	286
query90	2035	185	190	185
query91	137	100	106	100
query92	61	49	48	48
query93	1922	558	557	557
query94	760	294	282	282
query95	365	247	244	244
query96	617	278	273	273
query97	2895	2666	2675	2666
query98	226	188	195	188
query99	1603	1297	1305	1297
Total cold run time: 318105 ms
Total hot run time: 197769 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.24 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ccad421c5072729e13a277000d4e9236994c607b, data reload: false

query1	0.03	0.03	0.06
query2	0.07	0.03	0.04
query3	0.23	0.08	0.07
query4	1.61	0.10	0.11
query5	0.43	0.44	0.40
query6	1.15	0.64	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.51	0.50
query10	0.57	0.54	0.57
query11	0.13	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.61	0.58
query14	2.74	2.81	2.87
query15	0.90	0.82	0.82
query16	0.39	0.39	0.40
query17	1.05	1.04	1.02
query18	0.22	0.21	0.20
query19	1.90	1.85	1.95
query20	0.01	0.01	0.01
query21	15.37	0.61	0.60
query22	2.30	2.50	2.01
query23	16.88	1.13	0.79
query24	2.99	1.27	1.78
query25	0.27	0.24	0.13
query26	0.48	0.14	0.14
query27	0.06	0.04	0.04
query28	9.81	1.10	1.08
query29	12.56	3.28	3.27
query30	0.25	0.06	0.06
query31	2.88	0.38	0.37
query32	3.28	0.47	0.46
query33	2.97	3.06	3.09
query34	16.83	4.46	4.50
query35	4.50	4.48	4.47
query36	0.66	0.48	0.50
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.39 s
Total hot run time: 33.24 s

@koarz
Copy link
Contributor Author

koarz commented Dec 8, 2024

run p0

Copy link
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some be-ut to show we can't pass the check because of use wrong return type.(to mock FE send a function signature with wrong return type)

@@ -158,19 +167,21 @@ class SimpleFunctionFactory {
FunctionBasePtr get_function(const std::string& name, const ColumnsWithTypeAndName& arguments,
const DataTypePtr& return_type, const FunctionAttr& attr = {},
int be_version = BeExecVersionManager::get_newest_version()) {
std::string key_str = name;
std::string key_str, ori_name = name;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

split definition. use origin_name

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment on these two var

@@ -299,6 +299,10 @@ class FunctionBuilderImpl : public IFunctionBuilder {

ColumnNumbers get_arguments_that_are_always_constant() const override { return {}; }

// if a function's get_variadic_argument_types() not override and get_return_type_impl()
// result is not compile time be sure, the function should override return true
virtual bool dont_append_return_type_name_when_register_function() { return false; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

impl it in IFunctionBase, so we can do check in function factory.
if not possible, add comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the register_function() arg Creator& ptr is a FunctionBuilder so cant impl it in IFunctionBase it move to IFunctionBuilder

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add regression-test about null

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert this

@koarz
Copy link
Contributor Author

koarz commented Dec 14, 2024

run beut

@koarz
Copy link
Contributor Author

koarz commented Dec 14, 2024

run buildall

@koarz
Copy link
Contributor Author

koarz commented Dec 14, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 39.26% (10274/26172)
Line Coverage: 29.83% (85287/285941)
Region Coverage: 28.86% (43719/151503)
Branch Coverage: 25.32% (22122/87380)
Coverage Report: http://coverage.selectdb-in.cc/coverage/de8451ba13307a712d282850d246db34b34899fc_de8451ba13307a712d282850d246db34b34899fc/report/index.html

Copy link
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by anyone and no changes requested.

@koarz
Copy link
Contributor Author

koarz commented Dec 17, 2024

run performance

@doris-robot
Copy link

TPC-H: Total hot run time: 40379 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit de8451ba13307a712d282850d246db34b34899fc, data reload: false

------ Round 1 ----------------------------------
q1	17611	7597	7340	7340
q2	2051	176	175	175
q3	10537	1154	1161	1154
q4	10231	827	717	717
q5	7594	2766	2766	2766
q6	238	147	146	146
q7	1020	635	618	618
q8	9238	1897	1975	1897
q9	6733	6456	6484	6456
q10	6972	2359	2388	2359
q11	471	263	277	263
q12	417	225	221	221
q13	17777	3013	3047	3013
q14	233	207	211	207
q15	573	536	524	524
q16	648	576	589	576
q17	1032	563	534	534
q18	7260	6626	6753	6626
q19	1342	1023	942	942
q20	494	189	183	183
q21	4043	3339	3398	3339
q22	384	331	323	323
Total cold run time: 106899 ms
Total hot run time: 40379 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7277	7265	7267	7265
q2	335	225	246	225
q3	2963	2905	3119	2905
q4	2185	1874	1843	1843
q5	5631	5737	5679	5679
q6	229	140	144	140
q7	2180	1766	1899	1766
q8	3428	3533	3530	3530
q9	9029	9028	8993	8993
q10	3629	3533	3557	3533
q11	604	510	490	490
q12	802	634	596	596
q13	12129	3204	3171	3171
q14	310	292	286	286
q15	570	524	519	519
q16	676	653	641	641
q17	1832	1626	1586	1586
q18	8429	7713	7723	7713
q19	1744	1582	1544	1544
q20	2100	1798	1807	1798
q21	5445	5278	5360	5278
q22	644	553	602	553
Total cold run time: 72171 ms
Total hot run time: 60054 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191119 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit de8451ba13307a712d282850d246db34b34899fc, data reload: false

query1	993	408	388	388
query2	6518	2116	2094	2094
query3	6721	226	212	212
query4	33983	23523	23425	23425
query5	4439	465	468	465
query6	285	206	202	202
query7	4625	299	319	299
query8	313	250	234	234
query9	9347	2696	2696	2696
query10	486	258	254	254
query11	18112	15270	15261	15261
query12	164	104	110	104
query13	1696	452	436	436
query14	9836	7656	7223	7223
query15	294	177	183	177
query16	8216	449	489	449
query17	1822	590	567	567
query18	2153	305	338	305
query19	370	150	147	147
query20	115	114	110	110
query21	213	108	103	103
query22	4391	4301	4132	4132
query23	34438	33840	33926	33840
query24	11581	2429	2512	2429
query25	693	397	382	382
query26	1769	157	155	155
query27	2782	273	275	273
query28	7990	2452	2413	2413
query29	1028	427	431	427
query30	293	150	151	150
query31	1010	811	821	811
query32	103	60	82	60
query33	776	301	289	289
query34	982	554	528	528
query35	886	748	733	733
query36	1108	948	934	934
query37	277	77	77	77
query38	4371	4182	4165	4165
query39	1465	1441	1420	1420
query40	283	101	98	98
query41	49	43	42	42
query42	115	101	104	101
query43	529	503	480	480
query44	1275	811	806	806
query45	176	168	170	168
query46	1149	715	706	706
query47	1944	1841	1864	1841
query48	412	331	323	323
query49	1292	398	397	397
query50	800	384	390	384
query51	7255	7157	7261	7157
query52	99	93	89	89
query53	261	182	186	182
query54	1196	424	412	412
query55	97	79	79	79
query56	260	257	254	254
query57	1246	1139	1152	1139
query58	232	216	247	216
query59	3276	3133	3048	3048
query60	268	248	255	248
query61	108	107	104	104
query62	892	675	688	675
query63	216	188	193	188
query64	5047	703	669	669
query65	3296	3182	3227	3182
query66	1426	308	307	307
query67	15821	15615	15467	15467
query68	4995	566	591	566
query69	419	253	264	253
query70	1155	1140	1163	1140
query71	335	252	250	250
query72	6410	4057	4083	4057
query73	760	367	368	367
query74	10460	8966	8974	8966
query75	3428	2654	2694	2654
query76	2969	1126	1091	1091
query77	553	285	277	277
query78	10377	9523	9382	9382
query79	2373	610	619	610
query80	1020	417	422	417
query81	540	238	232	232
query82	918	119	119	119
query83	283	148	146	146
query84	239	77	73	73
query85	1305	317	310	310
query86	406	305	300	300
query87	4737	4492	4408	4408
query88	4088	2213	2179	2179
query89	406	292	297	292
query90	2213	222	190	190
query91	138	112	105	105
query92	63	52	53	52
query93	1801	551	557	551
query94	1133	298	305	298
query95	367	256	254	254
query96	638	281	288	281
query97	2782	2663	2714	2663
query98	216	211	197	197
query99	1520	1344	1308	1308
Total cold run time: 304711 ms
Total hot run time: 191119 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.52 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit de8451ba13307a712d282850d246db34b34899fc, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.08	0.08
query4	1.60	0.11	0.11
query5	0.42	0.39	0.39
query6	1.14	0.64	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.51	0.51
query10	0.55	0.58	0.55
query11	0.15	0.11	0.11
query12	0.15	0.10	0.11
query13	0.61	0.60	0.61
query14	2.71	2.87	2.78
query15	0.91	0.84	0.82
query16	0.39	0.37	0.38
query17	1.06	0.98	1.02
query18	0.23	0.21	0.20
query19	1.94	1.78	1.83
query20	0.02	0.01	0.01
query21	15.37	0.58	0.57
query22	3.03	2.74	1.59
query23	17.01	0.84	0.76
query24	3.68	1.60	1.24
query25	0.25	0.19	0.09
query26	0.54	0.13	0.13
query27	0.05	0.04	0.04
query28	9.82	1.10	1.06
query29	12.59	3.26	3.26
query30	0.25	0.06	0.06
query31	2.88	0.40	0.36
query32	3.27	0.46	0.46
query33	2.97	2.99	3.06
query34	16.52	4.50	4.54
query35	4.55	4.48	4.50
query36	0.72	0.47	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.83 s
Total hot run time: 32.52 s

@zclllyybb zclllyybb mentioned this pull request Dec 19, 2024
16 tasks
@koarz
Copy link
Contributor Author

koarz commented Dec 27, 2024

run buildall

@koarz
Copy link
Contributor Author

koarz commented Dec 27, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32789 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d5dc10b514cfba4259ad6ad7fca7c71b0942a6e, data reload: false

------ Round 1 ----------------------------------
q1	17594	6145	6049	6049
q2	2046	313	162	162
q3	10409	1324	727	727
q4	10209	872	438	438
q5	7522	2274	2030	2030
q6	206	182	149	149
q7	910	744	622	622
q8	9239	1416	1210	1210
q9	5200	5054	5002	5002
q10	6765	2332	1880	1880
q11	476	274	263	263
q12	347	366	226	226
q13	17788	3609	2981	2981
q14	240	237	214	214
q15	568	498	483	483
q16	651	627	576	576
q17	574	874	332	332
q18	7068	6581	6360	6360
q19	1634	997	583	583
q20	321	323	185	185
q21	3062	2207	2002	2002
q22	360	335	315	315
Total cold run time: 103189 ms
Total hot run time: 32789 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6288	6223	6260	6223
q2	243	331	233	233
q3	2230	2668	2341	2341
q4	1431	1820	1334	1334
q5	4426	4783	4936	4783
q6	193	175	143	143
q7	2120	2022	1785	1785
q8	2627	2837	2676	2676
q9	7329	7250	7228	7228
q10	3061	3335	2820	2820
q11	584	532	507	507
q12	634	758	597	597
q13	3348	3833	3157	3157
q14	306	316	293	293
q15	572	496	492	492
q16	637	690	640	640
q17	1204	1735	1230	1230
q18	7809	7402	7102	7102
q19	819	1148	1038	1038
q20	1912	1977	1889	1889
q21	5617	5124	4864	4864
q22	602	601	547	547
Total cold run time: 53992 ms
Total hot run time: 51922 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190518 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d5dc10b514cfba4259ad6ad7fca7c71b0942a6e, data reload: false

query1	970	377	390	377
query2	6511	2505	2544	2505
query3	6710	216	215	215
query4	33688	23673	23361	23361
query5	4354	652	501	501
query6	296	206	193	193
query7	4641	523	315	315
query8	303	246	237	237
query9	9606	2745	2726	2726
query10	465	299	272	272
query11	18311	15396	15594	15396
query12	166	109	105	105
query13	1680	548	415	415
query14	11675	6783	7363	6783
query15	235	197	187	187
query16	8300	634	452	452
query17	1620	775	576	576
query18	2113	430	287	287
query19	213	174	144	144
query20	114	121	113	113
query21	200	122	99	99
query22	4388	4491	4201	4201
query23	35048	33117	33492	33117
query24	6421	2218	2233	2218
query25	441	430	373	373
query26	1070	258	151	151
query27	2033	496	332	332
query28	5340	2433	2410	2410
query29	550	549	426	426
query30	224	181	149	149
query31	1008	934	795	795
query32	92	62	56	56
query33	498	339	298	298
query34	774	875	500	500
query35	812	832	745	745
query36	1011	1024	957	957
query37	123	99	70	70
query38	4135	4381	4119	4119
query39	1486	1448	1429	1429
query40	208	119	105	105
query41	49	49	44	44
query42	119	101	100	100
query43	540	543	515	515
query44	1339	806	821	806
query45	191	180	170	170
query46	890	1066	639	639
query47	1875	1903	1855	1855
query48	384	412	344	344
query49	782	474	374	374
query50	685	681	389	389
query51	6991	7030	7028	7028
query52	102	102	90	90
query53	227	254	185	185
query54	490	500	399	399
query55	85	78	83	78
query56	257	270	242	242
query57	1207	1152	1094	1094
query58	237	225	226	225
query59	3243	3209	3131	3131
query60	268	261	251	251
query61	108	105	104	104
query62	865	802	732	732
query63	244	191	186	186
query64	4445	985	653	653
query65	3265	3173	3240	3173
query66	1048	422	337	337
query67	16112	15838	15589	15589
query68	9421	774	510	510
query69	491	299	257	257
query70	1236	1140	1103	1103
query71	441	294	241	241
query72	5651	3875	3798	3798
query73	673	812	360	360
query74	9783	9403	9221	9221
query75	4626	3242	2618	2618
query76	5490	1260	815	815
query77	1003	384	271	271
query78	10002	10304	9802	9802
query79	3619	927	589	589
query80	724	540	421	421
query81	478	277	230	230
query82	345	150	116	116
query83	193	241	141	141
query84	279	91	69	69
query85	740	351	301	301
query86	360	309	285	285
query87	4653	4424	4468	4424
query88	3295	2217	2203	2203
query89	444	330	297	297
query90	2121	190	189	189
query91	128	138	102	102
query92	68	55	57	55
query93	2113	878	513	513
query94	655	405	295	295
query95	333	269	253	253
query96	505	665	286	286
query97	2762	2842	2697	2697
query98	234	203	200	200
query99	1639	1573	1413	1413
Total cold run time: 298386 ms
Total hot run time: 190518 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.32 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d5dc10b514cfba4259ad6ad7fca7c71b0942a6e, data reload: false

query1	0.03	0.03	0.04
query2	0.08	0.03	0.03
query3	0.24	0.07	0.06
query4	1.62	0.10	0.10
query5	0.45	0.40	0.40
query6	1.16	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.55	0.52	0.50
query10	0.59	0.59	0.55
query11	0.14	0.10	0.11
query12	0.14	0.10	0.10
query13	0.61	0.60	0.60
query14	2.81	2.77	2.79
query15	0.90	0.82	0.84
query16	0.38	0.39	0.39
query17	1.07	1.06	1.07
query18	0.23	0.21	0.20
query19	1.99	1.77	2.02
query20	0.01	0.01	0.01
query21	15.36	0.88	0.58
query22	0.76	0.78	0.63
query23	15.33	1.38	0.53
query24	3.27	0.96	1.17
query25	0.26	0.14	0.08
query26	0.24	0.14	0.14
query27	0.07	0.04	0.04
query28	13.30	1.58	1.05
query29	12.63	3.90	3.26
query30	0.26	0.09	0.06
query31	2.83	0.62	0.37
query32	3.22	0.56	0.49
query33	3.18	3.20	3.15
query34	16.70	5.09	4.53
query35	4.55	4.48	4.49
query36	0.62	0.48	0.48
query37	0.10	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.13 s
Total hot run time: 31.32 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 39.40% (10289/26112)
Line Coverage: 30.01% (85914/286277)
Region Coverage: 29.13% (43869/150614)
Branch Coverage: 25.57% (22288/87174)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9d5dc10b514cfba4259ad6ad7fca7c71b0942a6e_9d5dc10b514cfba4259ad6ad7fca7c71b0942a6e/report/index.html

@koarz
Copy link
Contributor Author

koarz commented Dec 27, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32699 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55e6a2e9b55fb8c30c2243338af3806765942533, data reload: false

------ Round 1 ----------------------------------
q1	17584	6194	6036	6036
q2	2053	308	178	178
q3	10775	1233	727	727
q4	10198	856	436	436
q5	7508	2175	2001	2001
q6	203	185	148	148
q7	900	746	609	609
q8	9250	1391	1186	1186
q9	5359	4996	4960	4960
q10	6786	2317	1861	1861
q11	493	281	257	257
q12	351	362	222	222
q13	17773	3577	2989	2989
q14	228	230	213	213
q15	567	500	492	492
q16	630	612	586	586
q17	585	861	329	329
q18	7122	6511	6445	6445
q19	2013	970	548	548
q20	299	310	194	194
q21	2823	2209	1972	1972
q22	362	340	310	310
Total cold run time: 103862 ms
Total hot run time: 32699 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6278	6209	6170	6170
q2	238	326	227	227
q3	2242	2658	2340	2340
q4	1411	1825	1332	1332
q5	4340	4772	4793	4772
q6	185	186	146	146
q7	2058	2010	1816	1816
q8	2654	2791	2692	2692
q9	7344	7302	7376	7302
q10	3077	3361	2763	2763
q11	578	496	482	482
q12	678	746	633	633
q13	3377	3806	3113	3113
q14	294	311	274	274
q15	584	536	503	503
q16	645	682	653	653
q17	1277	1750	1252	1252
q18	7715	7461	7377	7377
q19	817	1137	1093	1093
q20	2024	2016	1904	1904
q21	5721	5220	4930	4930
q22	592	616	594	594
Total cold run time: 54129 ms
Total hot run time: 52368 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196368 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 55e6a2e9b55fb8c30c2243338af3806765942533, data reload: false

query1	1293	982	918	918
query2	6520	2319	2317	2317
query3	11116	4700	4880	4700
query4	32890	23698	23491	23491
query5	3778	597	466	466
query6	271	189	186	186
query7	4004	527	311	311
query8	309	234	229	229
query9	9622	2737	2724	2724
query10	453	308	236	236
query11	18084	15404	15255	15255
query12	152	110	106	106
query13	1565	513	395	395
query14	9152	6744	7519	6744
query15	251	217	188	188
query16	8410	650	463	463
query17	1592	791	606	606
query18	2134	412	341	341
query19	219	200	167	167
query20	131	117	116	116
query21	202	131	110	110
query22	4577	4390	4384	4384
query23	35535	34325	33339	33339
query24	6613	2295	2338	2295
query25	493	437	399	399
query26	1136	284	159	159
query27	2464	476	391	391
query28	5847	2486	2443	2443
query29	709	545	424	424
query30	206	180	151	151
query31	982	910	838	838
query32	87	65	55	55
query33	497	348	311	311
query34	776	857	532	532
query35	829	835	781	781
query36	1016	1085	958	958
query37	118	100	74	74
query38	4167	4233	4233	4233
query39	1544	1484	1472	1472
query40	218	120	101	101
query41	46	50	46	46
query42	114	102	100	100
query43	535	525	495	495
query44	1364	869	846	846
query45	181	177	170	170
query46	890	1068	656	656
query47	1996	1970	1953	1953
query48	382	415	335	335
query49	763	472	382	382
query50	625	669	399	399
query51	7267	7254	7331	7254
query52	100	104	95	95
query53	228	255	194	194
query54	471	496	421	421
query55	83	87	80	80
query56	272	261	247	247
query57	1250	1249	1178	1178
query58	245	228	221	221
query59	3341	3259	3164	3164
query60	284	270	248	248
query61	111	124	115	115
query62	928	829	749	749
query63	231	191	187	187
query64	4307	1062	742	742
query65	3343	3238	3244	3238
query66	783	401	326	326
query67	16819	15830	15546	15546
query68	9916	774	509	509
query69	480	288	257	257
query70	1217	1167	1165	1165
query71	427	299	262	262
query72	6291	3841	3876	3841
query73	648	746	361	361
query74	10031	9212	9050	9050
query75	4468	3106	2656	2656
query76	5464	1192	778	778
query77	1045	345	273	273
query78	10170	10204	9428	9428
query79	4150	871	591	591
query80	725	537	433	433
query81	473	267	220	220
query82	222	160	124	124
query83	201	165	142	142
query84	329	86	72	72
query85	734	374	301	301
query86	358	298	308	298
query87	4627	4525	4244	4244
query88	3872	2222	2194	2194
query89	441	332	300	300
query90	2086	186	190	186
query91	132	141	104	104
query92	62	52	52	52
query93	3070	855	517	517
query94	724	386	290	290
query95	325	262	258	258
query96	489	613	289	289
query97	2742	2815	2692	2692
query98	219	213	199	199
query99	1673	1571	1486	1486
Total cold run time: 304732 ms
Total hot run time: 196368 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.36 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 55e6a2e9b55fb8c30c2243338af3806765942533, data reload: false

query1	0.04	0.03	0.05
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.63	0.11	0.11
query5	0.43	0.43	0.42
query6	1.15	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.51	0.50
query10	0.57	0.59	0.55
query11	0.15	0.10	0.10
query12	0.14	0.12	0.11
query13	0.61	0.60	0.61
query14	2.85	2.75	2.74
query15	0.89	0.84	0.82
query16	0.39	0.37	0.39
query17	1.01	1.01	1.04
query18	0.23	0.21	0.22
query19	1.93	1.77	2.05
query20	0.01	0.01	0.01
query21	15.39	0.88	0.58
query22	0.76	0.92	0.57
query23	15.27	1.50	0.57
query24	2.60	0.96	1.86
query25	0.20	0.19	0.21
query26	0.24	0.14	0.14
query27	0.05	0.05	0.05
query28	14.10	1.48	1.04
query29	12.54	3.90	3.27
query30	0.25	0.09	0.07
query31	2.83	0.61	0.39
query32	3.23	0.55	0.46
query33	3.16	3.11	3.11
query34	16.86	5.19	4.50
query35	4.51	4.53	4.51
query36	0.66	0.50	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.14 s
Total hot run time: 31.36 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 39.40% (10287/26112)
Line Coverage: 30.01% (85921/286337)
Region Coverage: 29.12% (43869/150661)
Branch Coverage: 25.56% (22288/87212)
Coverage Report: http://coverage.selectdb-in.cc/coverage/55e6a2e9b55fb8c30c2243338af3806765942533_55e6a2e9b55fb8c30c2243338af3806765942533/report/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants