Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](PreparedStatement) nondeterministic functions in prepared statement should not be short circuited #46003

Merged
merged 4 commits into from
Dec 27, 2024
Merged

[Fix](PreparedStatement) nondeterministic functions in prepared statement should not be short circuited #46003

merged 4 commits into from
Dec 27, 2024

Conversation

eldenmoon
Copy link
Member

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 26, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@eldenmoon
Copy link
Member Author

run buildall

@eldenmoon
Copy link
Member Author

run buildall

@eldenmoon eldenmoon force-pushed the fix-prep-nondeterministic branch from d743320 to 20e9098 Compare December 26, 2024 06:19
@eldenmoon
Copy link
Member Author

run buildall

morrySnow
morrySnow reviewed Dec 26, 2024
View reviewed changes
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/ExpressionAnalyzer.java Outdated
Comment on lines 413 to 416
if (ConnectContext.get() != null && ConnectContext.get().getCommand() == MysqlCommand.COM_STMT_EXECUTE
&& !buildResult.second.isDeterministic()) {
statementContext.setHasNondeterministicInPreparedStatement(true);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set this flag for all statement or mv this flag out of statement conctext

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommand.java Outdated Show resolved Hide resolved
@eldenmoon eldenmoon force-pushed the fix-prep-nondeterministic branch from ac9c499 to d2ed545 Compare December 26, 2024 06:42
@doris-robot
Copy link

TPC-H: Total hot run time: 32811 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 20e9098d465f14b02b30e324733be1463c02371c, data reload: false

------ Round 1 ----------------------------------
q1	17580	6106	6050	6050
q2	2038	298	169	169
q3	10432	1204	766	766
q4	10204	875	442	442
q5	7513	2195	2018	2018
q6	209	180	147	147
q7	906	766	600	600
q8	9240	1389	1252	1252
q9	5248	4930	4955	4930
q10	6738	2302	1893	1893
q11	470	285	257	257
q12	354	361	220	220
q13	17778	3542	2883	2883
q14	229	234	215	215
q15	574	510	494	494
q16	646	620	609	609
q17	596	852	336	336
q18	7128	6779	6478	6478
q19	1690	974	553	553
q20	302	312	193	193
q21	3127	2279	2003	2003
q22	374	347	303	303
Total cold run time: 103376 ms
Total hot run time: 32811 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6250	6203	6231	6203
q2	237	322	236	236
q3	2235	2659	2264	2264
q4	1451	1852	1355	1355
q5	4327	4708	4774	4708
q6	186	178	150	150
q7	2055	2003	1813	1813
q8	2635	2753	2664	2664
q9	7310	7239	7322	7239
q10	3051	3340	2815	2815
q11	578	511	507	507
q12	640	779	660	660
q13	3305	3698	3176	3176
q14	276	314	289	289
q15	582	514	504	504
q16	633	695	647	647
q17	1230	1717	1247	1247
q18	7638	7450	7332	7332
q19	896	1165	1088	1088
q20	1993	2100	1936	1936
q21	5741	5439	4801	4801
q22	626	617	604	604
Total cold run time: 53875 ms
Total hot run time: 52238 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196519 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 20e9098d465f14b02b30e324733be1463c02371c, data reload: false

query1	1283	951	904	904
query2	6503	2246	2237	2237
query3	11127	4921	4596	4596
query4	33008	24090	23404	23404
query5	4502	596	463	463
query6	286	199	183	183
query7	3994	500	301	301
query8	289	245	229	229
query9	9343	2758	2755	2755
query10	478	293	246	246
query11	18226	15389	15359	15359
query12	148	109	101	101
query13	1557	541	401	401
query14	10627	7693	7882	7693
query15	235	206	203	203
query16	8170	642	462	462
query17	1526	782	580	580
query18	2140	420	317	317
query19	195	183	159	159
query20	129	121	112	112
query21	203	128	101	101
query22	4804	4901	4483	4483
query23	34492	33634	33564	33564
query24	6566	2284	2294	2284
query25	507	455	379	379
query26	793	278	147	147
query27	2112	467	333	333
query28	5598	2498	2503	2498
query29	663	586	440	440
query30	207	181	150	150
query31	940	915	838	838
query32	88	61	59	59
query33	485	355	297	297
query34	756	852	519	519
query35	813	846	773	773
query36	1031	1036	954	954
query37	115	100	78	78
query38	4430	4196	4049	4049
query39	1508	1453	1466	1453
query40	221	116	102	102
query41	44	44	43	43
query42	120	102	99	99
query43	498	533	500	500
query44	1357	835	836	835
query45	181	189	170	170
query46	878	1061	665	665
query47	1999	1998	1964	1964
query48	386	415	325	325
query49	706	474	381	381
query50	649	685	392	392
query51	7221	7212	7117	7117
query52	108	103	93	93
query53	228	263	180	180
query54	492	522	413	413
query55	88	78	78	78
query56	263	262	240	240
query57	1261	1261	1146	1146
query58	257	228	228	228
query59	3209	3208	3081	3081
query60	326	261	253	253
query61	104	105	104	104
query62	863	809	733	733
query63	229	194	190	190
query64	3624	1023	657	657
query65	3306	3251	3202	3202
query66	818	419	325	325
query67	16589	15867	15510	15510
query68	8996	762	523	523
query69	470	289	253	253
query70	1192	1134	1087	1087
query71	428	296	262	262
query72	6391	3993	3712	3712
query73	663	751	367	367
query74	10314	9165	8945	8945
query75	3855	3167	2654	2654
query76	3605	1186	791	791
query77	761	372	282	282
query78	10165	10399	9415	9415
query79	3326	897	593	593
query80	608	522	432	432
query81	581	280	227	227
query82	680	157	119	119
query83	159	157	147	147
query84	251	85	69	69
query85	799	352	308	308
query86	380	322	280	280
query87	4479	4441	4577	4441
query88	4729	2222	2218	2218
query89	419	333	299	299
query90	1791	189	191	189
query91	131	140	104	104
query92	65	62	52	52
query93	1972	909	541	541
query94	645	386	295	295
query95	327	269	245	245
query96	490	604	279	279
query97	2705	2785	2655	2655
query98	217	197	192	192
query99	1540	1544	1413	1413
Total cold run time: 299036 ms
Total hot run time: 196519 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.91 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 20e9098d465f14b02b30e324733be1463c02371c, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.61	0.11	0.11
query5	0.43	0.43	0.41
query6	1.16	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.58	0.51	0.52
query10	0.56	0.56	0.57
query11	0.15	0.10	0.10
query12	0.13	0.11	0.11
query13	0.60	0.60	0.59
query14	2.71	2.75	2.85
query15	0.91	0.84	0.84
query16	0.39	0.39	0.37
query17	1.06	1.06	1.07
query18	0.22	0.20	0.21
query19	1.93	1.88	2.02
query20	0.01	0.01	0.01
query21	15.36	0.95	0.60
query22	0.77	1.08	0.80
query23	14.88	1.42	0.63
query24	2.99	1.52	1.18
query25	0.13	0.12	0.14
query26	0.35	0.16	0.12
query27	0.06	0.05	0.07
query28	13.81	1.50	1.05
query29	12.62	3.90	3.27
query30	0.25	0.09	0.06
query31	2.83	0.61	0.38
query32	3.23	0.53	0.46
query33	3.18	3.13	3.18
query34	16.91	5.13	4.49
query35	4.51	4.48	4.53
query36	0.63	0.49	0.48
query37	0.09	0.07	0.07
query38	0.06	0.03	0.03
query39	0.04	0.02	0.03
query40	0.16	0.14	0.12
query41	0.07	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 105.86 s
Total hot run time: 31.91 s

@eldenmoon
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32609 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d2ed545dfecbee7de09f3fff1be382ebf078007e, data reload: false

------ Round 1 ----------------------------------
q1	17606	6330	6048	6048
q2	2038	302	167	167
q3	10417	1304	747	747
q4	10218	882	456	456
q5	7503	2166	2021	2021
q6	206	180	145	145
q7	898	743	618	618
q8	9241	1374	1182	1182
q9	5331	4912	4864	4864
q10	6752	2273	1838	1838
q11	470	281	254	254
q12	363	357	220	220
q13	17801	3527	2983	2983
q14	230	233	215	215
q15	557	513	491	491
q16	618	611	585	585
q17	567	838	329	329
q18	7035	6492	6423	6423
q19	1233	958	552	552
q20	304	323	197	197
q21	2922	2212	1968	1968
q22	361	321	306	306
Total cold run time: 102671 ms
Total hot run time: 32609 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6234	6198	6220	6198
q2	235	319	243	243
q3	2230	2626	2294	2294
q4	1397	1816	1349	1349
q5	4283	4717	4624	4624
q6	180	174	140	140
q7	1952	1860	1714	1714
q8	2483	2652	2578	2578
q9	6955	6952	6903	6903
q10	2942	3243	2678	2678
q11	570	488	498	488
q12	627	712	579	579
q13	3223	3559	3016	3016
q14	289	297	256	256
q15	564	500	486	486
q16	627	678	622	622
q17	1167	1696	1214	1214
q18	7336	7069	7110	7069
q19	797	963	1112	963
q20	1942	2021	1906	1906
q21	5442	4996	4920	4920
q22	651	646	567	567
Total cold run time: 52126 ms
Total hot run time: 50807 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190034 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d2ed545dfecbee7de09f3fff1be382ebf078007e, data reload: false

query1	969	381	355	355
query2	6523	2425	2356	2356
query3	6711	210	214	210
query4	34073	23612	23712	23612
query5	4520	621	485	485
query6	293	199	194	194
query7	4635	497	299	299
query8	308	251	229	229
query9	9561	2756	2761	2756
query10	478	309	249	249
query11	18007	15363	15149	15149
query12	162	109	106	106
query13	1678	559	431	431
query14	10061	7322	7506	7322
query15	268	191	182	182
query16	8834	621	460	460
query17	1644	750	582	582
query18	2116	402	348	348
query19	222	180	145	145
query20	114	110	112	110
query21	213	129	106	106
query22	4481	4577	4520	4520
query23	34255	33297	33531	33297
query24	6437	2246	2307	2246
query25	474	434	384	384
query26	1166	281	153	153
query27	2032	457	332	332
query28	5270	2467	2477	2467
query29	726	535	405	405
query30	224	185	145	145
query31	972	906	814	814
query32	93	57	57	57
query33	509	348	317	317
query34	766	833	515	515
query35	791	808	728	728
query36	1019	1040	932	932
query37	113	92	82	82
query38	4220	4072	4293	4072
query39	1501	1429	1415	1415
query40	207	120	97	97
query41	47	45	45	45
query42	124	103	106	103
query43	516	514	482	482
query44	1293	794	809	794
query45	177	179	168	168
query46	872	1040	650	650
query47	1880	1913	1863	1863
query48	381	398	323	323
query49	786	470	382	382
query50	633	651	389	389
query51	7302	7154	6891	6891
query52	101	100	87	87
query53	226	261	186	186
query54	477	485	413	413
query55	84	77	78	77
query56	257	243	239	239
query57	1212	1197	1141	1141
query58	238	222	228	222
query59	3167	3114	2803	2803
query60	273	266	246	246
query61	111	137	117	117
query62	862	779	720	720
query63	232	182	190	182
query64	4159	988	652	652
query65	3239	3166	3280	3166
query66	1053	418	331	331
query67	15933	16007	15505	15505
query68	9652	764	534	534
query69	471	298	246	246
query70	1269	1124	1130	1124
query71	448	279	256	256
query72	5834	3836	3779	3779
query73	752	756	369	369
query74	10090	9061	8806	8806
query75	4617	3170	2645	2645
query76	5624	1180	761	761
query77	1022	364	270	270
query78	10194	10165	9323	9323
query79	5719	885	603	603
query80	758	531	429	429
query81	483	321	233	233
query82	349	164	136	136
query83	199	165	149	149
query84	279	88	72	72
query85	763	364	305	305
query86	340	326	298	298
query87	4424	4354	4286	4286
query88	3861	2255	2243	2243
query89	424	340	301	301
query90	2024	185	187	185
query91	131	142	115	115
query92	83	60	55	55
query93	3344	864	533	533
query94	665	386	279	279
query95	395	258	269	258
query96	505	606	276	276
query97	2708	2843	2694	2694
query98	223	225	201	201
query99	1640	1556	1429	1429
Total cold run time: 301571 ms
Total hot run time: 190034 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.03 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d2ed545dfecbee7de09f3fff1be382ebf078007e, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.62	0.10	0.11
query5	0.42	0.41	0.42
query6	1.18	0.65	0.65
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.58	0.50	0.53
query10	0.55	0.55	0.55
query11	0.15	0.10	0.10
query12	0.14	0.12	0.11
query13	0.60	0.61	0.59
query14	2.86	2.73	2.74
query15	0.89	0.83	0.82
query16	0.38	0.40	0.38
query17	1.00	1.02	1.06
query18	0.23	0.21	0.20
query19	1.98	1.84	2.00
query20	0.01	0.01	0.01
query21	15.36	0.85	0.60
query22	0.75	0.88	0.98
query23	14.86	1.37	0.54
query24	3.27	1.39	1.98
query25	0.17	0.14	0.15
query26	0.19	0.14	0.13
query27	0.07	0.05	0.04
query28	14.42	1.46	1.05
query29	12.59	3.99	3.32
query30	0.25	0.10	0.06
query31	2.81	0.61	0.37
query32	3.24	0.54	0.45
query33	3.06	3.10	3.16
query34	16.70	5.16	4.54
query35	4.55	4.46	4.53
query36	0.64	0.48	0.48
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.42 s
Total hot run time: 32.03 s

@eldenmoon eldenmoon force-pushed the fix-prep-nondeterministic branch from d2ed545 to f4a7c82 Compare December 26, 2024 08:04
@eldenmoon
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32440 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f4a7c8267c27bef9aac9171cc78620063eb9f26e, data reload: false

------ Round 1 ----------------------------------
q1	17593	6090	6052	6052
q2	2040	303	165	165
q3	10423	1285	714	714
q4	10192	866	426	426
q5	7491	2142	1951	1951
q6	211	179	144	144
q7	894	743	620	620
q8	9237	1387	1177	1177
q9	5440	4875	4928	4875
q10	6741	2307	1857	1857
q11	481	276	266	266
q12	355	365	219	219
q13	17760	3614	2943	2943
q14	245	231	212	212
q15	562	499	492	492
q16	631	608	582	582
q17	560	840	326	326
q18	6951	6403	6333	6333
q19	1259	964	568	568
q20	312	321	183	183
q21	2834	2367	2016	2016
q22	367	330	319	319
Total cold run time: 102579 ms
Total hot run time: 32440 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6197	6251	6214	6214
q2	237	328	238	238
q3	2227	2656	2343	2343
q4	1443	1807	1393	1393
q5	4346	4772	4611	4611
q6	177	174	138	138
q7	1959	1840	1694	1694
q8	2480	2722	2608	2608
q9	6970	6878	6916	6878
q10	2976	3197	2756	2756
q11	568	527	487	487
q12	608	724	575	575
q13	3192	3556	3026	3026
q14	288	282	267	267
q15	562	495	489	489
q16	628	695	653	653
q17	1162	1678	1208	1208
q18	7351	7112	6967	6967
q19	823	989	1097	989
q20	1941	1966	1879	1879
q21	5504	5178	5000	5000
q22	626	618	604	604
Total cold run time: 52265 ms
Total hot run time: 51017 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190291 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f4a7c8267c27bef9aac9171cc78620063eb9f26e, data reload: false

query1	958	394	388	388
query2	6526	2379	2306	2306
query3	6700	209	209	209
query4	33592	23837	23346	23346
query5	4330	631	444	444
query6	280	199	195	195
query7	4634	514	291	291
query8	298	253	244	244
query9	9844	2757	2747	2747
query10	480	309	255	255
query11	18421	15466	15549	15466
query12	175	104	110	104
query13	1722	540	403	403
query14	11751	7251	6920	6920
query15	236	200	181	181
query16	7924	608	488	488
query17	1541	726	545	545
query18	2052	402	296	296
query19	219	175	150	150
query20	124	113	127	113
query21	216	126	102	102
query22	4301	4392	4082	4082
query23	34453	33358	33510	33358
query24	6125	2247	2289	2247
query25	466	440	373	373
query26	898	266	155	155
query27	2038	458	325	325
query28	5485	2484	2449	2449
query29	608	530	432	432
query30	227	186	143	143
query31	1018	886	836	836
query32	73	58	57	57
query33	494	348	309	309
query34	743	839	527	527
query35	788	810	755	755
query36	1006	1044	950	950
query37	117	98	71	71
query38	4117	4142	4191	4142
query39	1511	1426	1436	1426
query40	200	113	100	100
query41	47	44	45	44
query42	122	99	99	99
query43	520	532	490	490
query44	1280	795	805	795
query45	180	201	167	167
query46	846	1038	642	642
query47	1911	1903	1833	1833
query48	385	395	327	327
query49	723	472	410	410
query50	615	650	391	391
query51	7177	7297	7063	7063
query52	103	100	89	89
query53	218	248	183	183
query54	480	490	405	405
query55	82	75	75	75
query56	253	255	235	235
query57	1204	1148	1138	1138
query58	241	236	216	216
query59	3145	3286	2851	2851
query60	272	261	248	248
query61	107	112	109	109
query62	862	797	734	734
query63	218	187	188	187
query64	3418	1003	634	634
query65	3315	3174	3221	3174
query66	901	424	305	305
query67	15920	15710	15453	15453
query68	7919	758	521	521
query69	459	289	265	265
query70	1185	1119	1133	1119
query71	456	285	253	253
query72	5856	3887	3845	3845
query73	661	741	365	365
query74	10036	9235	9021	9021
query75	4353	3154	2710	2710
query76	3723	1178	773	773
query77	775	354	292	292
query78	10183	10159	9535	9535
query79	3520	906	592	592
query80	729	521	430	430
query81	474	268	224	224
query82	642	147	122	122
query83	199	163	148	148
query84	281	91	70	70
query85	785	356	309	309
query86	394	300	290	290
query87	4504	4727	4625	4625
query88	4361	2245	2218	2218
query89	406	336	306	306
query90	1945	186	192	186
query91	131	129	106	106
query92	66	57	56	56
query93	1465	895	546	546
query94	661	395	295	295
query95	341	261	254	254
query96	494	603	277	277
query97	2790	2853	2700	2700
query98	230	214	198	198
query99	1648	1579	1413	1413
Total cold run time: 292908 ms
Total hot run time: 190291 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.09 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f4a7c8267c27bef9aac9171cc78620063eb9f26e, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.60	0.11	0.11
query5	0.42	0.41	0.42
query6	1.15	0.65	0.64
query7	0.02	0.01	0.02
query8	0.04	0.03	0.02
query9	0.58	0.51	0.50
query10	0.55	0.57	0.54
query11	0.15	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.60	0.59
query14	2.82	2.76	2.87
query15	0.91	0.83	0.83
query16	0.38	0.39	0.39
query17	1.06	1.02	1.06
query18	0.23	0.21	0.22
query19	1.92	1.80	1.98
query20	0.02	0.01	0.01
query21	15.36	0.97	0.59
query22	0.75	0.84	0.78
query23	15.05	1.39	0.61
query24	3.37	1.91	0.45
query25	0.23	0.21	0.07
query26	0.31	0.14	0.15
query27	0.07	0.04	0.04
query28	13.14	1.54	1.05
query29	12.57	4.08	3.38
query30	0.26	0.09	0.07
query31	2.83	0.58	0.37
query32	3.28	0.55	0.47
query33	3.09	3.09	3.20
query34	16.67	5.13	4.52
query35	4.56	4.48	4.56
query36	0.69	0.50	0.48
query37	0.09	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 105.68 s
Total hot run time: 31.09 s

@eldenmoon
Copy link
Member Author

run buildall

@eldenmoon eldenmoon force-pushed the fix-prep-nondeterministic branch from f4a7c82 to ecb87d6 Compare December 26, 2024 08:47
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 26, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 32496 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ecb87d6c4eb805018a350401cfae463126ab8363, data reload: false

------ Round 1 ----------------------------------
q1	17647	6106	6006	6006
q2	2050	306	166	166
q3	10412	1282	723	723
q4	10219	889	431	431
q5	7540	2217	1956	1956
q6	211	186	153	153
q7	889	728	611	611
q8	9255	1352	1161	1161
q9	5401	4938	4934	4934
q10	6731	2279	1879	1879
q11	469	276	262	262
q12	341	355	231	231
q13	17763	3585	2931	2931
q14	224	247	215	215
q15	578	525	508	508
q16	638	638	590	590
q17	549	862	328	328
q18	7032	6461	6343	6343
q19	1222	956	551	551
q20	322	326	187	187
q21	3087	2235	2019	2019
q22	365	331	311	311
Total cold run time: 102945 ms
Total hot run time: 32496 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6224	6184	6203	6184
q2	232	326	228	228
q3	2234	2614	2339	2339
q4	1431	1902	1373	1373
q5	4323	4744	4649	4649
q6	173	171	138	138
q7	1966	1848	1700	1700
q8	2486	2672	2619	2619
q9	6927	6828	6893	6828
q10	3001	3235	2728	2728
q11	585	510	505	505
q12	627	722	538	538
q13	3247	3649	2983	2983
q14	273	288	256	256
q15	562	498	496	496
q16	657	671	632	632
q17	1182	1689	1218	1218
q18	7181	7127	6896	6896
q19	785	1075	1071	1071
q20	1927	1946	1820	1820
q21	5522	5256	4789	4789
q22	591	623	570	570
Total cold run time: 52136 ms
Total hot run time: 50560 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190614 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ecb87d6c4eb805018a350401cfae463126ab8363, data reload: false

query1	974	366	376	366
query2	6534	2405	2269	2269
query3	6721	213	205	205
query4	33905	23556	23637	23556
query5	4323	623	472	472
query6	291	209	206	206
query7	4629	490	304	304
query8	305	254	243	243
query9	9720	2771	2757	2757
query10	459	330	257	257
query11	18607	15484	15145	15145
query12	169	106	110	106
query13	1679	544	406	406
query14	11883	7406	8212	7406
query15	243	193	183	183
query16	8316	645	491	491
query17	1628	761	584	584
query18	1607	387	327	327
query19	221	177	155	155
query20	123	118	118	118
query21	203	121	104	104
query22	4452	4522	4354	4354
query23	34155	33447	33361	33361
query24	6026	2248	2309	2248
query25	456	456	386	386
query26	771	275	148	148
query27	2004	474	329	329
query28	5338	2464	2439	2439
query29	571	529	405	405
query30	229	177	150	150
query31	1013	900	783	783
query32	74	56	57	56
query33	509	348	286	286
query34	763	826	511	511
query35	787	811	756	756
query36	1016	1041	961	961
query37	110	100	70	70
query38	4377	4123	4134	4123
query39	1484	1441	1421	1421
query40	239	119	112	112
query41	47	45	44	44
query42	119	101	100	100
query43	523	521	500	500
query44	1289	800	801	800
query45	176	179	166	166
query46	880	1028	662	662
query47	1858	1929	1851	1851
query48	383	407	333	333
query49	714	462	381	381
query50	614	648	371	371
query51	7325	7229	7030	7030
query52	101	103	96	96
query53	223	254	193	193
query54	483	505	414	414
query55	85	82	82	82
query56	260	267	238	238
query57	1213	1166	1105	1105
query58	251	230	228	228
query59	2996	3105	3064	3064
query60	269	258	244	244
query61	109	125	102	102
query62	898	804	724	724
query63	234	192	192	192
query64	3142	984	639	639
query65	3308	3158	3192	3158
query66	757	407	327	327
query67	15851	15885	15503	15503
query68	9249	765	547	547
query69	471	301	252	252
query70	1210	1105	1149	1105
query71	462	281	269	269
query72	5792	3642	3898	3642
query73	900	745	373	373
query74	10455	9238	8932	8932
query75	4491	3113	2669	2669
query76	5619	1189	778	778
query77	1004	348	269	269
query78	10085	10203	9356	9356
query79	3035	887	611	611
query80	763	516	437	437
query81	487	264	243	243
query82	645	148	122	122
query83	196	171	150	150
query84	284	90	69	69
query85	783	358	329	329
query86	354	319	320	319
query87	4588	4698	4435	4435
query88	3209	2273	2214	2214
query89	420	343	298	298
query90	1910	191	194	191
query91	136	139	105	105
query92	69	57	52	52
query93	1507	901	554	554
query94	670	388	294	294
query95	336	269	255	255
query96	491	617	281	281
query97	2806	2836	2694	2694
query98	228	207	210	207
query99	1698	1566	1456	1456
Total cold run time: 295375 ms
Total hot run time: 190614 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.77 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ecb87d6c4eb805018a350401cfae463126ab8363, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.60	0.10	0.11
query5	0.42	0.40	0.41
query6	1.17	0.66	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.58	0.50	0.53
query10	0.55	0.57	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.60	0.61	0.60
query14	2.72	2.74	2.83
query15	0.88	0.83	0.82
query16	0.38	0.38	0.38
query17	1.04	1.06	1.04
query18	0.22	0.21	0.20
query19	1.84	1.78	2.03
query20	0.01	0.01	0.01
query21	15.36	0.93	0.57
query22	0.75	0.85	0.69
query23	15.20	1.40	0.56
query24	3.49	0.38	1.72
query25	0.19	0.11	0.07
query26	0.23	0.15	0.13
query27	0.07	0.05	0.03
query28	13.49	1.56	1.05
query29	12.61	4.04	3.29
query30	0.25	0.09	0.07
query31	2.83	0.60	0.38
query32	3.23	0.55	0.46
query33	3.06	3.18	3.08
query34	16.72	5.13	4.54
query35	4.50	4.52	4.54
query36	0.63	0.49	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.07	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 105.78 s
Total hot run time: 30.77 s

csun5285
csun5285 approved these changes Dec 27, 2024
View reviewed changes
Copy link
Contributor

@csun5285 csun5285 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon eldenmoon merged commit 0c9673f into apache:master Dec 27, 2024
25 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 27, 2024
@eldenmoon
[Fix](PreparedStatement) nondeterministic functions in prepared state…
63c916a 
…ment should not be short circuited (#46003)
@github-actions github-actions bot mentioned this pull request Dec 27, 2024
Merged
github-actions bot pushed a commit that referenced this pull request Dec 27, 2024
@eldenmoon
[Fix](PreparedStatement) nondeterministic functions in prepared state…
bb07a54 
…ment should not be short circuited (#46003)
@github-actions github-actions bot mentioned this pull request Dec 27, 2024
Merged
@eldenmoon eldenmoon deleted the fix-prep-nondeterministic branch December 27, 2024 09:12
eldenmoon added a commit that referenced this pull request Dec 27, 2024
@github-actions @eldenmoon
branch-3.0: [Fix](PreparedStatement) nondeterministic functions in pr…
fe56490 
…epared statement should not be short circuited #46003 (#46108)

Cherry-picked from #46003

Co-authored-by: lihangyu <[email protected]>
eldenmoon added a commit that referenced this pull request Dec 27, 2024
@github-actions @eldenmoon
branch-2.1: [Fix](PreparedStatement) nondeterministic functions in pr…
829d9b3 
…epared statement should not be short circuited #46003 (#46109)

Cherry-picked from #46003

Co-authored-by: lihangyu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@csun5285 csun5285 csun5285 approved these changes

@924060929 924060929 Awaiting requested review from 924060929

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@eldenmoon @Thearas @doris-robot @csun5285 @morrySnow