Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

branch-3.0: [fix](fold) fixed an issue with be computing constants #43410 #45691

Merged
merged 1 commit into from
Dec 20, 2024
Merged

branch-3.0: [fix](fold) fixed an issue with be computing constants #43410 #45691

merged 1 commit into from
Dec 20, 2024

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #43410

@felixwluo
[fix](fold) fixed an issue with be computing constants (#43410)
b9346f8 
### What problem does this PR solve?

issue close: #43061

1、Problem
When enable_fold_constant_by_be=true is set,the results of
between below queries are inconsistent

select hex(from_base64('wr2JEDVXzL9+2XtRhgIloA==')) 
+----------------------------------------------+
| hex(from_base64('wr2JEDVXzL9+2XtRhgIloA==')) |
+----------------------------------------------+
| C2BD89103557CCBF7ED97B51860225A0             |
+----------------------------------------------+

select hex(s) from (select from_base64('wr2JEDVXzL9+2XtRhgIloA==') as s) t 
+--------------------------------------------------+
| hex(s)                                           |
+--------------------------------------------------+
| C2BDEFBFBD103557CCBF7EEFBFBD7B51EFBFBD0225EFBFBD |
+--------------------------------------------------+

2、mysql results

select hex(s) from (select from_base64('wr2JEDVXzL9+2XtRhgIloA==') as s) t;
+----------------------------------+
| hex(s)                           |
+----------------------------------+
| C2BD89103557CCBF7ED97B51860225A0 |
+----------------------------------+

3、cause
When processing binary data such as FromBase64, BE will return the
original binary data through the bytesValue field, and the previous code
only uses the stringValue field, resulting in the binary data being
corrupted during the string encoding conversion process
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Dec 20, 2024
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40608 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9346f82e79b70b8f8f5fb38f27d434b7ad82236, data reload: false

------ Round 1 ----------------------------------
q1	17763	7413	7304	7304
q2	2064	159	153	153
q3	10672	1126	1177	1126
q4	10215	751	734	734
q5	7748	2850	2755	2755
q6	231	149	144	144
q7	949	609	593	593
q8	9354	1923	1919	1919
q9	6642	6457	6418	6418
q10	6997	2317	2319	2317
q11	461	256	266	256
q12	386	211	211	211
q13	17778	2967	2948	2948
q14	250	213	206	206
q15	558	522	511	511
q16	716	609	591	591
q17	971	547	501	501
q18	7446	6623	6698	6623
q19	3427	1098	1082	1082
q20	474	212	187	187
q21	3920	3222	3055	3055
q22	1117	974	988	974
Total cold run time: 110139 ms
Total hot run time: 40608 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7389	7265	7261	7261
q2	323	234	230	230
q3	2875	2702	2659	2659
q4	1880	1611	1591	1591
q5	5393	5431	5428	5428
q6	213	135	137	135
q7	2075	1658	1669	1658
q8	3201	3374	3402	3374
q9	8522	8452	8524	8452
q10	3460	3393	3388	3388
q11	595	486	494	486
q12	785	548	558	548
q13	16914	2988	3009	2988
q14	288	255	271	255
q15	553	515	498	498
q16	703	655	639	639
q17	1778	1576	1542	1542
q18	7592	7439	7359	7359
q19	1632	1553	1501	1501
q20	1989	1773	1780	1773
q21	5129	5057	5162	5057
q22	1099	982	988	982
Total cold run time: 74388 ms
Total hot run time: 57804 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190647 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9346f82e79b70b8f8f5fb38f27d434b7ad82236, data reload: false

query1	1000	374	367	367
query2	6516	2076	1998	1998
query3	6710	216	230	216
query4	34001	23565	23673	23565
query5	4284	478	463	463
query6	262	172	196	172
query7	4628	311	307	307
query8	282	225	227	225
query9	9482	2714	2675	2675
query10	480	263	271	263
query11	18117	15201	15216	15201
query12	152	99	100	99
query13	1640	424	415	415
query14	9637	7311	6930	6930
query15	217	174	175	174
query16	7998	480	460	460
query17	1626	575	560	560
query18	2081	316	323	316
query19	235	160	151	151
query20	115	112	110	110
query21	72	49	50	49
query22	4502	4071	4378	4071
query23	34604	33737	33762	33737
query24	12187	2815	2947	2815
query25	718	413	403	403
query26	1870	173	173	173
query27	2956	313	296	296
query28	8227	2476	2453	2453
query29	1094	459	451	451
query30	339	171	164	164
query31	1019	772	787	772
query32	96	60	60	60
query33	793	297	295	295
query34	1000	506	506	506
query35	925	747	716	716
query36	1109	928	937	928
query37	274	76	75	75
query38	3955	3886	3862	3862
query39	1497	1414	1427	1414
query40	235	85	89	85
query41	55	51	51	51
query42	113	102	106	102
query43	549	480	497	480
query44	1226	783	786	783
query45	188	173	171	171
query46	1172	733	731	731
query47	1888	1785	1782	1782
query48	475	364	365	364
query49	1337	397	402	397
query50	819	407	411	407
query51	7305	7244	7114	7114
query52	103	97	105	97
query53	272	199	187	187
query54	1257	461	463	461
query55	75	76	79	76
query56	276	254	258	254
query57	1217	1126	1105	1105
query58	250	217	216	216
query59	3262	2809	2791	2791
query60	289	266	284	266
query61	107	107	107	107
query62	849	658	641	641
query63	217	192	189	189
query64	5402	657	606	606
query65	3298	3210	3214	3210
query66	1403	304	312	304
query67	15826	15250	15243	15243
query68	4466	558	548	548
query69	421	260	261	260
query70	1198	1130	1114	1114
query71	408	254	247	247
query72	6614	3915	3928	3915
query73	776	342	343	342
query74	10066	9056	9053	9053
query75	3382	2646	2621	2621
query76	2734	1142	1068	1068
query77	390	286	268	268
query78	10708	9750	9479	9479
query79	8900	623	601	601
query80	1913	437	429	429
query81	563	241	236	236
query82	1551	122	119	119
query83	311	147	145	145
query84	297	84	79	79
query85	1682	315	290	290
query86	477	296	301	296
query87	4455	4339	4418	4339
query88	5802	2409	2415	2409
query89	557	294	301	294
query90	2074	184	187	184
query91	180	146	144	144
query92	63	53	50	50
query93	6919	562	541	541
query94	921	294	291	291
query95	350	252	254	252
query96	637	289	279	279
query97	3318	3145	3174	3145
query98	225	210	200	200
query99	1624	1342	1282	1282
Total cold run time: 320147 ms
Total hot run time: 190647 ms

@morrySnow
Copy link
Contributor

run external

@morrySnow morrySnow merged commit c4bee9a into branch-3.0 Dec 20, 2024
19 of 21 checks passed
@morrySnow morrySnow deleted the auto-pick-43410-branch-3.0 branch December 20, 2024 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@hello-stephen @doris-robot @morrySnow @dataroaring @felixwluo