Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug](analytic) fix column string overflow in analytic operator #47423

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Jan 24, 2025

What problem does this PR solve?

Problem Summary:

before the column string insert will be have error when size is big 4G

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32580 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6c17760e804fc72f26af1f5054da176c6ea48df1, data reload: false

------ Round 1 ----------------------------------
q1	17587	5504	5390	5390
q2	2051	309	171	171
q3	10413	1332	733	733
q4	10213	971	519	519
q5	7532	2420	2169	2169
q6	194	160	134	134
q7	923	805	620	620
q8	9242	1365	1199	1199
q9	5409	4888	4926	4888
q10	6834	2344	1891	1891
q11	478	283	272	272
q12	340	358	223	223
q13	17751	3786	3172	3172
q14	235	216	204	204
q15	514	487	475	475
q16	646	623	586	586
q17	560	887	325	325
q18	6919	6523	6550	6523
q19	1216	949	557	557
q20	318	336	193	193
q21	2920	2263	2027	2027
q22	374	338	309	309
Total cold run time: 102669 ms
Total hot run time: 32580 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5505	5472	5472	5472
q2	254	327	231	231
q3	2258	2676	2276	2276
q4	1461	1844	1389	1389
q5	4317	4736	4674	4674
q6	162	158	125	125
q7	2080	2025	1837	1837
q8	2712	2856	2656	2656
q9	7241	7135	7173	7135
q10	2995	3308	2818	2818
q11	585	505	514	505
q12	678	784	617	617
q13	3577	3911	3279	3279
q14	284	293	283	283
q15	513	480	488	480
q16	664	685	642	642
q17	1259	1715	1264	1264
q18	7808	7535	7229	7229
q19	815	1137	1146	1137
q20	1990	2092	1886	1886
q21	5826	5190	4930	4930
q22	586	593	563	563
Total cold run time: 53570 ms
Total hot run time: 51428 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185698 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6c17760e804fc72f26af1f5054da176c6ea48df1, data reload: false

query1	968	395	374	374
query2	6528	2085	2043	2043
query3	6791	224	213	213
query4	33556	23325	23566	23325
query5	4456	626	496	496
query6	288	197	186	186
query7	4599	498	321	321
query8	311	252	235	235
query9	9375	2591	2574	2574
query10	471	312	250	250
query11	18124	15327	15194	15194
query12	151	103	106	103
query13	1640	518	385	385
query14	9730	7211	7470	7211
query15	249	183	187	183
query16	8629	631	483	483
query17	1588	715	562	562
query18	2096	392	303	303
query19	219	187	162	162
query20	119	113	112	112
query21	216	123	97	97
query22	4317	4269	4254	4254
query23	34326	32996	33136	32996
query24	6635	2232	2298	2232
query25	494	473	410	410
query26	1124	266	161	161
query27	2070	470	336	336
query28	5313	2422	2415	2415
query29	582	557	442	442
query30	238	186	160	160
query31	939	886	799	799
query32	103	62	65	62
query33	525	368	305	305
query34	743	856	501	501
query35	839	810	756	756
query36	982	1074	947	947
query37	125	101	82	82
query38	4103	4165	4120	4120
query39	1461	1421	1422	1421
query40	212	124	107	107
query41	58	56	55	55
query42	122	109	103	103
query43	520	520	492	492
query44	1361	816	811	811
query45	181	177	165	165
query46	856	1029	637	637
query47	1828	1831	1786	1786
query48	372	400	307	307
query49	776	483	398	398
query50	632	665	404	404
query51	4214	4157	4120	4120
query52	110	105	89	89
query53	232	262	195	195
query54	528	495	443	443
query55	84	77	82	77
query56	264	255	250	250
query57	1136	1136	1052	1052
query58	243	237	237	237
query59	2804	3128	2852	2852
query60	278	268	271	268
query61	119	116	116	116
query62	772	723	652	652
query63	230	192	195	192
query64	4300	986	666	666
query65	3273	3238	3171	3171
query66	1080	448	307	307
query67	15933	15409	15679	15409
query68	5480	821	539	539
query69	477	289	268	268
query70	1169	1141	1144	1141
query71	389	304	254	254
query72	5833	3793	3821	3793
query73	633	753	356	356
query74	10345	9056	8840	8840
query75	3177	3144	2643	2643
query76	3167	1166	768	768
query77	465	362	272	272
query78	9985	9963	9256	9256
query79	3205	830	598	598
query80	1301	537	459	459
query81	568	267	240	240
query82	364	154	125	125
query83	178	176	169	169
query84	245	102	71	71
query85	749	411	301	301
query86	433	309	298	298
query87	4462	4453	4386	4386
query88	5092	2179	2149	2149
query89	391	332	289	289
query90	1859	197	191	191
query91	139	136	108	108
query92	69	60	57	57
query93	2737	867	538	538
query94	731	421	303	303
query95	338	279	254	254
query96	478	606	277	277
query97	2768	2869	2722	2722
query98	235	203	197	197
query99	1268	1407	1264	1264
Total cold run time: 286271 ms
Total hot run time: 185698 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6c17760e804fc72f26af1f5054da176c6ea48df1, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.62	0.10	0.10
query5	0.43	0.43	0.43
query6	1.16	0.67	0.65
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.58	0.50	0.51
query10	0.56	0.55	0.55
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.73	2.79	2.83
query15	0.90	0.83	0.84
query16	0.39	0.39	0.38
query17	1.07	1.08	1.05
query18	0.23	0.21	0.21
query19	1.93	1.86	2.03
query20	0.01	0.02	0.01
query21	15.36	0.98	0.62
query22	0.76	0.85	0.75
query23	15.12	1.49	0.52
query24	2.62	0.99	1.16
query25	0.12	0.34	0.11
query26	0.28	0.14	0.14
query27	0.06	0.07	0.04
query28	14.06	1.06	0.44
query29	12.57	3.94	3.29
query30	0.26	0.08	0.06
query31	2.82	0.59	0.40
query32	3.22	0.57	0.48
query33	3.06	3.04	3.06
query34	16.50	5.19	4.58
query35	4.58	4.56	4.54
query36	0.63	0.48	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.14	0.12
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.45 s
Total hot run time: 31.13 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 41.98% (10948/26080)
Line Coverage: 32.23% (92423/286748)
Region Coverage: 31.36% (47360/151016)
Branch Coverage: 27.43% (23998/87486)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6c17760e804fc72f26af1f5054da176c6ea48df1_6c17760e804fc72f26af1f5054da176c6ea48df1/report/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants