Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improve](ut)add data_type_array ut test and serde test #46063

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

[improve](ut)add data_type_array ut test and serde test #46063

wants to merge 5 commits into from
+435 −57

Conversation

amorynan
Copy link
Contributor

What problem does this PR solve?

this pr :
1/ data_type_array ut cover
2/ data_type_array_serde ut cover
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 26, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@amorynan
Copy link
Contributor Author

run buildall

@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32625 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b627fa473bb938626399bd6c0c214c6ff89bbddb, data reload: false

------ Round 1 ----------------------------------
q1	17619	6143	6017	6017
q2	2037	304	166	166
q3	10487	1291	761	761
q4	10214	881	455	455
q5	7590	2196	2027	2027
q6	206	179	142	142
q7	910	745	599	599
q8	9227	1401	1245	1245
q9	5264	4924	5031	4924
q10	6750	2320	1861	1861
q11	474	294	263	263
q12	346	367	221	221
q13	17771	3614	2930	2930
q14	235	240	214	214
q15	550	508	482	482
q16	650	623	597	597
q17	589	864	330	330
q18	7143	6601	6343	6343
q19	2488	959	562	562
q20	292	315	186	186
q21	2830	2169	1995	1995
q22	352	332	305	305
Total cold run time: 104024 ms
Total hot run time: 32625 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6451	6241	6221	6221
q2	235	325	227	227
q3	2278	2664	2320	2320
q4	1433	1842	1344	1344
q5	4345	4747	4962	4747
q6	187	180	139	139
q7	2170	1978	1859	1859
q8	2579	2765	2651	2651
q9	7369	7234	7329	7234
q10	3060	3366	2801	2801
q11	591	540	500	500
q12	673	742	616	616
q13	3384	3760	3155	3155
q14	284	315	289	289
q15	560	525	508	508
q16	654	690	656	656
q17	1235	1764	1240	1240
q18	7638	7452	7115	7115
q19	808	1060	1068	1060
q20	1950	1968	1830	1830
q21	5615	5181	5006	5006
q22	625	628	592	592
Total cold run time: 54124 ms
Total hot run time: 52110 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191368 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b627fa473bb938626399bd6c0c214c6ff89bbddb, data reload: false

query1	969	381	393	381
query2	6503	2435	2434	2434
query3	6707	212	217	212
query4	33820	23633	23229	23229
query5	4365	635	464	464
query6	284	199	205	199
query7	4628	507	313	313
query8	313	261	249	249
query9	9717	2755	2761	2755
query10	456	305	255	255
query11	18531	15497	15319	15319
query12	168	110	105	105
query13	1676	528	414	414
query14	12011	7898	7075	7075
query15	259	201	198	198
query16	8634	578	457	457
query17	1605	734	582	582
query18	2145	436	288	288
query19	201	179	149	149
query20	116	119	113	113
query21	213	121	103	103
query22	4192	4226	4256	4226
query23	34402	33343	33252	33252
query24	6069	2396	2257	2257
query25	491	448	377	377
query26	791	272	152	152
query27	2038	472	333	333
query28	5097	2456	2420	2420
query29	675	544	413	413
query30	230	180	147	147
query31	999	917	820	820
query32	92	58	61	58
query33	531	345	316	316
query34	772	826	510	510
query35	781	818	721	721
query36	1003	1029	938	938
query37	116	105	75	75
query38	4163	4215	4054	4054
query39	1485	1447	1442	1442
query40	201	115	98	98
query41	51	45	45	45
query42	123	98	102	98
query43	520	540	504	504
query44	1282	790	814	790
query45	180	179	171	171
query46	857	1038	657	657
query47	1901	1935	1871	1871
query48	385	408	324	324
query49	724	474	381	381
query50	617	635	393	393
query51	7107	7195	7107	7107
query52	104	101	92	92
query53	232	249	184	184
query54	482	489	397	397
query55	87	81	95	81
query56	257	259	246	246
query57	1194	1166	1105	1105
query58	241	221	220	220
query59	3081	3118	3053	3053
query60	273	271	256	256
query61	116	114	107	107
query62	881	807	706	706
query63	227	194	201	194
query64	3471	987	643	643
query65	3281	3199	3267	3199
query66	875	410	310	310
query67	16071	15844	15610	15610
query68	9580	743	516	516
query69	462	288	267	267
query70	1253	1150	1161	1150
query71	438	282	268	268
query72	5950	3904	3916	3904
query73	670	739	357	357
query74	10233	9328	9254	9254
query75	4660	3172	2652	2652
query76	5387	1183	789	789
query77	973	386	285	285
query78	9912	10298	9934	9934
query79	3740	877	595	595
query80	756	514	433	433
query81	480	269	236	236
query82	273	146	129	129
query83	195	165	149	149
query84	295	90	72	72
query85	757	379	315	315
query86	357	327	291	291
query87	4430	4577	4527	4527
query88	3375	2254	2215	2215
query89	419	342	310	310
query90	2089	193	195	193
query91	136	140	101	101
query92	68	55	55	55
query93	1795	849	533	533
query94	656	414	289	289
query95	336	266	303	266
query96	487	622	279	279
query97	2779	2828	2706	2706
query98	231	202	196	196
query99	1626	1599	1455	1455
Total cold run time: 296867 ms
Total hot run time: 191368 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.98 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b627fa473bb938626399bd6c0c214c6ff89bbddb, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.08	0.07
query4	1.60	0.10	0.10
query5	0.43	0.41	0.42
query6	1.14	0.66	0.64
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.51
query10	0.56	0.58	0.55
query11	0.16	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.62	0.60
query14	2.85	2.73	2.73
query15	0.90	0.84	0.82
query16	0.39	0.38	0.39
query17	1.07	1.08	1.00
query18	0.22	0.21	0.20
query19	1.90	1.90	2.04
query20	0.01	0.01	0.01
query21	15.74	0.89	0.58
query22	1.02	0.76	0.67
query23	15.34	1.36	0.59
query24	3.40	1.92	1.45
query25	0.13	0.30	0.12
query26	0.22	0.14	0.14
query27	0.08	0.06	0.04
query28	14.54	1.47	1.05
query29	12.60	3.97	3.30
query30	0.25	0.09	0.07
query31	2.83	0.59	0.37
query32	3.23	0.54	0.46
query33	3.15	3.07	3.17
query34	16.72	5.09	4.54
query35	4.49	4.50	4.54
query36	0.66	0.48	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.86 s
Total hot run time: 31.98 s

@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32843 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b3cbc31319adf8fb698170a83efb3b1071fa415, data reload: false

------ Round 1 ----------------------------------
q1	17588	6179	6050	6050
q2	2061	309	168	168
q3	10414	1315	722	722
q4	10211	870	435	435
q5	7551	2210	2038	2038
q6	213	185	149	149
q7	933	773	625	625
q8	9241	1408	1198	1198
q9	5225	4902	4993	4902
q10	6762	2318	1862	1862
q11	473	280	263	263
q12	352	365	220	220
q13	17749	3675	2975	2975
q14	241	246	229	229
q15	559	511	495	495
q16	627	605	580	580
q17	582	863	328	328
q18	7149	6498	6516	6498
q19	1725	981	585	585
q20	318	318	192	192
q21	3002	2236	2015	2015
q22	369	335	314	314
Total cold run time: 103345 ms
Total hot run time: 32843 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6297	6224	6256	6224
q2	243	326	243	243
q3	2221	2698	2339	2339
q4	1376	1792	1338	1338
q5	4422	4776	4918	4776
q6	194	181	148	148
q7	2119	1934	1820	1820
q8	2636	2808	2701	2701
q9	7347	7267	7202	7202
q10	3060	3282	2840	2840
q11	599	539	490	490
q12	672	743	608	608
q13	3378	3735	3137	3137
q14	299	300	286	286
q15	592	502	496	496
q16	671	687	627	627
q17	1240	1742	1240	1240
q18	7708	7474	7201	7201
q19	841	1133	1049	1049
q20	1915	2024	1853	1853
q21	5405	5183	4835	4835
q22	639	593	583	583
Total cold run time: 53874 ms
Total hot run time: 52036 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190519 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b3cbc31319adf8fb698170a83efb3b1071fa415, data reload: false

query1	970	372	381	372
query2	6546	2415	2413	2413
query3	6729	219	213	213
query4	33858	23703	23242	23242
query5	4340	637	454	454
query6	283	203	204	203
query7	4629	515	305	305
query8	300	266	246	246
query9	9772	2781	2764	2764
query10	447	303	237	237
query11	18392	15307	15384	15307
query12	161	107	106	106
query13	1660	528	391	391
query14	10028	6912	7627	6912
query15	247	193	183	183
query16	8092	594	454	454
query17	1546	733	544	544
query18	2121	383	300	300
query19	212	188	150	150
query20	115	114	115	114
query21	211	129	106	106
query22	4732	4524	4129	4129
query23	34252	33653	33685	33653
query24	6316	2266	2277	2266
query25	506	447	412	412
query26	1191	271	155	155
query27	2299	457	344	344
query28	5600	2465	2436	2436
query29	753	534	405	405
query30	230	180	151	151
query31	1007	896	822	822
query32	89	60	59	59
query33	503	349	301	301
query34	771	828	519	519
query35	800	814	752	752
query36	1017	1054	957	957
query37	125	99	77	77
query38	4260	4118	4167	4118
query39	1526	1423	1466	1423
query40	203	111	99	99
query41	48	44	44	44
query42	119	106	103	103
query43	530	552	509	509
query44	1319	805	792	792
query45	190	179	173	173
query46	906	1030	653	653
query47	1901	1969	1886	1886
query48	370	403	326	326
query49	783	481	391	391
query50	633	657	421	421
query51	7190	7113	6944	6944
query52	98	98	93	93
query53	226	251	197	197
query54	483	481	399	399
query55	77	80	78	78
query56	257	267	237	237
query57	1184	1168	1126	1126
query58	241	228	234	228
query59	2952	3159	3032	3032
query60	279	275	251	251
query61	111	108	124	108
query62	832	789	737	737
query63	231	202	195	195
query64	4619	1002	684	684
query65	3309	3210	3296	3210
query66	1064	429	334	334
query67	16133	16029	15525	15525
query68	9045	754	502	502
query69	464	282	258	258
query70	1198	1132	1149	1132
query71	439	280	250	250
query72	5845	3840	3851	3840
query73	653	748	441	441
query74	10000	9067	8817	8817
query75	4441	3122	2634	2634
query76	4346	1186	770	770
query77	781	364	279	279
query78	10065	10374	9674	9674
query79	3409	887	575	575
query80	701	531	459	459
query81	474	268	235	235
query82	598	152	119	119
query83	193	163	144	144
query84	278	94	70	70
query85	806	348	321	321
query86	351	321	285	285
query87	4574	4447	4568	4447
query88	4076	2231	2195	2195
query89	399	334	307	307
query90	1926	191	191	191
query91	132	130	109	109
query92	69	58	53	53
query93	942	894	526	526
query94	660	381	290	290
query95	341	252	249	249
query96	497	599	282	282
query97	2798	2848	2651	2651
query98	237	195	199	195
query99	1699	1569	1449	1449
Total cold run time: 295361 ms
Total hot run time: 190519 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.57 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4b3cbc31319adf8fb698170a83efb3b1071fa415, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.07
query4	1.61	0.10	0.11
query5	0.43	0.40	0.41
query6	1.16	0.67	0.64
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.49	0.50
query10	0.54	0.57	0.54
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.59
query14	2.81	2.85	2.89
query15	0.90	0.84	0.82
query16	0.37	0.39	0.37
query17	1.06	1.04	1.01
query18	0.23	0.21	0.21
query19	1.91	1.80	1.99
query20	0.02	0.01	0.01
query21	15.37	0.93	0.62
query22	0.76	0.83	0.62
query23	15.27	1.46	0.54
query24	3.16	1.33	1.18
query25	0.26	0.13	0.12
query26	0.26	0.15	0.14
query27	0.04	0.06	0.05
query28	13.14	1.58	1.06
query29	12.58	4.03	3.28
query30	0.25	0.09	0.07
query31	2.83	0.64	0.38
query32	3.22	0.55	0.46
query33	3.16	3.11	3.13
query34	16.71	5.15	4.47
query35	4.48	4.48	4.46
query36	0.62	0.50	0.49
query37	0.09	0.06	0.05
query38	0.04	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.51 s
Total hot run time: 31.57 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@amorynan @Thearas @doris-robot