Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature](table-function)support posexplode table function #43221

Merged
merged 7 commits into from
Nov 6, 2024
Merged

[feature](table-function)support posexplode table function #43221

merged 7 commits into from
Nov 6, 2024

Conversation

zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Nov 4, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
support posexplode table function
doc: apache/doris-website#1283

Check List (For Committer)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No colde files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

  • Release note

    None

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions[bot]
github-actions bot reviewed Nov 4, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/vec/exprs/table_function/vpose_explode.cpp Outdated Show resolved Hide resolved
be/src/vec/exprs/table_function/vpose_explode.h Outdated Show resolved Hide resolved
@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.80% (9824/25986)
Line Coverage: 28.97% (81666/281939)
Region Coverage: 28.22% (42114/149255)
Branch Coverage: 24.79% (21364/86168)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8ff11a7c93e6cdcea5269d034d25ebb21f1ef8a0_8ff11a7c93e6cdcea5269d034d25ebb21f1ef8a0/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 43222 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8ff11a7c93e6cdcea5269d034d25ebb21f1ef8a0, data reload: false

------ Round 1 ----------------------------------
q1	17592	8081	7968	7968
q2	2045	171	179	171
q3	10614	1140	1244	1140
q4	10568	861	868	861
q5	7769	3260	3295	3260
q6	256	146	143	143
q7	1091	611	597	597
q8	9350	2016	2103	2016
q9	7037	6815	6854	6815
q10	7183	2480	2479	2479
q11	481	260	257	257
q12	444	215	213	213
q13	17767	3001	2990	2990
q14	243	203	203	203
q15	590	516	507	507
q16	655	585	582	582
q17	1004	595	493	493
q18	7345	6797	6755	6755
q19	1367	1131	1112	1112
q20	503	187	183	183
q21	4235	3465	3555	3465
q22	1114	1016	1012	1012
Total cold run time: 109253 ms
Total hot run time: 43222 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7966	7923	7988	7923
q2	357	237	229	229
q3	3174	3041	3094	3041
q4	2166	1797	1874	1797
q5	5812	5905	5984	5905
q6	251	144	144	144
q7	2269	1824	1877	1824
q8	3540	3809	3832	3809
q9	9020	8997	9022	8997
q10	3730	3741	3780	3741
q11	639	512	503	503
q12	841	628	644	628
q13	10128	3144	3230	3144
q14	311	283	290	283
q15	585	515	518	515
q16	688	649	649	649
q17	2039	1728	1696	1696
q18	8183	7634	7830	7634
q19	1895	1732	1707	1707
q20	2177	1862	1864	1862
q21	5867	5481	5467	5467
q22	1178	1090	1037	1037
Total cold run time: 72816 ms
Total hot run time: 62535 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195819 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8ff11a7c93e6cdcea5269d034d25ebb21f1ef8a0, data reload: false

query1	1197	920	896	896
query2	6249	2074	2082	2074
query3	10800	3928	4044	3928
query4	67381	29778	23523	23523
query5	4968	457	434	434
query6	401	182	184	182
query7	5628	292	288	288
query8	306	224	226	224
query9	8933	2666	2657	2657
query10	454	254	245	245
query11	17442	15477	15907	15477
query12	155	104	102	102
query13	1542	432	426	426
query14	10388	7631	7093	7093
query15	205	187	174	174
query16	7007	424	418	418
query17	990	529	529	529
query18	1764	283	284	283
query19	194	143	139	139
query20	114	109	107	107
query21	202	99	98	98
query22	4593	4153	4175	4153
query23	34367	34131	34089	34089
query24	5981	2721	2748	2721
query25	475	373	377	373
query26	635	164	151	151
query27	1699	284	286	284
query28	4164	2421	2412	2412
query29	653	403	398	398
query30	235	154	152	152
query31	987	770	808	770
query32	66	50	54	50
query33	421	271	261	261
query34	906	505	516	505
query35	837	736	747	736
query36	1053	908	953	908
query37	117	76	71	71
query38	4483	4292	4186	4186
query39	1463	1445	1562	1445
query40	195	94	92	92
query41	44	42	42	42
query42	104	93	95	93
query43	531	499	485	485
query44	1153	834	834	834
query45	180	165	165	165
query46	1122	717	713	713
query47	1947	1853	1857	1853
query48	435	330	319	319
query49	748	406	403	403
query50	817	414	388	388
query51	7220	7169	7179	7169
query52	106	89	88	88
query53	257	179	179	179
query54	532	409	414	409
query55	79	78	74	74
query56	263	243	255	243
query57	1289	1176	1154	1154
query58	220	207	206	206
query59	3149	3009	3036	3009
query60	288	255	257	255
query61	128	122	123	122
query62	789	676	679	676
query63	222	191	181	181
query64	1421	742	710	710
query65	3252	3191	3233	3191
query66	706	313	314	313
query67	15981	15607	15723	15607
query68	2669	580	592	580
query69	409	264	258	258
query70	1172	1104	1126	1104
query71	350	254	251	251
query72	6319	4012	4023	4012
query73	759	358	353	353
query74	9697	8982	9099	8982
query75	3405	2648	2691	2648
query76	1426	1091	1057	1057
query77	465	274	263	263
query78	10404	9344	9362	9344
query79	1356	591	592	591
query80	836	422	411	411
query81	517	239	239	239
query82	1314	111	114	111
query83	170	134	132	132
query84	283	71	69	69
query85	869	307	291	291
query86	352	294	279	279
query87	4865	4643	4723	4643
query88	3255	2195	2160	2160
query89	408	294	280	280
query90	1974	191	186	186
query91	131	96	108	96
query92	67	49	48	48
query93	1363	544	564	544
query94	768	278	277	277
query95	349	251	240	240
query96	611	279	280	279
query97	2902	2710	2676	2676
query98	215	212	201	201
query99	1773	1300	1279	1279
Total cold run time: 314086 ms
Total hot run time: 195819 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.66 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8ff11a7c93e6cdcea5269d034d25ebb21f1ef8a0, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.64	0.10	0.10
query5	0.41	0.40	0.41
query6	1.14	0.67	0.65
query7	0.02	0.03	0.01
query8	0.04	0.03	0.03
query9	0.56	0.50	0.50
query10	0.54	0.54	0.54
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.59
query14	2.72	2.74	2.71
query15	0.92	0.83	0.82
query16	0.38	0.39	0.40
query17	1.04	1.05	0.99
query18	0.20	0.20	0.20
query19	1.87	1.79	1.86
query20	0.02	0.02	0.02
query21	15.36	0.58	0.59
query22	2.69	2.10	1.81
query23	17.09	1.05	0.82
query24	3.19	1.67	1.27
query25	0.21	0.05	0.21
query26	0.68	0.15	0.14
query27	0.04	0.03	0.04
query28	9.60	1.11	1.06
query29	12.56	3.28	3.25
query30	0.24	0.06	0.06
query31	2.88	0.41	0.39
query32	3.22	0.47	0.47
query33	2.99	2.96	3.07
query34	16.67	4.48	4.44
query35	4.53	4.48	4.54
query36	0.66	0.50	0.48
query37	0.09	0.06	0.07
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.8 s
Total hot run time: 32.66 s

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.80% (9822/25986)
Line Coverage: 28.96% (81656/281939)
Region Coverage: 28.20% (42097/149255)
Branch Coverage: 24.79% (21361/86168)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fbdcd1697891b62cdc9b458775c43245f2d1c432_fbdcd1697891b62cdc9b458775c43245f2d1c432/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41517 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fbdcd1697891b62cdc9b458775c43245f2d1c432, data reload: false

------ Round 1 ----------------------------------
q1	17602	7370	7310	7310
q2	2073	162	164	162
q3	10827	1092	1143	1092
q4	10557	830	884	830
q5	7730	3076	3070	3070
q6	233	152	143	143
q7	1023	600	596	596
q8	9340	1931	2051	1931
q9	6598	6420	6415	6415
q10	7085	2421	2458	2421
q11	459	256	258	256
q12	398	210	207	207
q13	17784	2987	3013	2987
q14	230	208	206	206
q15	567	519	515	515
q16	650	572	581	572
q17	966	556	518	518
q18	7246	6731	6740	6731
q19	1340	1067	1090	1067
q20	460	180	182	180
q21	3928	3325	3302	3302
q22	1115	1006	1013	1006
Total cold run time: 108211 ms
Total hot run time: 41517 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7289	7219	7288	7219
q2	323	226	233	226
q3	2986	2942	2919	2919
q4	2066	1866	1751	1751
q5	5752	5738	5774	5738
q6	225	142	143	142
q7	2293	1845	1804	1804
q8	3369	3537	3417	3417
q9	8922	8920	8903	8903
q10	3562	3560	3549	3549
q11	598	495	502	495
q12	845	634	623	623
q13	10684	3206	3160	3160
q14	303	275	286	275
q15	600	522	532	522
q16	691	644	634	634
q17	1854	1614	1585	1585
q18	8356	7722	7664	7664
q19	1721	1597	1498	1498
q20	2133	1880	1855	1855
q21	5557	5427	5329	5329
q22	1202	1053	1030	1030
Total cold run time: 71331 ms
Total hot run time: 60338 ms

@cambyzju cambyzju mentioned this pull request Nov 4, 2024
Closed
3 tasks
@doris-robot
Copy link

TPC-DS: Total hot run time: 195578 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fbdcd1697891b62cdc9b458775c43245f2d1c432, data reload: false

query1	1228	920	910	910
query2	6228	2122	2073	2073
query3	10809	4064	4029	4029
query4	67995	29354	23606	23606
query5	4938	428	425	425
query6	391	176	172	172
query7	5560	297	288	288
query8	304	242	227	227
query9	8848	2648	2634	2634
query10	435	259	244	244
query11	17475	15485	15805	15485
query12	159	104	103	103
query13	1536	442	443	442
query14	10790	7706	6727	6727
query15	207	185	177	177
query16	7112	467	418	418
query17	1017	556	578	556
query18	1773	298	288	288
query19	189	159	156	156
query20	122	111	110	110
query21	199	103	99	99
query22	4629	4254	4063	4063
query23	34403	34014	34011	34011
query24	6033	2820	2780	2780
query25	516	392	393	392
query26	648	158	155	155
query27	1671	287	282	282
query28	4333	2469	2412	2412
query29	682	440	431	431
query30	239	159	150	150
query31	977	814	831	814
query32	64	54	58	54
query33	457	279	266	266
query34	915	507	516	507
query35	831	724	745	724
query36	1077	954	947	947
query37	119	74	72	72
query38	4353	4295	4250	4250
query39	1451	1451	1419	1419
query40	208	98	97	97
query41	48	44	47	44
query42	105	99	98	98
query43	536	488	459	459
query44	1173	820	819	819
query45	181	164	168	164
query46	1106	700	692	692
query47	1955	1821	1891	1821
query48	417	327	321	321
query49	732	405	397	397
query50	820	384	399	384
query51	7308	7112	7068	7068
query52	100	90	87	87
query53	252	184	185	184
query54	526	414	426	414
query55	80	71	74	71
query56	253	231	234	231
query57	1262	1164	1135	1135
query58	216	196	201	196
query59	3273	3180	2971	2971
query60	261	242	243	242
query61	109	101	108	101
query62	779	669	693	669
query63	216	185	191	185
query64	1347	642	632	632
query65	3248	3176	3403	3176
query66	688	306	306	306
query67	15918	15740	15724	15724
query68	3550	596	602	596
query69	415	253	254	253
query70	1188	1128	1146	1128
query71	351	246	250	246
query72	6176	4030	4053	4030
query73	761	359	370	359
query74	10062	8901	9108	8901
query75	3396	2635	2670	2635
query76	1911	1020	1120	1020
query77	467	266	281	266
query78	10404	9466	9421	9421
query79	1417	597	591	591
query80	880	420	420	420
query81	510	239	242	239
query82	1276	114	115	114
query83	218	136	158	136
query84	276	70	70	70
query85	891	294	297	294
query86	346	307	288	288
query87	4951	4693	4732	4693
query88	3711	2200	2178	2178
query89	410	294	295	294
query90	1996	186	185	185
query91	139	100	101	100
query92	61	49	46	46
query93	1939	554	553	553
query94	786	300	293	293
query95	341	244	240	240
query96	611	272	272	272
query97	2835	2641	2687	2641
query98	219	200	194	194
query99	1593	1296	1317	1296
Total cold run time: 317917 ms
Total hot run time: 195578 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.12 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fbdcd1697891b62cdc9b458775c43245f2d1c432, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.24	0.07	0.07
query4	1.65	0.11	0.10
query5	0.41	0.40	0.40
query6	1.16	0.64	0.67
query7	0.02	0.01	0.01
query8	0.04	0.02	0.03
query9	0.56	0.50	0.50
query10	0.56	0.55	0.55
query11	0.14	0.10	0.10
query12	0.14	0.12	0.12
query13	0.61	0.60	0.59
query14	2.70	2.82	2.72
query15	0.89	0.82	0.82
query16	0.39	0.37	0.38
query17	0.99	1.04	0.99
query18	0.20	0.19	0.20
query19	1.83	1.84	1.93
query20	0.02	0.00	0.02
query21	15.36	0.59	0.56
query22	3.10	2.09	1.75
query23	16.72	0.96	0.97
query24	3.61	0.41	1.47
query25	0.31	0.18	0.14
query26	0.50	0.14	0.13
query27	0.04	0.04	0.03
query28	10.73	1.09	1.07
query29	12.53	3.28	3.27
query30	0.24	0.06	0.06
query31	2.88	0.37	0.37
query32	3.31	0.46	0.46
query33	2.97	3.08	3.05
query34	17.26	4.52	4.46
query35	4.60	4.53	4.65
query36	0.65	0.51	0.48
query37	0.09	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.03
query40	0.16	0.12	0.12
query41	0.07	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.92 s
Total hot run time: 32.12 s

@zhangstar333
Copy link
Contributor Author

run buildall

github-actions[bot]
github-actions bot reviewed Nov 4, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/vec/exprs/table_function/vposexplode.cpp Show resolved Hide resolved
@zhangstar333 zhangstar333 changed the title [feature](table-function)support pose_explode table function [feature](table-function)support posexplode table function Nov 4, 2024
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.80% (9822/25986)
Line Coverage: 28.98% (81693/281939)
Region Coverage: 28.21% (42102/149255)
Branch Coverage: 24.80% (21368/86168)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d1736863c27ba2b9a2b03b41edaccbe7f57a91e3_d1736863c27ba2b9a2b03b41edaccbe7f57a91e3/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41906 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d1736863c27ba2b9a2b03b41edaccbe7f57a91e3, data reload: false

------ Round 1 ----------------------------------
q1	18187	8290	7399	7399
q2	2044	164	170	164
q3	11147	1158	1199	1158
q4	10823	833	947	833
q5	7777	3100	3084	3084
q6	241	147	145	145
q7	1030	626	612	612
q8	9843	2036	2037	2036
q9	6712	6549	6576	6549
q10	7226	2486	2501	2486
q11	482	274	270	270
q12	407	216	211	211
q13	18922	3040	3039	3039
q14	240	224	232	224
q15	586	535	528	528
q16	1913	583	588	583
q17	1157	661	654	654
q18	7272	6791	6703	6703
q19	1320	988	1051	988
q20	477	173	179	173
q21	3948	3063	3112	3063
q22	1126	1027	1004	1004
Total cold run time: 112880 ms
Total hot run time: 41906 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7228	7286	7346	7286
q2	332	230	231	230
q3	3082	2918	3059	2918
q4	2038	1843	1770	1770
q5	5667	5747	5837	5747
q6	230	140	140	140
q7	2211	1824	1780	1780
q8	3311	3501	3497	3497
q9	8931	8845	8864	8845
q10	3558	3550	3563	3550
q11	613	490	504	490
q12	826	629	630	629
q13	11785	3172	3144	3144
q14	317	281	278	278
q15	587	514	526	514
q16	696	653	638	638
q17	1854	1645	1599	1599
q18	8304	7953	7689	7689
q19	1686	1444	1658	1444
q20	2128	1879	1901	1879
q21	5631	5330	5394	5330
q22	1185	1052	1086	1052
Total cold run time: 72200 ms
Total hot run time: 60449 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196646 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d1736863c27ba2b9a2b03b41edaccbe7f57a91e3, data reload: false

query1	1179	904	901	901
query2	6238	2075	2056	2056
query3	10938	4112	4029	4029
query4	67918	29693	23656	23656
query5	4930	441	436	436
query6	419	200	173	173
query7	5644	285	292	285
query8	301	218	213	213
query9	9046	2649	2630	2630
query10	443	246	249	246
query11	17553	15359	15855	15359
query12	156	110	102	102
query13	1556	432	419	419
query14	11367	7550	7520	7520
query15	217	197	180	180
query16	6716	465	459	459
query17	1036	555	560	555
query18	1714	295	295	295
query19	195	160	146	146
query20	118	110	111	110
query21	197	103	102	102
query22	4829	4495	4379	4379
query23	34836	34023	34190	34023
query24	6006	2784	2741	2741
query25	511	396	408	396
query26	646	159	158	158
query27	1699	291	284	284
query28	4159	2460	2421	2421
query29	695	433	429	429
query30	241	155	149	149
query31	981	808	772	772
query32	70	54	58	54
query33	400	262	274	262
query34	904	506	505	505
query35	850	739	731	731
query36	1068	963	959	959
query37	118	85	74	74
query38	4324	4243	4300	4243
query39	1602	1428	1408	1408
query40	202	99	97	97
query41	47	44	45	44
query42	108	100	99	99
query43	519	490	472	472
query44	1163	826	823	823
query45	183	163	163	163
query46	1131	691	686	686
query47	1964	1877	1885	1877
query48	419	340	327	327
query49	745	416	397	397
query50	794	386	384	384
query51	7353	7072	7286	7072
query52	106	97	86	86
query53	253	177	177	177
query54	516	415	397	397
query55	83	76	74	74
query56	258	242	252	242
query57	1326	1216	1166	1166
query58	224	204	205	204
query59	3178	3048	3159	3048
query60	267	238	245	238
query61	103	103	113	103
query62	794	680	678	678
query63	210	188	189	188
query64	1377	651	614	614
query65	3260	3208	3176	3176
query66	709	297	302	297
query67	15943	15852	15780	15780
query68	3654	585	565	565
query69	432	256	248	248
query70	1184	1153	1050	1050
query71	355	255	247	247
query72	6412	3948	3955	3948
query73	765	370	357	357
query74	10009	8988	8985	8985
query75	3390	2696	2690	2690
query76	1951	1035	1135	1035
query77	496	288	282	282
query78	10394	9458	9369	9369
query79	2071	605	588	588
query80	1312	437	421	421
query81	528	240	245	240
query82	1291	114	115	114
query83	205	140	140	140
query84	272	74	72	72
query85	990	293	290	290
query86	438	302	298	298
query87	4828	4605	4694	4605
query88	3809	2176	2155	2155
query89	409	292	292	292
query90	1934	188	180	180
query91	134	102	109	102
query92	63	47	49	47
query93	2888	544	547	544
query94	862	305	282	282
query95	357	247	238	238
query96	643	281	275	275
query97	2877	2717	2714	2714
query98	211	202	199	199
query99	1600	1305	1313	1305
Total cold run time: 321819 ms
Total hot run time: 196646 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.53 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d1736863c27ba2b9a2b03b41edaccbe7f57a91e3, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.02
query3	0.23	0.06	0.06
query4	1.64	0.10	0.10
query5	0.41	0.38	0.42
query6	1.15	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.06	0.03
query9	0.57	0.50	0.50
query10	0.56	0.56	0.56
query11	0.14	0.09	0.11
query12	0.14	0.11	0.11
query13	0.61	0.59	0.60
query14	2.74	2.85	2.75
query15	0.91	0.83	0.82
query16	0.41	0.38	0.37
query17	1.03	1.01	1.09
query18	0.20	0.19	0.20
query19	1.88	1.89	1.99
query20	0.01	0.01	0.01
query21	15.37	0.58	0.59
query22	2.56	1.87	1.96
query23	17.16	1.04	0.80
query24	2.74	0.89	1.23
query25	0.20	0.06	0.04
query26	0.55	0.14	0.12
query27	0.04	0.04	0.04
query28	10.90	1.09	1.05
query29	12.59	3.24	3.25
query30	0.24	0.06	0.06
query31	2.88	0.37	0.38
query32	3.29	0.47	0.46
query33	3.00	3.04	3.06
query34	16.99	4.47	4.47
query35	4.57	4.56	4.54
query36	0.66	0.51	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 107 s
Total hot run time: 32.53 s

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41231 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 509169d51eba44cdd1ecc29c516114130037ad8a, data reload: false

------ Round 1 ----------------------------------
q1	17556	7481	7265	7265
q2	2047	167	166	166
q3	10574	1080	1140	1080
q4	10566	804	792	792
q5	7761	3049	3066	3049
q6	231	148	145	145
q7	999	612	602	602
q8	9381	1960	2037	1960
q9	6477	6367	6494	6367
q10	7025	2424	2403	2403
q11	450	253	248	248
q12	407	210	209	209
q13	17771	3005	3035	3005
q14	229	210	205	205
q15	595	513	504	504
q16	622	582	597	582
q17	961	586	561	561
q18	7348	6753	6699	6699
q19	1350	1068	998	998
q20	467	184	183	183
q21	4095	3387	3199	3199
q22	1138	1009	1020	1009
Total cold run time: 108050 ms
Total hot run time: 41231 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7374	7344	7307	7307
q2	329	232	234	232
q3	3023	2969	2997	2969
q4	2114	1879	1775	1775
q5	5769	5775	5847	5775
q6	224	140	142	140
q7	2300	1860	1811	1811
q8	3464	3533	3441	3441
q9	9061	9134	9189	9134
q10	3652	3640	3938	3640
q11	619	516	529	516
q12	849	694	628	628
q13	9695	3316	3288	3288
q14	317	278	289	278
q15	598	538	528	528
q16	695	631	638	631
q17	1859	1671	1649	1649
q18	8378	7847	7507	7507
q19	1759	1482	1512	1482
q20	2128	1888	1901	1888
q21	5636	5439	5315	5315
q22	1184	1042	1033	1033
Total cold run time: 71027 ms
Total hot run time: 60967 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195665 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 509169d51eba44cdd1ecc29c516114130037ad8a, data reload: false

query1	1203	911	899	899
query2	6242	2058	2066	2058
query3	10908	4102	4122	4102
query4	68208	29327	23552	23552
query5	4915	457	450	450
query6	396	182	170	170
query7	5617	291	295	291
query8	299	233	224	224
query9	9102	2618	2596	2596
query10	445	281	239	239
query11	17195	15404	15775	15404
query12	156	105	100	100
query13	1628	447	420	420
query14	10297	7364	6922	6922
query15	207	175	184	175
query16	7041	486	444	444
query17	1022	535	540	535
query18	1779	275	291	275
query19	185	138	143	138
query20	117	111	115	111
query21	204	95	95	95
query22	4769	4560	4473	4473
query23	34370	33824	34029	33824
query24	5961	2755	2712	2712
query25	482	382	367	367
query26	644	152	150	150
query27	1709	270	281	270
query28	4137	2410	2375	2375
query29	646	401	393	393
query30	229	155	151	151
query31	949	802	815	802
query32	70	50	52	50
query33	441	253	247	247
query34	893	497	501	497
query35	845	726	718	718
query36	1061	942	966	942
query37	112	66	71	66
query38	4226	4238	4217	4217
query39	1479	1400	1421	1400
query40	189	92	91	91
query41	48	41	45	41
query42	105	96	92	92
query43	523	503	475	475
query44	1152	816	799	799
query45	177	161	159	159
query46	1125	684	691	684
query47	1937	1850	1871	1850
query48	404	301	313	301
query49	721	373	378	373
query50	798	386	381	381
query51	7288	7059	7125	7059
query52	96	85	82	82
query53	246	176	173	173
query54	504	387	394	387
query55	73	74	70	70
query56	276	227	231	227
query57	1321	1209	1160	1160
query58	206	196	195	195
query59	3148	2910	3046	2910
query60	257	247	242	242
query61	100	106	100	100
query62	769	686	689	686
query63	206	188	176	176
query64	1344	740	699	699
query65	3297	3190	3209	3190
query66	717	305	313	305
query67	15904	15750	15874	15750
query68	2645	592	593	592
query69	414	265	255	255
query70	1216	1081	1135	1081
query71	354	256	254	254
query72	5326	4265	4199	4199
query73	757	357	358	357
query74	9565	8996	9029	8996
query75	3386	2691	2683	2683
query76	1456	1093	1027	1027
query77	477	285	270	270
query78	10244	9431	9409	9409
query79	1750	593	602	593
query80	1226	422	429	422
query81	529	241	240	240
query82	464	110	122	110
query83	162	133	139	133
query84	271	68	68	68
query85	975	303	291	291
query86	395	305	287	287
query87	4717	4756	4590	4590
query88	3652	2190	2139	2139
query89	410	290	280	280
query90	1975	180	179	179
query91	135	101	101	101
query92	57	48	53	48
query93	1933	538	543	538
query94	863	299	298	298
query95	341	238	237	237
query96	611	279	277	277
query97	2860	2715	2741	2715
query98	209	206	193	193
query99	1584	1292	1298	1292
Total cold run time: 314079 ms
Total hot run time: 195665 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.66 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 509169d51eba44cdd1ecc29c516114130037ad8a, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.02
query3	0.23	0.06	0.06
query4	1.65	0.09	0.10
query5	0.41	0.39	0.40
query6	1.15	0.64	0.64
query7	0.01	0.02	0.02
query8	0.06	0.03	0.03
query9	0.56	0.50	0.49
query10	0.55	0.57	0.55
query11	0.14	0.10	0.10
query12	0.15	0.12	0.10
query13	0.59	0.59	0.59
query14	2.68	2.70	2.82
query15	0.90	0.82	0.81
query16	0.38	0.37	0.37
query17	1.01	1.02	1.01
query18	0.20	0.20	0.20
query19	1.93	1.90	1.94
query20	0.01	0.01	0.01
query21	15.36	0.58	0.59
query22	2.28	1.78	2.08
query23	17.01	0.82	0.83
query24	3.23	1.12	1.37
query25	0.36	0.14	0.23
query26	0.40	0.13	0.13
query27	0.04	0.05	0.03
query28	10.46	1.09	1.08
query29	12.55	3.22	3.26
query30	0.25	0.06	0.06
query31	2.87	0.38	0.37
query32	3.29	0.48	0.46
query33	3.06	2.98	3.12
query34	16.94	4.49	4.51
query35	4.58	4.54	4.53
query36	0.65	0.48	0.47
query37	0.09	0.05	0.06
query38	0.04	0.04	0.03
query39	0.02	0.02	0.03
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.5 s
Total hot run time: 32.66 s

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41181 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 45ddfa7a3d63cac80be12f9b5ddd7fc085f6294d, data reload: false

------ Round 1 ----------------------------------
q1	17597	7454	7318	7318
q2	2041	177	181	177
q3	10539	1127	1241	1127
q4	10223	869	839	839
q5	7770	3047	3049	3047
q6	237	146	147	146
q7	1000	600	599	599
q8	9351	1933	2022	1933
q9	6667	6396	6423	6396
q10	7048	2406	2456	2406
q11	456	260	270	260
q12	411	217	213	213
q13	17758	3072	3018	3018
q14	237	207	208	207
q15	572	523	522	522
q16	620	594	584	584
q17	967	535	568	535
q18	7356	6721	6781	6721
q19	1328	1019	1000	1000
q20	484	178	181	178
q21	3975	3225	2945	2945
q22	1123	1010	1018	1010
Total cold run time: 107760 ms
Total hot run time: 41181 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7359	7302	7257	7257
q2	323	226	230	226
q3	2953	2805	2819	2805
q4	1942	1747	1704	1704
q5	5440	5568	5466	5466
q6	219	136	140	136
q7	2136	1744	1724	1724
q8	3254	3437	3413	3413
q9	8541	8556	8523	8523
q10	3474	3491	3456	3456
q11	588	506	505	505
q12	771	611	599	599
q13	9481	2991	3024	2991
q14	288	263	265	263
q15	550	519	521	519
q16	661	628	627	627
q17	1836	1633	1571	1571
q18	7747	7340	7648	7340
q19	1689	1544	1577	1544
q20	2035	1802	1794	1794
q21	5268	5204	5047	5047
q22	1098	1008	986	986
Total cold run time: 67653 ms
Total hot run time: 58496 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.86% (9840/25993)
Line Coverage: 29.03% (81844/281972)
Region Coverage: 28.26% (42164/149216)
Branch Coverage: 24.84% (21397/86124)
Coverage Report: http://coverage.selectdb-in.cc/coverage/45ddfa7a3d63cac80be12f9b5ddd7fc085f6294d_45ddfa7a3d63cac80be12f9b5ddd7fc085f6294d/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 191516 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 45ddfa7a3d63cac80be12f9b5ddd7fc085f6294d, data reload: false

query1	992	358	377	358
query2	6512	2037	2063	2037
query3	6792	231	212	212
query4	34294	23888	23684	23684
query5	4368	445	454	445
query6	271	175	185	175
query7	4595	289	296	289
query8	287	221	218	218
query9	9566	2639	2626	2626
query10	462	245	247	245
query11	18146	15373	15377	15373
query12	162	104	100	100
query13	1682	414	409	409
query14	10308	7135	7330	7135
query15	302	175	173	173
query16	8048	437	491	437
query17	1726	572	577	572
query18	2110	294	305	294
query19	362	147	144	144
query20	115	107	111	107
query21	205	101	100	100
query22	4620	4115	4255	4115
query23	35013	33948	34165	33948
query24	11243	2702	2806	2702
query25	693	402	406	402
query26	1365	159	156	156
query27	2796	274	279	274
query28	7948	2410	2409	2409
query29	876	424	427	424
query30	319	160	153	153
query31	1033	822	819	819
query32	96	56	66	56
query33	773	271	268	268
query34	996	511	502	502
query35	880	739	726	726
query36	1089	970	967	967
query37	136	82	78	78
query38	4405	4234	4270	4234
query39	1458	1432	1445	1432
query40	275	100	98	98
query41	50	46	47	46
query42	109	100	95	95
query43	527	489	480	480
query44	1261	824	801	801
query45	186	166	162	162
query46	1131	678	674	674
query47	1988	1889	1889	1889
query48	417	312	322	312
query49	1145	399	401	399
query50	824	388	395	388
query51	7379	7195	7150	7150
query52	98	88	89	88
query53	251	182	182	182
query54	1224	418	401	401
query55	78	74	79	74
query56	270	276	230	230
query57	1314	1171	1157	1157
query58	231	201	205	201
query59	3122	3091	2928	2928
query60	277	239	248	239
query61	109	110	102	102
query62	886	681	678	678
query63	209	181	187	181
query64	5275	637	620	620
query65	3293	3216	3184	3184
query66	1462	317	301	301
query67	16256	15697	15659	15659
query68	4950	537	549	537
query69	430	282	245	245
query70	1193	1078	1147	1078
query71	423	258	240	240
query72	6432	4105	4035	4035
query73	763	347	362	347
query74	10478	9046	9047	9046
query75	3703	2671	2676	2671
query76	2918	1040	1104	1040
query77	483	266	263	263
query78	10538	9395	9374	9374
query79	1348	585	590	585
query80	952	418	420	418
query81	541	237	238	237
query82	1048	127	119	119
query83	208	133	138	133
query84	235	68	76	68
query85	1250	301	278	278
query86	380	298	298	298
query87	4922	4805	4715	4715
query88	3303	2171	2146	2146
query89	392	288	283	283
query90	2047	192	183	183
query91	136	98	101	98
query92	57	48	52	48
query93	1054	534	529	529
query94	969	292	303	292
query95	346	250	248	248
query96	603	276	280	276
query97	2872	2720	2664	2664
query98	209	199	200	199
query99	1543	1328	1306	1306
Total cold run time: 303417 ms
Total hot run time: 191516 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.67 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 45ddfa7a3d63cac80be12f9b5ddd7fc085f6294d, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.02	0.03
query3	0.24	0.06	0.07
query4	1.63	0.10	0.11
query5	0.42	0.39	0.40
query6	1.17	0.68	0.65
query7	0.02	0.02	0.01
query8	0.04	0.05	0.03
query9	0.57	0.50	0.48
query10	0.54	0.55	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.59
query14	2.73	2.73	2.73
query15	0.90	0.82	0.81
query16	0.39	0.38	0.39
query17	1.06	1.05	1.04
query18	0.24	0.22	0.22
query19	1.97	1.79	1.86
query20	0.02	0.01	0.01
query21	15.35	0.58	0.57
query22	2.84	2.08	1.68
query23	16.97	1.10	0.76
query24	3.73	0.51	0.59
query25	0.33	0.12	0.04
query26	0.36	0.15	0.13
query27	0.05	0.04	0.04
query28	10.99	1.09	1.06
query29	12.53	3.22	3.22
query30	0.24	0.06	0.07
query31	2.84	0.37	0.38
query32	3.29	0.46	0.45
query33	2.97	2.99	3.07
query34	17.02	4.46	4.40
query35	4.51	4.55	4.52
query36	0.64	0.50	0.49
query37	0.09	0.05	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 108.07 s
Total hot run time: 31.67 s

HappenLee
HappenLee approved these changes Nov 6, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 6, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Nov 6, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 6, 2024

PR approved by anyone and no changes requested.

@zhangstar333 zhangstar333 merged commit 360d354 into apache:master Nov 6, 2024
25 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 23, 2024
@zhangstar333
[feature](table-function)support posexplode table function (#43221)
54da28c 
### What problem does this PR solve?
<!--
You need to clearly describe your PR in this part:

1. What problem was fixed (it's best to include specific error reporting
information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what
is it now, why was it modified, and what possible impacts might there
be.
3. What features were added. Why this function was added.
4. Which codes were refactored and why this part of the code was
refactored.
5. Which functions were optimized and what is the difference before and
after the optimization.

The description of the PR needs to enable reviewers to quickly and
clearly understand the logic of the code modification.
-->

<!--
If there are related issues, please fill in the issue number.
- If you want the issue to be closed after the PR is merged, please use
"close #12345". Otherwise, use "ref #12345"
-->
Issue Number: close #xxx

<!--
If this PR is followup a preivous PR, for example, fix the bug that
introduced by a related PR,
link the PR here
-->
Related PR: #xxx

Problem Summary:
support posexplode table function
doc: apache/doris-website#1283
@github-actions github-actions bot mentioned this pull request Dec 23, 2024
Merged
zhangstar333 added a commit to zhangstar333/incubator-doris that referenced this pull request Dec 23, 2024
@zhangstar333
[feature](table-function)support posexplode table function (apache#43221
8b067a0 
)

<!--
You need to clearly describe your PR in this part:

1. What problem was fixed (it's best to include specific error reporting
information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what
is it now, why was it modified, and what possible impacts might there
be.
3. What features were added. Why this function was added.
4. Which codes were refactored and why this part of the code was
refactored.
5. Which functions were optimized and what is the difference before and
after the optimization.

The description of the PR needs to enable reviewers to quickly and
clearly understand the logic of the code modification.
-->

<!--
If there are related issues, please fill in the issue number.
- If you want the issue to be closed after the PR is merged, please use
"close apache#12345". Otherwise, use "ref apache#12345"
-->
Issue Number: close #xxx

<!--
If this PR is followup a preivous PR, for example, fix the bug that
introduced by a related PR,
link the PR here
-->
Related PR: #xxx

Problem Summary:
support posexplode table function
doc: apache/doris-website#1283
@zhangstar333 zhangstar333 mentioned this pull request Dec 23, 2024
Merged
yiguolei pushed a commit that referenced this pull request Dec 24, 2024
@zhangstar333
[cherry-pick](branch-21)support posexplode table function (#43221) (#…
f0031d9 
…45783)

cherry-pick from master #43221
zhangstar333 added a commit that referenced this pull request Dec 26, 2024
@github-actions @zhangstar333
branch-3.0: [feature](table-function)support posexplode table function …
0cfdf4b 
…#43221 (#45782)

Cherry-picked from #43221

Co-authored-by: zhangstar333 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@cambyzju cambyzju cambyzju left review comments

@HappenLee HappenLee HappenLee approved these changes

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@zhangstar333 @doris-robot @BiteTheDDDDt @HappenLee @cambyzju @yiguolei