Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] (nereids)implement SHOW CATALOG COMMAND in nereids #44633

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Yao-MR
Copy link
Contributor

@Yao-MR Yao-MR commented Nov 26, 2024

What problem does this PR solve?

Issue Number: close #42745

Related PR:

NO

Problem Summary:

Implement SHOW CATALOG COMMAND in nereids

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Yao-MR Yao-MR force-pushed the enhancement_nereids_show-catalog-support branch from d0fc1e6 to 775a876 Compare November 26, 2024 12:01
@Yao-MR
Copy link
Contributor Author

Yao-MR commented Nov 26, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39957 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 79efca542d804f74b316a205b7a153101170f377, data reload: false

------ Round 1 ----------------------------------
q1	17677	7518	7268	7268
q2	2065	171	159	159
q3	10718	1088	1199	1088
q4	10570	769	861	769
q5	7601	2717	2649	2649
q6	238	145	147	145
q7	978	629	596	596
q8	9226	1881	1909	1881
q9	6564	6431	6438	6431
q10	7055	2276	2285	2276
q11	475	278	268	268
q12	408	212	215	212
q13	17767	3038	2974	2974
q14	251	214	220	214
q15	568	549	530	530
q16	651	606	593	593
q17	973	548	555	548
q18	7276	6787	6707	6707
q19	1331	930	982	930
q20	458	192	200	192
q21	3911	3276	3214	3214
q22	371	325	313	313
Total cold run time: 107132 ms
Total hot run time: 39957 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7296	7220	7551	7220
q2	326	226	227	226
q3	2930	2842	2925	2842
q4	2032	1829	1831	1829
q5	5686	5661	5634	5634
q6	222	140	148	140
q7	2261	1777	1832	1777
q8	3416	3574	3496	3496
q9	8838	8922	8882	8882
q10	3595	3538	3544	3538
q11	613	532	500	500
q12	849	604	593	593
q13	11053	3206	3332	3206
q14	308	264	270	264
q15	573	528	519	519
q16	685	662	655	655
q17	1856	1630	1631	1630
q18	8476	7761	7644	7644
q19	2498	1519	1465	1465
q20	2139	1836	1867	1836
q21	5593	5385	5301	5301
q22	656	597	566	566
Total cold run time: 71901 ms
Total hot run time: 59763 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196571 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 79efca542d804f74b316a205b7a153101170f377, data reload: false

query1	1244	955	953	953
query2	6241	2087	2009	2009
query3	10825	3935	3925	3925
query4	67586	27471	23511	23511
query5	4946	458	440	440
query6	402	175	186	175
query7	5522	301	294	294
query8	314	226	219	219
query9	8712	2630	2622	2622
query10	437	259	266	259
query11	17296	15346	15913	15346
query12	155	106	103	103
query13	1467	407	429	407
query14	10733	7550	7540	7540
query15	201	190	179	179
query16	7170	454	491	454
query17	1040	597	549	549
query18	1878	284	293	284
query19	195	191	156	156
query20	113	109	108	108
query21	211	101	103	101
query22	4698	4581	4807	4581
query23	34614	34068	35026	34068
query24	5512	2536	2549	2536
query25	481	388	395	388
query26	645	151	152	151
query27	1903	282	305	282
query28	4723	2499	2487	2487
query29	709	461	440	440
query30	218	145	145	145
query31	1007	816	847	816
query32	66	55	55	55
query33	392	285	279	279
query34	922	504	525	504
query35	872	738	741	738
query36	1079	978	971	971
query37	118	71	70	70
query38	4499	4357	4373	4357
query39	1554	1510	1472	1472
query40	206	100	104	100
query41	47	49	53	49
query42	113	99	97	97
query43	535	491	481	481
query44	1177	817	831	817
query45	186	166	163	163
query46	1155	732	726	726
query47	2094	1922	1949	1922
query48	408	312	316	312
query49	718	384	390	384
query50	859	400	410	400
query51	7445	7217	7049	7049
query52	99	91	89	89
query53	249	179	175	175
query54	499	390	391	390
query55	79	74	73	73
query56	269	238	236	236
query57	1301	1186	1155	1155
query58	213	220	210	210
query59	3160	2941	3069	2941
query60	265	243	250	243
query61	113	111	104	104
query62	805	663	669	663
query63	219	184	182	182
query64	1369	665	714	665
query65	3323	3198	3190	3190
query66	696	294	297	294
query67	16199	15638	15764	15638
query68	4078	571	538	538
query69	409	254	260	254
query70	1165	1065	1130	1065
query71	343	238	254	238
query72	6117	4077	4135	4077
query73	760	363	358	358
query74	10150	8997	9054	8997
query75	3384	2686	2684	2684
query76	2101	1009	1071	1009
query77	496	307	267	267
query78	10424	9368	9482	9368
query79	2222	581	597	581
query80	1370	416	456	416
query81	523	230	227	227
query82	1272	124	120	120
query83	177	152	156	152
query84	292	72	71	71
query85	988	303	294	294
query86	416	300	299	299
query87	4809	4614	4586	4586
query88	3981	2168	2124	2124
query89	416	290	288	288
query90	1995	185	187	185
query91	140	102	102	102
query92	64	46	50	46
query93	2827	554	547	547
query94	931	283	300	283
query95	362	259	257	257
query96	665	282	273	273
query97	2842	2698	2628	2628
query98	228	200	198	198
query99	1561	1317	1325	1317
Total cold run time: 322009 ms
Total hot run time: 196571 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.97 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 79efca542d804f74b316a205b7a153101170f377, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.08	0.07
query4	1.61	0.10	0.11
query5	0.42	0.40	0.40
query6	1.17	0.68	0.65
query7	0.01	0.02	0.01
query8	0.03	0.04	0.03
query9	0.58	0.51	0.51
query10	0.55	0.54	0.55
query11	0.13	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.83	2.81	2.83
query15	0.90	0.83	0.82
query16	0.38	0.38	0.37
query17	1.00	1.01	1.01
query18	0.21	0.22	0.21
query19	1.96	1.90	1.98
query20	0.01	0.01	0.02
query21	15.37	0.59	0.60
query22	2.73	1.82	2.07
query23	17.13	0.94	0.80
query24	2.94	1.21	0.38
query25	0.27	0.14	0.09
query26	0.45	0.15	0.14
query27	0.05	0.04	0.04
query28	11.25	1.10	1.06
query29	12.54	3.23	3.21
query30	0.24	0.06	0.07
query31	2.87	0.39	0.37
query32	3.27	0.46	0.46
query33	3.00	2.97	3.02
query34	16.96	4.44	4.50
query35	4.56	4.47	4.50
query36	0.68	0.50	0.47
query37	0.09	0.05	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.15	0.12	0.12
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.67 s
Total hot run time: 31.97 s

@Yao-MR
Copy link
Contributor Author

Yao-MR commented Nov 27, 2024

run p0

@Yao-MR Yao-MR force-pushed the enhancement_nereids_show-catalog-support branch from e3f3c29 to 0b61c75 Compare November 27, 2024 06:16
@Yao-MR
Copy link
Contributor Author

Yao-MR commented Nov 28, 2024

hi @LiBinfeng-01 @starocean999 @CalvinKirs, pls help review when you are available

@Yao-MR
Copy link
Contributor Author

Yao-MR commented Nov 28, 2024

run cloud_p0

@Yao-MR Yao-MR force-pushed the enhancement_nereids_show-catalog-support branch from b4b035b to 7fbba8d Compare November 28, 2024 06:56
CalvinKirs
CalvinKirs previously approved these changes Nov 28, 2024
@CalvinKirs CalvinKirs requested a review from morrySnow November 28, 2024 07:05
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 28, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@LiBinfeng-01
Copy link
Collaborator

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39966 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 041ad5b3e1cc4a6234f242ad10d8f227cefe0b9d, data reload: false

------ Round 1 ----------------------------------
q1	17643	7449	7251	7251
q2	2039	172	179	172
q3	10545	1125	1221	1125
q4	10229	714	737	714
q5	7590	2717	2664	2664
q6	236	145	144	144
q7	985	636	612	612
q8	9239	1841	1919	1841
q9	6530	6431	6392	6392
q10	6963	2337	2346	2337
q11	466	245	255	245
q12	426	218	204	204
q13	17793	3060	3010	3010
q14	255	219	212	212
q15	575	539	520	520
q16	631	595	579	579
q17	962	596	531	531
q18	7439	6735	6704	6704
q19	1363	978	1024	978
q20	479	183	179	179
q21	4431	3235	3251	3235
q22	389	318	317	317
Total cold run time: 107208 ms
Total hot run time: 39966 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7220	7257	7294	7257
q2	330	228	231	228
q3	2881	2823	3112	2823
q4	2122	1843	1798	1798
q5	5559	5723	5678	5678
q6	224	137	139	137
q7	2238	1792	1853	1792
q8	3386	3540	3496	3496
q9	8944	8837	8847	8837
q10	3632	3567	3549	3549
q11	607	518	498	498
q12	806	636	582	582
q13	11286	3215	3247	3215
q14	300	272	277	272
q15	571	520	537	520
q16	670	675	644	644
q17	1783	1593	1592	1592
q18	7896	7276	7315	7276
q19	1648	1524	1620	1524
q20	2084	1814	1842	1814
q21	5323	5341	5295	5295
q22	671	570	548	548
Total cold run time: 70181 ms
Total hot run time: 59375 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191342 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 041ad5b3e1cc4a6234f242ad10d8f227cefe0b9d, data reload: false

query1	966	380	391	380
query2	6521	2039	2038	2038
query3	6712	213	209	209
query4	34403	23505	23686	23505
query5	4392	505	477	477
query6	287	211	184	184
query7	4620	307	308	307
query8	292	231	219	219
query9	9528	2747	2735	2735
query10	481	263	252	252
query11	18220	15230	15408	15230
query12	154	104	105	104
query13	1665	422	413	413
query14	9142	6694	7838	6694
query15	262	171	181	171
query16	8187	456	438	438
query17	1649	551	532	532
query18	2144	288	285	285
query19	360	161	146	146
query20	129	112	114	112
query21	209	102	105	102
query22	4550	4333	4412	4333
query23	35189	34476	34499	34476
query24	10440	2488	2482	2482
query25	654	382	390	382
query26	1502	146	149	146
query27	2943	276	290	276
query28	8321	2465	2447	2447
query29	928	410	409	409
query30	293	149	146	146
query31	1026	815	820	815
query32	100	59	58	58
query33	783	279	282	279
query34	964	514	517	514
query35	883	735	750	735
query36	1086	939	973	939
query37	141	75	74	74
query38	4391	4210	4328	4210
query39	1538	1434	1413	1413
query40	283	101	103	101
query41	48	45	44	44
query42	113	102	101	101
query43	574	486	488	486
query44	1237	805	794	794
query45	187	167	171	167
query46	1126	707	699	699
query47	1945	1820	1864	1820
query48	409	310	305	305
query49	1148	386	365	365
query50	809	382	388	382
query51	7331	7116	7078	7078
query52	100	91	93	91
query53	256	177	177	177
query54	1213	406	407	406
query55	83	77	78	77
query56	271	247	246	246
query57	1313	1198	1159	1159
query58	229	220	220	220
query59	3069	3060	2909	2909
query60	292	273	263	263
query61	142	137	131	131
query62	914	663	703	663
query63	216	191	181	181
query64	5330	668	627	627
query65	3329	3205	3247	3205
query66	1445	321	360	321
query67	16121	15558	15707	15558
query68	4553	542	549	542
query69	414	253	257	253
query70	1219	1099	1139	1099
query71	339	243	246	243
query72	6447	4128	3858	3858
query73	762	365	358	358
query74	10491	9129	9059	9059
query75	3422	2677	2674	2674
query76	2658	1121	1090	1090
query77	390	277	279	277
query78	10361	9442	9446	9442
query79	1789	594	614	594
query80	1146	441	485	441
query81	549	248	238	238
query82	917	117	116	116
query83	251	153	151	151
query84	237	73	71	71
query85	1334	318	300	300
query86	426	273	315	273
query87	4895	4621	4553	4553
query88	3823	2258	2259	2258
query89	404	301	308	301
query90	2078	192	193	192
query91	140	104	106	104
query92	64	50	52	50
query93	1712	539	543	539
query94	1017	359	291	291
query95	354	246	245	245
query96	620	282	291	282
query97	2844	2702	2722	2702
query98	221	191	199	191
query99	1562	1321	1312	1312
Total cold run time: 303027 ms
Total hot run time: 191342 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.23 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 041ad5b3e1cc4a6234f242ad10d8f227cefe0b9d, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.04	0.03
query3	0.24	0.08	0.07
query4	1.62	0.10	0.11
query5	0.43	0.41	0.42
query6	1.16	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.57	0.54	0.50
query10	0.55	0.56	0.56
query11	0.15	0.10	0.10
query12	0.15	0.11	0.12
query13	0.62	0.61	0.61
query14	2.85	2.74	2.84
query15	0.91	0.84	0.84
query16	0.39	0.40	0.38
query17	1.04	1.05	1.07
query18	0.22	0.20	0.21
query19	1.96	1.81	1.96
query20	0.01	0.01	0.01
query21	15.36	0.58	0.57
query22	2.27	2.98	2.45
query23	17.00	1.16	0.75
query24	3.00	0.86	1.02
query25	0.33	0.08	0.09
query26	0.38	0.14	0.14
query27	0.03	0.04	0.05
query28	10.77	1.10	1.07
query29	12.61	3.26	3.24
query30	0.24	0.06	0.07
query31	2.86	0.38	0.39
query32	3.28	0.47	0.47
query33	2.98	3.02	3.08
query34	17.14	4.51	4.51
query35	4.57	4.54	4.53
query36	0.69	0.51	0.50
query37	0.09	0.07	0.06
query38	0.06	0.04	0.04
query39	0.04	0.02	0.02
query40	0.15	0.12	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.02 s
Total hot run time: 33.23 s

@Yao-MR Yao-MR force-pushed the enhancement_nereids_show-catalog-support branch from 041ad5b to 18d9fc1 Compare November 29, 2024 08:22
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 29, 2024
@Yao-MR
Copy link
Contributor Author

Yao-MR commented Nov 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39993 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18d9fc12be2985f940d8463f838c0682fe485579, data reload: false

------ Round 1 ----------------------------------
q1	17727	7504	7291	7291
q2	2050	187	177	177
q3	10691	1067	1143	1067
q4	10550	751	777	751
q5	7617	2775	2746	2746
q6	242	153	148	148
q7	992	621	607	607
q8	9242	1896	1940	1896
q9	6758	6513	6539	6513
q10	7000	2285	2328	2285
q11	458	257	254	254
q12	422	220	218	218
q13	17763	3056	2998	2998
q14	251	212	212	212
q15	572	529	516	516
q16	656	604	592	592
q17	972	566	482	482
q18	7269	6565	6592	6565
q19	1342	1030	975	975
q20	480	190	183	183
q21	4011	3207	3288	3207
q22	383	314	310	310
Total cold run time: 107448 ms
Total hot run time: 39993 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7260	7279	7237	7237
q2	327	232	233	232
q3	2951	2813	2944	2813
q4	2072	1828	1868	1828
q5	5701	5721	5686	5686
q6	230	143	148	143
q7	2300	1823	1804	1804
q8	3399	3553	3545	3545
q9	8983	9166	9085	9085
q10	3652	3545	3565	3545
q11	605	503	510	503
q12	875	607	616	607
q13	11789	3292	3266	3266
q14	329	279	281	279
q15	568	536	521	521
q16	672	679	674	674
q17	1869	1609	1617	1609
q18	8486	7980	7581	7581
q19	2327	1467	1604	1467
q20	2114	1868	1883	1868
q21	5555	5388	5329	5329
q22	631	583	585	583
Total cold run time: 72695 ms
Total hot run time: 60205 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197072 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18d9fc12be2985f940d8463f838c0682fe485579, data reload: false

query1	1275	998	933	933
query2	6263	2071	2041	2041
query3	10981	4414	4414	4414
query4	66085	29631	23658	23658
query5	4992	457	484	457
query6	419	184	179	179
query7	5666	299	298	298
query8	311	228	234	228
query9	9230	2698	2660	2660
query10	469	259	252	252
query11	17349	15436	15976	15436
query12	158	106	121	106
query13	1555	434	417	417
query14	10122	7904	6698	6698
query15	208	190	199	190
query16	7176	511	465	465
query17	1393	579	582	579
query18	1853	293	297	293
query19	235	148	146	146
query20	119	116	111	111
query21	241	106	99	99
query22	4852	4561	4604	4561
query23	34881	34065	34447	34065
query24	5341	2443	2435	2435
query25	489	401	386	386
query26	647	153	145	145
query27	1770	287	287	287
query28	4845	2462	2468	2462
query29	680	408	402	402
query30	222	151	148	148
query31	1022	827	853	827
query32	64	56	59	56
query33	423	312	278	278
query34	904	517	506	506
query35	874	764	781	764
query36	1122	965	957	957
query37	125	73	77	73
query38	4471	4479	4510	4479
query39	1531	1467	1439	1439
query40	213	100	99	99
query41	45	41	49	41
query42	110	97	102	97
query43	539	503	501	501
query44	1190	838	846	838
query45	193	175	169	169
query46	1150	718	749	718
query47	2022	1914	1895	1895
query48	432	316	331	316
query49	725	381	403	381
query50	856	385	396	385
query51	7508	7328	7162	7162
query52	98	86	87	86
query53	256	175	177	175
query54	508	393	385	385
query55	77	76	73	73
query56	258	237	230	230
query57	1247	1120	1100	1100
query58	213	215	209	209
query59	3297	3261	3335	3261
query60	265	244	250	244
query61	105	106	104	104
query62	781	660	684	660
query63	215	185	188	185
query64	1347	674	638	638
query65	3278	3209	3240	3209
query66	699	299	313	299
query67	16080	15859	15578	15578
query68	4320	571	552	552
query69	422	255	256	255
query70	1204	1158	1121	1121
query71	383	247	258	247
query72	6401	4024	4101	4024
query73	763	355	365	355
query74	10235	9040	9093	9040
query75	3358	2694	2665	2665
query76	1869	1084	1126	1084
query77	475	277	296	277
query78	10506	9547	9471	9471
query79	1558	577	590	577
query80	892	427	433	427
query81	515	237	231	231
query82	235	123	123	123
query83	192	140	143	140
query84	288	171	71	71
query85	940	300	295	295
query86	373	308	315	308
query87	4756	4740	4540	4540
query88	3880	2199	2179	2179
query89	413	295	293	293
query90	2159	188	188	188
query91	131	102	103	102
query92	63	50	49	49
query93	2447	542	538	538
query94	920	270	293	270
query95	342	245	250	245
query96	630	270	316	270
query97	2837	2673	2681	2673
query98	214	203	193	193
query99	1591	1348	1301	1301
Total cold run time: 319382 ms
Total hot run time: 197072 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.21 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 18d9fc12be2985f940d8463f838c0682fe485579, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.02
query3	0.23	0.07	0.07
query4	1.62	0.10	0.11
query5	0.43	0.40	0.39
query6	1.15	0.66	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.04
query9	0.57	0.51	0.50
query10	0.55	0.58	0.56
query11	0.13	0.10	0.11
query12	0.15	0.11	0.12
query13	0.62	0.60	0.61
query14	2.72	2.86	2.76
query15	0.90	0.83	0.81
query16	0.37	0.39	0.38
query17	1.05	1.00	1.05
query18	0.22	0.21	0.21
query19	2.00	1.88	2.03
query20	0.01	0.01	0.01
query21	15.37	0.58	0.58
query22	2.30	2.76	1.81
query23	16.92	1.16	0.87
query24	2.83	1.80	0.67
query25	0.22	0.07	0.13
query26	0.50	0.14	0.14
query27	0.05	0.04	0.04
query28	10.39	1.10	1.09
query29	12.55	3.22	3.28
query30	0.25	0.06	0.06
query31	2.86	0.38	0.37
query32	3.29	0.46	0.47
query33	2.99	3.00	3.09
query34	17.16	4.38	4.45
query35	4.57	4.45	4.50
query36	0.63	0.48	0.48
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.02	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.25 s
Total hot run time: 32.21 s

@Yao-MR Yao-MR requested a review from starocean999 December 3, 2024 01:58
@morrySnow morrySnow requested a review from CalvinKirs December 4, 2024 03:35
@Yao-MR Yao-MR force-pushed the enhancement_nereids_show-catalog-support branch from 7a20bfa to 6c70c00 Compare December 4, 2024 09:05
@doris-robot
Copy link

TPC-H: Total hot run time: 41297 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 043ff6e8550fe8ba3f9c6db9530889a6c0810c04, data reload: false

------ Round 1 ----------------------------------
q1	17837	7451	7270	7270
q2	2057	1320	1279	1279
q3	10098	1159	1143	1143
q4	10308	785	730	730
q5	7591	2910	2695	2695
q6	235	146	145	145
q7	1009	641	609	609
q8	9268	1886	1956	1886
q9	6708	6555	6492	6492
q10	6990	2280	2292	2280
q11	468	259	271	259
q12	413	232	231	231
q13	17778	3041	3039	3039
q14	245	209	211	209
q15	578	535	509	509
q16	644	581	573	573
q17	1009	560	510	510
q18	7455	6621	6663	6621
q19	1326	1062	1048	1048
q20	485	181	183	181
q21	4083	3434	3280	3280
q22	389	308	324	308
Total cold run time: 106974 ms
Total hot run time: 41297 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7317	7238	7228	7228
q2	323	237	233	233
q3	2964	3024	2943	2943
q4	2103	1855	1918	1855
q5	5702	5555	5690	5555
q6	237	150	148	148
q7	2212	1766	1816	1766
q8	3438	3541	3534	3534
q9	8999	9115	9134	9115
q10	3604	3604	3558	3558
q11	629	522	503	503
q12	801	608	640	608
q13	12333	3312	3230	3230
q14	301	273	274	273
q15	580	522	559	522
q16	687	665	646	646
q17	1854	1594	1569	1569
q18	8331	7769	7757	7757
q19	4455	1638	1547	1547
q20	2137	1873	1873	1873
q21	5815	5466	5366	5366
q22	668	574	559	559
Total cold run time: 75490 ms
Total hot run time: 60388 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196394 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 043ff6e8550fe8ba3f9c6db9530889a6c0810c04, data reload: false

query1	1230	783	762	762
query2	6231	2079	2002	2002
query3	10755	4480	4407	4407
query4	66192	28865	23560	23560
query5	4951	486	481	481
query6	426	189	177	177
query7	5651	315	303	303
query8	333	242	240	240
query9	9394	2739	2712	2712
query10	461	242	247	242
query11	18171	15347	16012	15347
query12	141	107	109	107
query13	1585	416	413	413
query14	10539	7217	7681	7217
query15	227	197	188	188
query16	7134	485	504	485
query17	1149	602	579	579
query18	1835	306	295	295
query19	224	149	156	149
query20	114	109	108	108
query21	201	97	98	97
query22	4544	4411	4437	4411
query23	34981	34566	35020	34566
query24	5366	2415	2421	2415
query25	495	430	396	396
query26	643	154	148	148
query27	1859	280	296	280
query28	4104	2482	2451	2451
query29	658	417	400	400
query30	202	145	145	145
query31	974	779	829	779
query32	69	51	59	51
query33	448	315	268	268
query34	923	507	528	507
query35	859	744	748	744
query36	1069	934	936	934
query37	115	72	72	72
query38	4260	4246	4208	4208
query39	1443	1429	1415	1415
query40	200	95	96	95
query41	41	40	42	40
query42	107	96	95	95
query43	534	480	499	480
query44	1143	798	813	798
query45	183	171	175	171
query46	1134	725	717	717
query47	1944	1847	1859	1847
query48	412	307	316	307
query49	718	409	385	385
query50	799	384	392	384
query51	7191	7018	7051	7018
query52	98	87	87	87
query53	250	184	182	182
query54	507	408	402	402
query55	75	77	79	77
query56	247	237	241	237
query57	1242	1099	1124	1099
query58	213	201	207	201
query59	3073	3010	2789	2789
query60	269	247	241	241
query61	112	112	107	107
query62	797	674	691	674
query63	219	192	191	191
query64	1501	775	743	743
query65	3274	3214	3205	3205
query66	723	315	319	315
query67	15922	15513	15336	15336
query68	4160	576	555	555
query69	428	254	258	254
query70	1170	1157	1142	1142
query71	361	290	268	268
query72	6406	4307	4140	4140
query73	752	362	360	360
query74	10026	9016	9008	9008
query75	3397	2664	2678	2664
query76	1859	964	1014	964
query77	523	267	289	267
query78	10534	9464	9436	9436
query79	2191	613	616	613
query80	1394	427	430	427
query81	502	236	232	232
query82	1286	119	120	119
query83	259	154	149	149
query84	284	74	72	72
query85	1025	301	310	301
query86	432	299	294	294
query87	4725	4550	4465	4465
query88	3643	2252	2211	2211
query89	437	290	297	290
query90	1954	188	182	182
query91	169	104	104	104
query92	62	47	47	47
query93	2765	555	551	551
query94	862	299	281	281
query95	362	243	250	243
query96	638	272	278	272
query97	2858	2688	2678	2678
query98	220	197	199	197
query99	1639	1335	1324	1324
Total cold run time: 320207 ms
Total hot run time: 196394 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.56 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 043ff6e8550fe8ba3f9c6db9530889a6c0810c04, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.11
query5	0.44	0.40	0.43
query6	1.14	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.57	0.53	0.50
query10	0.55	0.57	0.57
query11	0.15	0.11	0.10
query12	0.15	0.11	0.11
query13	0.61	0.60	0.60
query14	2.75	2.76	2.75
query15	0.90	0.83	0.83
query16	0.40	0.39	0.38
query17	1.06	1.05	1.05
query18	0.22	0.21	0.21
query19	1.92	1.76	1.90
query20	0.02	0.01	0.01
query21	15.37	0.57	0.56
query22	2.48	2.55	1.40
query23	17.01	0.91	0.79
query24	2.68	1.33	1.44
query25	0.23	0.12	0.16
query26	0.53	0.14	0.14
query27	0.05	0.04	0.05
query28	10.36	1.10	1.08
query29	12.59	3.29	3.27
query30	0.24	0.06	0.06
query31	2.87	0.39	0.38
query32	3.23	0.47	0.46
query33	3.02	3.03	3.03
query34	16.84	4.48	4.44
query35	4.56	4.48	4.49
query36	0.65	0.48	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.18	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.03
query43	0.04	0.04	0.03
Total cold run time: 106.11 s
Total hot run time: 32.56 s

@doris-robot
Copy link

TPC-H: Total hot run time: 40013 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0ee0b4f4bb7646ea96f0eded61f433713079f7f0, data reload: false

------ Round 1 ----------------------------------
q1	17595	7396	7319	7319
q2	2042	197	175	175
q3	10779	1084	1149	1084
q4	10501	783	705	705
q5	7621	2725	2718	2718
q6	239	146	145	145
q7	994	621	609	609
q8	9247	1808	1879	1808
q9	6624	6552	6520	6520
q10	7051	2261	2303	2261
q11	469	260	265	260
q12	427	225	215	215
q13	17796	3014	3026	3014
q14	255	223	209	209
q15	590	541	538	538
q16	674	604	588	588
q17	1008	567	517	517
q18	7246	6783	6590	6590
q19	1348	989	1106	989
q20	463	184	182	182
q21	3963	3288	3248	3248
q22	381	327	319	319
Total cold run time: 107313 ms
Total hot run time: 40013 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7227	7228	7270	7228
q2	323	224	230	224
q3	2899	2766	2950	2766
q4	2038	1781	1783	1781
q5	5747	5666	5638	5638
q6	225	140	140	140
q7	2230	1801	1812	1801
q8	3347	3544	3469	3469
q9	9039	9042	8941	8941
q10	3572	3561	3545	3545
q11	613	500	503	500
q12	815	611	655	611
q13	11553	3311	3247	3247
q14	313	279	269	269
q15	571	534	528	528
q16	663	635	633	633
q17	1870	1627	1612	1612
q18	8217	7696	7624	7624
q19	1661	1547	1586	1547
q20	2102	1859	1864	1859
q21	5670	5486	5476	5476
q22	639	598	584	584
Total cold run time: 71334 ms
Total hot run time: 60023 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197430 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0ee0b4f4bb7646ea96f0eded61f433713079f7f0, data reload: false

query1	1467	961	953	953
query2	6233	2067	1967	1967
query3	10997	4449	4318	4318
query4	33248	23543	23494	23494
query5	3668	476	439	439
query6	281	185	176	176
query7	3989	310	300	300
query8	299	252	234	234
query9	9570	2736	2737	2736
query10	437	246	253	246
query11	17822	15223	15209	15209
query12	160	104	105	104
query13	1557	432	384	384
query14	9825	6599	6748	6599
query15	276	194	208	194
query16	7963	457	490	457
query17	1559	576	567	567
query18	2188	306	299	299
query19	354	153	155	153
query20	127	114	115	114
query21	205	105	106	105
query22	4864	4638	4585	4585
query23	35333	35994	35537	35537
query24	10349	2519	2496	2496
query25	513	397	389	389
query26	696	154	144	144
query27	2048	285	285	285
query28	6706	2483	2471	2471
query29	660	405	418	405
query30	242	153	159	153
query31	1038	867	833	833
query32	93	53	55	53
query33	795	288	285	285
query34	1288	520	532	520
query35	896	764	751	751
query36	1151	956	968	956
query37	121	76	75	75
query38	4556	4321	4538	4321
query39	1521	1456	1497	1456
query40	216	100	97	97
query41	43	40	43	40
query42	110	99	97	97
query43	542	498	490	490
query44	1284	817	814	814
query45	193	169	168	168
query46	1198	710	737	710
query47	2068	1965	1976	1965
query48	418	322	328	322
query49	933	408	389	389
query50	826	403	391	391
query51	7475	7181	7309	7181
query52	101	97	86	86
query53	262	194	193	193
query54	1157	443	399	399
query55	77	78	77	77
query56	272	234	218	218
query57	1221	1149	1126	1126
query58	225	206	222	206
query59	3212	2996	3008	2996
query60	264	234	240	234
query61	104	104	105	104
query62	829	661	684	661
query63	218	193	186	186
query64	3915	662	648	648
query65	3284	3188	3237	3188
query66	734	317	331	317
query67	15836	15756	15423	15423
query68	4535	580	566	566
query69	423	250	263	250
query70	1138	1137	1123	1123
query71	325	248	254	248
query72	6114	3988	3991	3988
query73	767	356	362	356
query74	10073	8950	9045	8950
query75	3403	2646	2679	2646
query76	2701	1138	1051	1051
query77	375	265	272	265
query78	10382	9473	9454	9454
query79	1840	596	598	596
query80	886	525	430	430
query81	542	225	228	225
query82	640	112	122	112
query83	239	138	143	138
query84	238	66	71	66
query85	1482	297	299	297
query86	462	294	284	284
query87	4642	4577	4549	4549
query88	3977	2208	2176	2176
query89	475	287	280	280
query90	1910	188	187	187
query91	136	102	102	102
query92	60	48	50	48
query93	2368	538	538	538
query94	735	295	296	295
query95	351	247	242	242
query96	620	276	282	276
query97	2817	2709	2748	2709
query98	231	189	193	189
query99	1585	1334	1305	1305
Total cold run time: 298584 ms
Total hot run time: 197430 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.22 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0ee0b4f4bb7646ea96f0eded61f433713079f7f0, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.05
query3	0.23	0.06	0.07
query4	1.62	0.11	0.10
query5	0.40	0.40	0.42
query6	1.17	0.68	0.66
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.50
query10	0.56	0.55	0.56
query11	0.14	0.11	0.10
query12	0.13	0.11	0.10
query13	0.61	0.60	0.59
query14	2.71	2.85	2.86
query15	0.91	0.82	0.82
query16	0.39	0.38	0.39
query17	1.06	1.05	1.07
query18	0.21	0.21	0.21
query19	1.91	1.82	2.04
query20	0.02	0.01	0.02
query21	15.38	0.60	0.58
query22	2.33	1.88	2.22
query23	17.06	0.89	0.84
query24	3.23	1.26	0.58
query25	0.14	0.13	0.04
query26	0.65	0.14	0.13
query27	0.04	0.04	0.04
query28	10.90	1.11	1.07
query29	12.61	3.24	3.19
query30	0.25	0.07	0.06
query31	2.86	0.38	0.38
query32	3.27	0.46	0.46
query33	3.10	2.98	2.98
query34	16.93	4.43	4.45
query35	4.50	4.45	4.48
query36	0.67	0.48	0.48
query37	0.09	0.05	0.05
query38	0.04	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.14	0.13
query41	0.07	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.25 s
Total hot run time: 32.22 s

@Yao-MR Yao-MR force-pushed the enhancement_nereids_show-catalog-support branch from 0ee0b4f to d360f66 Compare December 6, 2024 06:28
@Yao-MR
Copy link
Contributor Author

Yao-MR commented Dec 10, 2024

hi @CalvinKirs can you help have a review when you are available,

@doris-robot
Copy link

TPC-H: Total hot run time: 39723 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5956e09fcd78ae8de41c941fd80447cc26ef2f88, data reload: false

------ Round 1 ----------------------------------
q1	17634	7438	7265	7265
q2	2048	184	169	169
q3	10679	1120	1150	1120
q4	10507	676	707	676
q5	7606	2728	2596	2596
q6	233	146	148	146
q7	953	612	615	612
q8	9246	1852	1887	1852
q9	6663	6405	6469	6405
q10	6991	2323	2287	2287
q11	465	261	261	261
q12	412	223	221	221
q13	17789	2997	3035	2997
q14	242	213	201	201
q15	565	516	504	504
q16	675	597	585	585
q17	967	601	595	595
q18	7359	6567	6685	6567
q19	1365	959	974	959
q20	482	185	180	180
q21	4141	3214	3309	3214
q22	382	315	311	311
Total cold run time: 107404 ms
Total hot run time: 39723 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7253	7191	7195	7191
q2	322	228	231	228
q3	2867	2737	3066	2737
q4	2140	1865	1854	1854
q5	5508	5683	5607	5607
q6	223	136	137	136
q7	2173	1769	1788	1769
q8	3340	3587	3491	3491
q9	8862	8938	8969	8938
q10	3568	3506	3544	3506
q11	596	496	500	496
q12	799	609	619	609
q13	12053	3209	3139	3139
q14	298	285	267	267
q15	574	523	542	523
q16	673	610	633	610
q17	1794	1587	1555	1555
q18	7919	7375	7348	7348
q19	1663	1414	1374	1374
q20	2036	1797	1797	1797
q21	5390	5208	5323	5208
q22	627	560	594	560
Total cold run time: 70678 ms
Total hot run time: 58943 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189899 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5956e09fcd78ae8de41c941fd80447cc26ef2f88, data reload: false

query1	988	392	367	367
query2	6528	2178	2120	2120
query3	6702	220	208	208
query4	33837	23296	23332	23296
query5	4286	452	458	452
query6	275	191	184	184
query7	4620	300	304	300
query8	308	242	236	236
query9	9534	2678	2689	2678
query10	465	260	250	250
query11	17862	15227	15011	15011
query12	148	105	104	104
query13	1696	441	430	430
query14	9758	7414	7959	7414
query15	321	181	188	181
query16	7600	436	459	436
query17	1827	595	534	534
query18	1465	289	299	289
query19	360	152	149	149
query20	126	113	123	113
query21	204	106	100	100
query22	4550	4319	4233	4233
query23	34659	33535	33813	33535
query24	11433	2403	2577	2403
query25	668	369	376	369
query26	1839	147	152	147
query27	2541	269	288	269
query28	7536	2432	2417	2417
query29	946	400	396	396
query30	294	157	145	145
query31	1044	831	780	780
query32	99	60	62	60
query33	795	295	288	288
query34	988	521	542	521
query35	873	767	735	735
query36	1092	953	950	950
query37	228	75	71	71
query38	4272	4190	4119	4119
query39	1474	1384	1397	1384
query40	281	112	99	99
query41	49	43	43	43
query42	112	96	98	96
query43	544	501	486	486
query44	1244	815	809	809
query45	181	170	165	165
query46	1151	709	690	690
query47	1947	1871	1852	1852
query48	419	328	321	321
query49	1318	405	370	370
query50	812	385	391	385
query51	7128	7017	7131	7017
query52	102	89	90	89
query53	251	181	181	181
query54	1354	404	412	404
query55	77	82	84	82
query56	257	232	244	232
query57	1287	1125	1122	1122
query58	237	208	214	208
query59	3405	3200	3006	3006
query60	270	252	242	242
query61	107	107	104	104
query62	914	676	672	672
query63	206	185	192	185
query64	5334	664	647	647
query65	3321	3194	3220	3194
query66	1325	307	305	305
query67	16228	15728	15503	15503
query68	4520	578	568	568
query69	426	259	254	254
query70	1209	1063	1141	1063
query71	398	259	258	258
query72	6348	4024	4060	4024
query73	752	357	361	357
query74	10095	8991	8769	8769
query75	3388	2633	2613	2613
query76	2682	1208	1188	1188
query77	605	279	273	273
query78	10369	9441	9337	9337
query79	1478	594	591	591
query80	1187	429	413	413
query81	525	226	241	226
query82	918	112	120	112
query83	240	146	148	146
query84	234	127	68	68
query85	1352	303	300	300
query86	378	303	287	287
query87	4613	4606	4532	4532
query88	3352	2239	2180	2180
query89	413	294	302	294
query90	2006	186	183	183
query91	142	106	103	103
query92	64	51	52	51
query93	1104	543	542	542
query94	1126	294	278	278
query95	354	250	254	250
query96	604	275	280	275
query97	2850	2701	2699	2699
query98	220	191	199	191
query99	1567	1337	1291	1291
Total cold run time: 299621 ms
Total hot run time: 189899 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5956e09fcd78ae8de41c941fd80447cc26ef2f88, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.10	0.10
query5	0.42	0.42	0.41
query6	1.18	0.66	0.64
query7	0.02	0.01	0.01
query8	0.04	0.04	0.03
query9	0.58	0.52	0.51
query10	0.56	0.57	0.56
query11	0.14	0.10	0.11
query12	0.14	0.10	0.11
query13	0.61	0.60	0.60
query14	2.83	2.83	2.70
query15	0.90	0.83	0.82
query16	0.39	0.39	0.39
query17	1.05	1.02	1.00
query18	0.23	0.21	0.21
query19	1.97	1.73	1.98
query20	0.02	0.01	0.02
query21	15.36	0.60	0.58
query22	2.92	1.80	2.11
query23	17.07	1.01	0.79
query24	3.33	1.84	1.50
query25	0.32	0.06	0.26
query26	0.50	0.13	0.14
query27	0.04	0.04	0.04
query28	9.52	1.10	1.06
query29	12.55	3.16	3.16
query30	0.26	0.06	0.07
query31	2.87	0.38	0.38
query32	3.26	0.46	0.48
query33	2.98	2.99	3.02
query34	16.89	4.44	4.50
query35	4.50	4.47	4.44
query36	0.67	0.47	0.48
query37	0.10	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.03
query40	0.17	0.12	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.55 s
Total hot run time: 32.72 s

CalvinKirs
CalvinKirs previously approved these changes Dec 13, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 13, 2024
@Yao-MR
Copy link
Contributor Author

Yao-MR commented Dec 17, 2024

run buildall

1 similar comment
@Yao-MR
Copy link
Contributor Author

Yao-MR commented Dec 17, 2024

run buildall

@Yao-MR
Copy link
Contributor Author

Yao-MR commented Dec 17, 2024

run P0

@Yao-MR Yao-MR force-pushed the enhancement_nereids_show-catalog-support branch from 316befc to d360f66 Compare December 17, 2024 12:05
@starocean999
Copy link
Contributor

please resolve conflicts

@Yao-MR Yao-MR requested a review from CalvinKirs December 20, 2024 01:42
@Yao-MR
Copy link
Contributor Author

Yao-MR commented Dec 20, 2024

please resolve conflicts

done @starocean999

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement] (nereids)implement showCatalogCommand in nereids
5 participants