Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature-WIP](dictionary) Support Dictonary Table (FE Part I) #45926

Draft
wants to merge 23 commits into
base: master
Choose a base branch
from

Conversation

zclllyybb
Copy link
Contributor

@zclllyybb zclllyybb commented Dec 25, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

In this PR we support all required meta data structure for Directionary table in FE. and some of base operations' grammar.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes. but not yet

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32648 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2d10132b06bbde190fb1da9f06af3569ba73eaef, data reload: false

------ Round 1 ----------------------------------
q1	17910	6210	6143	6143
q2	2054	314	169	169
q3	10677	1251	728	728
q4	10234	873	437	437
q5	7997	2113	1965	1965
q6	207	185	152	152
q7	900	762	618	618
q8	9239	1363	1132	1132
q9	5240	4951	4913	4913
q10	6758	2296	1861	1861
q11	484	276	254	254
q12	351	362	229	229
q13	17795	3619	2991	2991
q14	225	234	205	205
q15	569	515	491	491
q16	637	609	591	591
q17	580	843	326	326
q18	7110	6440	6399	6399
q19	2563	968	557	557
q20	310	316	184	184
q21	2805	2193	1992	1992
q22	367	336	311	311
Total cold run time: 105012 ms
Total hot run time: 32648 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6371	6223	6210	6210
q2	229	333	226	226
q3	2239	2646	2340	2340
q4	1395	1827	1334	1334
q5	4317	4705	4879	4705
q6	186	182	145	145
q7	2060	1963	1796	1796
q8	2624	2817	2678	2678
q9	7291	7281	7285	7281
q10	3055	3337	2743	2743
q11	567	510	492	492
q12	615	749	602	602
q13	3482	3852	3184	3184
q14	313	316	295	295
q15	565	509	505	505
q16	646	687	665	665
q17	1215	1730	1257	1257
q18	7638	7400	7362	7362
q19	850	1052	1137	1052
q20	2036	2027	1885	1885
q21	5706	5206	4944	4944
q22	616	629	602	602
Total cold run time: 54016 ms
Total hot run time: 52303 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196728 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2d10132b06bbde190fb1da9f06af3569ba73eaef, data reload: false

query1	1273	962	969	962
query2	6490	2387	2299	2299
query3	10929	4786	4624	4624
query4	33256	24101	23355	23355
query5	4058	606	459	459
query6	306	209	184	184
query7	3979	488	317	317
query8	326	250	238	238
query9	9347	2765	2733	2733
query10	467	303	251	251
query11	17989	15342	15213	15213
query12	165	105	110	105
query13	1591	557	415	415
query14	10312	7456	6972	6972
query15	257	220	186	186
query16	7759	663	469	469
query17	1584	775	601	601
query18	1970	402	359	359
query19	224	182	167	167
query20	131	112	122	112
query21	210	133	106	106
query22	4639	4569	4555	4555
query23	34661	33771	33695	33695
query24	6472	2255	2422	2255
query25	484	435	385	385
query26	840	275	158	158
query27	2071	467	331	331
query28	5829	2537	2495	2495
query29	650	554	426	426
query30	220	213	150	150
query31	1000	940	854	854
query32	93	56	87	56
query33	481	360	311	311
query34	766	855	538	538
query35	833	834	776	776
query36	1012	1053	964	964
query37	124	104	78	78
query38	4291	4322	4063	4063
query39	1549	1476	1466	1466
query40	214	114	102	102
query41	46	43	46	43
query42	125	103	102	102
query43	511	509	474	474
query44	1311	833	833	833
query45	187	180	174	174
query46	890	1058	670	670
query47	2058	2008	1929	1929
query48	391	418	323	323
query49	706	473	379	379
query50	642	698	405	405
query51	7298	7343	7299	7299
query52	112	102	93	93
query53	241	254	187	187
query54	478	494	434	434
query55	87	88	88	88
query56	277	258	242	242
query57	1222	1229	1174	1174
query58	234	222	230	222
query59	3106	3242	3194	3194
query60	272	253	258	253
query61	107	104	120	104
query62	865	850	759	759
query63	246	202	206	202
query64	3705	1021	697	697
query65	3344	3308	3255	3255
query66	798	408	302	302
query67	16504	15867	15519	15519
query68	10275	760	539	539
query69	488	317	250	250
query70	1207	1081	1088	1081
query71	437	297	266	266
query72	6218	3788	3904	3788
query73	679	735	365	365
query74	10383	9374	9101	9101
query75	4490	3140	2647	2647
query76	5525	1194	761	761
query77	1050	383	275	275
query78	10292	10174	9443	9443
query79	3535	894	583	583
query80	728	519	424	424
query81	484	270	227	227
query82	641	148	126	126
query83	191	157	143	143
query84	283	86	68	68
query85	778	349	297	297
query86	344	314	292	292
query87	4692	4398	4384	4384
query88	3296	2283	2204	2204
query89	468	330	293	293
query90	1999	192	187	187
query91	134	132	101	101
query92	64	67	49	49
query93	2135	882	541	541
query94	670	377	279	279
query95	342	265	252	252
query96	492	600	281	281
query97	2780	2813	2693	2693
query98	222	197	188	188
query99	1672	1575	1459	1459
Total cold run time: 301609 ms
Total hot run time: 196728 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2d10132b06bbde190fb1da9f06af3569ba73eaef, data reload: false

query1	0.03	0.03	0.03
query2	0.09	0.05	0.05
query3	0.23	0.06	0.05
query4	1.64	0.08	0.09
query5	0.42	0.41	0.42
query6	1.13	0.67	0.67
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.49	0.50
query10	0.55	0.58	0.57
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.62	0.62	0.59
query14	2.73	2.72	2.75
query15	0.92	0.85	0.85
query16	0.38	0.37	0.37
query17	1.08	1.07	1.08
query18	0.19	0.18	0.19
query19	1.86	1.73	2.07
query20	0.02	0.02	0.02
query21	15.37	0.95	0.66
query22	0.77	0.79	0.73
query23	14.94	1.44	0.69
query24	2.15	0.38	0.23
query25	0.14	0.08	0.08
query26	0.28	0.19	0.18
query27	0.08	0.08	0.08
query28	13.46	1.79	1.14
query29	12.69	4.02	3.39
query30	0.24	0.08	0.07
query31	2.83	0.61	0.41
query32	3.22	0.57	0.47
query33	3.14	3.14	3.14
query34	16.64	5.16	4.49
query35	4.54	4.57	4.54
query36	0.62	0.50	0.48
query37	0.19	0.16	0.17
query38	0.16	0.15	0.16
query39	0.06	0.05	0.05
query40	0.18	0.14	0.13
query41	0.10	0.06	0.06
query42	0.06	0.06	0.06
query43	0.05	0.06	0.05
Total cold run time: 104.74 s
Total hot run time: 31.62 s

HappenLee
HappenLee previously approved these changes Dec 27, 2024
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 27, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@zclllyybb zclllyybb marked this pull request as draft December 30, 2024 03:47
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 30, 2024
@Mryange
Copy link
Contributor

Mryange commented Jan 2, 2025

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.93% (10153/26079)
Line Coverage: 29.94% (85880/286823)
Region Coverage: 29.05% (43825/150852)
Branch Coverage: 25.60% (22377/87426)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0dc1cf8c55a8a430ad785e934effe02bdb34fe67_0dc1cf8c55a8a430ad785e934effe02bdb34fe67/report/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants