Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve](nereids) use hash set replace three set in DiscreteValue to improve in predicate performance (#45181) #46222

Merged
merged 1 commit into from
Dec 31, 2024

Conversation

qzsee
Copy link
Contributor

@qzsee qzsee commented Dec 31, 2024

What problem does this PR solve?

cp #45181

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dqz123
Copy link

dqz123 commented Dec 31, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40962 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b20e22e0ec7c85c1610a0eb697bc44bc3e5ddda5, data reload: false

------ Round 1 ----------------------------------
q1	17579	7557	7392	7392
q2	2847	179	167	167
q3	11182	1142	1147	1142
q4	10445	764	801	764
q5	7803	2859	2777	2777
q6	244	158	153	153
q7	987	625	614	614
q8	9433	1923	2027	1923
q9	6720	6511	6432	6432
q10	7004	2277	2300	2277
q11	450	263	268	263
q12	397	213	211	211
q13	17779	2982	3018	2982
q14	232	223	220	220
q15	563	531	518	518
q16	688	612	621	612
q17	965	567	532	532
q18	7461	6738	6602	6602
q19	1375	1020	978	978
q20	483	211	200	200
q21	3945	3300	3199	3199
q22	1115	1004	1056	1004
Total cold run time: 109697 ms
Total hot run time: 40962 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7370	7262	7244	7244
q2	341	230	231	230
q3	3107	3029	2942	2942
q4	1997	1772	1767	1767
q5	6000	5689	5627	5627
q6	223	135	136	135
q7	2234	1795	1803	1795
q8	3306	3527	3446	3446
q9	8892	8810	8767	8767
q10	3594	3655	3562	3562
q11	596	504	500	500
q12	799	642	627	627
q13	7418	3187	3044	3044
q14	290	253	268	253
q15	556	510	515	510
q16	701	668	665	665
q17	1783	1546	1559	1546
q18	7755	7277	7421	7277
q19	1633	1578	1624	1578
q20	2035	1783	1811	1783
q21	5536	5246	5280	5246
q22	1109	1008	1010	1008
Total cold run time: 67275 ms
Total hot run time: 59552 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191127 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b20e22e0ec7c85c1610a0eb697bc44bc3e5ddda5, data reload: false

query1	962	366	373	366
query2	6531	2227	2221	2221
query3	6697	219	219	219
query4	33886	23610	23470	23470
query5	4297	455	450	450
query6	270	178	203	178
query7	4626	313	311	311
query8	298	236	232	232
query9	9591	2701	2723	2701
query10	473	276	265	265
query11	17980	15190	15262	15190
query12	153	102	100	100
query13	1628	437	422	422
query14	9983	6541	7062	6541
query15	255	169	174	169
query16	7894	470	468	468
query17	1642	581	561	561
query18	1931	312	315	312
query19	340	152	147	147
query20	113	107	112	107
query21	67	47	47	47
query22	4888	4214	4119	4119
query23	36546	34314	33766	33766
query24	11612	2887	2835	2835
query25	565	411	409	409
query26	1202	176	171	171
query27	2658	300	302	300
query28	7905	2460	2443	2443
query29	676	445	428	428
query30	333	161	168	161
query31	1035	822	786	786
query32	95	59	59	59
query33	799	301	293	293
query34	932	494	499	494
query35	867	730	733	730
query36	1095	944	953	944
query37	127	74	82	74
query38	4078	3951	3886	3886
query39	1471	1472	1455	1455
query40	221	82	83	82
query41	52	46	49	46
query42	114	95	94	94
query43	547	506	505	505
query44	1293	815	812	812
query45	182	166	170	166
query46	1133	693	725	693
query47	1940	1845	1853	1845
query48	482	383	374	374
query49	1258	409	384	384
query50	807	407	404	404
query51	7139	7078	6928	6928
query52	98	90	87	87
query53	253	183	181	181
query54	1096	451	438	438
query55	77	77	77	77
query56	272	249	252	249
query57	1232	1111	1089	1089
query58	229	202	232	202
query59	3458	3123	3023	3023
query60	291	238	267	238
query61	116	110	112	110
query62	852	688	677	677
query63	212	188	179	179
query64	5169	652	618	618
query65	3345	3224	3186	3186
query66	1313	323	309	309
query67	16023	15835	15977	15835
query68	4535	599	553	553
query69	421	264	253	253
query70	1122	1151	1124	1124
query71	341	254	247	247
query72	6163	4020	4046	4020
query73	748	344	351	344
query74	10373	8983	9090	8983
query75	3346	2624	2621	2621
query76	2703	1119	1256	1119
query77	384	292	281	281
query78	10414	9534	9486	9486
query79	1586	607	589	589
query80	1133	447	424	424
query81	533	242	242	242
query82	915	119	114	114
query83	217	145	151	145
query84	246	84	83	83
query85	1341	312	300	300
query86	376	303	302	302
query87	4398	4175	4284	4175
query88	3593	2402	2379	2379
query89	397	287	286	286
query90	2107	184	182	182
query91	185	151	147	147
query92	59	49	51	49
query93	1082	539	541	539
query94	1074	308	289	289
query95	358	257	259	257
query96	606	276	278	276
query97	3300	3158	3197	3158
query98	207	207	203	203
query99	1571	1341	1296	1296
Total cold run time: 301539 ms
Total hot run time: 191127 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.82 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b20e22e0ec7c85c1610a0eb697bc44bc3e5ddda5, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.25	0.07	0.07
query4	1.61	0.10	0.10
query5	0.53	0.51	0.50
query6	1.12	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.05	0.03
query9	0.55	0.50	0.50
query10	0.55	0.55	0.55
query11	0.13	0.10	0.11
query12	0.14	0.11	0.12
query13	0.60	0.60	0.60
query14	2.96	2.95	2.96
query15	0.89	0.84	0.82
query16	0.39	0.38	0.38
query17	1.04	1.04	1.05
query18	0.23	0.21	0.23
query19	1.98	1.79	2.00
query20	0.02	0.01	0.02
query21	15.37	0.58	0.58
query22	2.83	3.24	2.00
query23	17.51	0.96	0.75
query24	3.32	0.64	0.90
query25	0.15	0.22	0.18
query26	0.39	0.14	0.14
query27	0.05	0.04	0.05
query28	10.64	1.12	1.07
query29	12.57	3.24	3.25
query30	0.25	0.06	0.06
query31	2.85	0.40	0.39
query32	3.23	0.46	0.47
query33	2.98	3.00	3.06
query34	16.90	4.50	4.44
query35	4.53	4.55	4.47
query36	0.66	0.48	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.87 s
Total hot run time: 32.82 s

@yiguolei yiguolei closed this Dec 31, 2024
@yiguolei yiguolei reopened this Dec 31, 2024
@yiguolei yiguolei merged commit f983411 into apache:branch-3.0 Dec 31, 2024
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants