Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0: [fix](memory) Fix adjust cache capacity #45603 #45665

Merged
merged 1 commit into from
Dec 19, 2024

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #45603

### What problem does this PR solve?

If the cache capacity adjustment is not completed within 500ms
(conf::memory_gc_sleep_time_ms), the next adjustment will be skipped.

In some scenarios, after Memory GC adjusts the cache capacity to 0, the
next adjustment to restore the cache capacity is skipped, the cache
capacity will remain at 0 for a long time.
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Dec 19, 2024
@hello-stephen
Copy link
Contributor

run buildall

Copy link
Contributor Author

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40491 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 62283869d9e3767d02ddbf4504c6e7bf95275132, data reload: false

------ Round 1 ----------------------------------
q1	17586	7440	7283	7283
q2	2046	170	165	165
q3	10710	1077	1127	1077
q4	10552	768	685	685
q5	7770	2833	2803	2803
q6	233	152	148	148
q7	989	610	594	594
q8	9578	1944	1970	1944
q9	8092	6371	6369	6369
q10	6959	2293	2314	2293
q11	466	267	270	267
q12	423	214	206	206
q13	17771	2997	2976	2976
q14	244	217	204	204
q15	556	520	520	520
q16	692	615	603	603
q17	960	604	583	583
q18	7185	6584	6675	6584
q19	2601	1090	957	957
q20	487	199	203	199
q21	3991	3094	3050	3050
q22	1063	990	981	981
Total cold run time: 110954 ms
Total hot run time: 40491 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7357	7208	7210	7208
q2	321	232	231	231
q3	2979	2916	2872	2872
q4	2055	1821	1780	1780
q5	5679	5711	5677	5677
q6	230	146	155	146
q7	2157	1783	1790	1783
q8	3310	3537	3404	3404
q9	8801	8881	8763	8763
q10	3587	3499	3465	3465
q11	597	505	505	505
q12	794	620	609	609
q13	16569	3157	3165	3157
q14	301	273	286	273
q15	569	517	527	517
q16	698	665	706	665
q17	1841	1608	1597	1597
q18	8235	7786	7396	7396
q19	3842	1600	1591	1591
q20	2082	1870	1821	1821
q21	5891	5219	5257	5219
q22	1114	998	1006	998
Total cold run time: 79009 ms
Total hot run time: 59677 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195852 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 62283869d9e3767d02ddbf4504c6e7bf95275132, data reload: false

query1	1241	921	909	909
query2	6238	2111	2064	2064
query3	10844	4108	4035	4035
query4	66695	29690	23435	23435
query5	5347	427	422	422
query6	429	174	167	167
query7	5672	313	303	303
query8	305	231	228	228
query9	9133	2685	2683	2683
query10	535	265	263	263
query11	17913	15249	15754	15249
query12	154	101	98	98
query13	1524	440	424	424
query14	11171	7327	6982	6982
query15	211	176	174	174
query16	7621	527	535	527
query17	1069	578	591	578
query18	1927	316	311	311
query19	227	158	168	158
query20	119	110	115	110
query21	67	48	43	43
query22	4817	4512	4512	4512
query23	34406	34364	34327	34327
query24	6169	2847	2902	2847
query25	546	430	424	424
query26	685	171	176	171
query27	2072	310	307	307
query28	4209	2594	2479	2479
query29	758	470	477	470
query30	257	172	165	165
query31	1019	845	822	822
query32	70	59	53	53
query33	451	278	276	276
query34	924	492	490	490
query35	837	723	743	723
query36	1083	963	974	963
query37	115	70	72	70
query38	4129	4068	3996	3996
query39	1515	1468	1467	1467
query40	148	91	83	83
query41	54	47	44	44
query42	104	97	95	95
query43	552	497	499	497
query44	1197	803	809	803
query45	187	170	161	161
query46	1153	733	721	721
query47	1995	1907	1931	1907
query48	473	370	373	370
query49	735	392	401	392
query50	826	405	404	404
query51	7372	7182	7014	7014
query52	101	83	91	83
query53	252	185	187	185
query54	562	443	444	443
query55	74	70	72	70
query56	251	227	222	222
query57	1202	1096	1115	1096
query58	202	194	197	194
query59	3175	2999	2914	2914
query60	272	259	240	240
query61	109	103	107	103
query62	774	651	655	651
query63	202	182	188	182
query64	1780	652	611	611
query65	3226	3194	3165	3165
query66	678	299	294	294
query67	15877	15326	15289	15289
query68	5083	548	560	548
query69	417	256	246	246
query70	1169	1119	1104	1104
query71	398	252	256	252
query72	6603	3905	3928	3905
query73	770	337	340	337
query74	9257	8812	8921	8812
query75	3315	2598	2657	2598
query76	2171	1028	1086	1028
query77	485	259	254	254
query78	10870	9741	9439	9439
query79	8261	595	580	580
query80	2274	419	414	414
query81	552	243	237	237
query82	1378	123	112	112
query83	248	140	143	140
query84	291	81	83	81
query85	1800	322	287	287
query86	483	305	302	302
query87	4438	4230	4204	4204
query88	5764	2395	2425	2395
query89	557	285	283	283
query90	2126	182	184	182
query91	180	143	145	143
query92	66	49	46	46
query93	6614	541	529	529
query94	964	302	283	283
query95	346	242	243	242
query96	627	280	276	276
query97	3316	3147	3198	3147
query98	214	194	203	194
query99	1617	1306	1294	1294
Total cold run time: 338691 ms
Total hot run time: 195852 ms

@dataroaring dataroaring merged commit d84ee07 into branch-3.0 Dec 19, 2024
21 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-45603-branch-3.0 branch December 19, 2024 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants