Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improvement](build index)Optimize failed task check on same tablet (#42295) #43581

Merged
merged 1 commit into from
Nov 12, 2024

Conversation

qidaye
Copy link
Contributor

@qidaye qidaye commented Nov 11, 2024

bp #42295

…pache#42295)

## Proposed changes

1. Remove logic for replica majority failure checking
2. Introduce `OBTAIN_LOCK_FAILED` status code both in `status.h` and
`Status.thrift`
3. Add `MIN_FAILED_NUM = 3` for non try_lock_failed failure task. The
task will try 3 times before canceling the job.
4. Add `MAX_FAILED_NUM = 10` for try_lock_failed failure task. The task
will try 10 times before canceling the job.

---------

Co-authored-by: qidaye <[email protected]>
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@qidaye
Copy link
Contributor Author

qidaye commented Nov 11, 2024

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40621 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c48f0d7b897922836c127a9e4929da3b35a2d457, data reload: false

------ Round 1 ----------------------------------
q1	17647	7490	7307	7307
q2	2095	166	162	162
q3	10691	1079	1177	1079
q4	10561	754	795	754
q5	7756	2876	2763	2763
q6	239	146	141	141
q7	957	603	616	603
q8	9606	1965	2006	1965
q9	7060	6398	6517	6398
q10	7009	2317	2312	2312
q11	447	260	252	252
q12	407	207	215	207
q13	17790	2996	2975	2975
q14	233	204	223	204
q15	579	523	521	521
q16	662	605	590	590
q17	982	532	601	532
q18	7261	6735	6633	6633
q19	1786	1063	1012	1012
q20	475	201	189	189
q21	3944	3145	3027	3027
q22	1108	1002	995	995
Total cold run time: 109295 ms
Total hot run time: 40621 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7384	7284	7198	7198
q2	397	311	308	308
q3	2978	2885	2927	2885
q4	2033	1826	1782	1782
q5	5685	5754	5714	5714
q6	236	144	140	140
q7	2188	1779	1764	1764
q8	3404	3551	3497	3497
q9	8917	8893	8850	8850
q10	3563	3561	3511	3511
q11	598	505	485	485
q12	814	576	629	576
q13	16486	3144	3098	3098
q14	314	264	270	264
q15	597	566	556	556
q16	709	667	660	660
q17	1849	1645	1621	1621
q18	8244	7617	7602	7602
q19	3446	1506	1687	1506
q20	2079	1860	1841	1841
q21	5406	5209	5188	5188
q22	1156	1012	1030	1012
Total cold run time: 78483 ms
Total hot run time: 60058 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192689 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c48f0d7b897922836c127a9e4929da3b35a2d457, data reload: false

query1	2486	2230	2013	2013
query2	6240	2023	2030	2023
query3	14822	10456	234	234
query4	33084	23342	23381	23342
query5	4773	480	464	464
query6	294	187	190	187
query7	3975	308	305	305
query8	299	238	234	234
query9	9478	2672	2661	2661
query10	495	267	260	260
query11	17967	15244	15149	15149
query12	155	101	101	101
query13	1541	444	411	411
query14	8936	6792	7363	6792
query15	236	181	172	172
query16	7684	503	495	495
query17	1516	590	587	587
query18	2329	614	601	601
query19	236	205	199	199
query20	121	121	115	115
query21	205	108	107	107
query22	4636	4574	4118	4118
query23	35007	35196	34074	34074
query24	11596	3480	3425	3425
query25	521	410	421	410
query26	731	182	181	181
query27	1988	291	295	291
query28	6535	2511	2511	2511
query29	684	435	441	435
query30	390	307	314	307
query31	1020	768	827	768
query32	97	54	55	54
query33	649	300	282	282
query34	907	478	499	478
query35	829	714	721	714
query36	1106	953	946	946
query37	118	70	70	70
query38	3887	3800	3885	3800
query39	1471	1441	1432	1432
query40	201	103	100	100
query41	52	56	50	50
query42	105	97	98	97
query43	513	492	491	491
query44	1151	781	780	780
query45	182	166	168	166
query46	1158	749	740	740
query47	1892	1827	1829	1827
query48	459	373	366	366
query49	898	395	384	384
query50	806	406	421	406
query51	7201	6965	7013	6965
query52	105	92	94	92
query53	255	188	188	188
query54	1049	467	453	453
query55	76	73	76	73
query56	280	266	256	256
query57	1188	1083	1095	1083
query58	233	217	208	208
query59	3231	2944	2836	2836
query60	298	260	279	260
query61	136	129	138	129
query62	809	672	660	660
query63	214	191	190	190
query64	3877	642	614	614
query65	3273	3143	3173	3143
query66	716	313	306	306
query67	15788	15266	15082	15082
query68	4529	580	580	580
query69	404	250	251	250
query70	1147	1103	1103	1103
query71	403	264	272	264
query72	6497	3901	3926	3901
query73	766	342	335	335
query74	9934	8937	8899	8899
query75	3363	2635	2603	2603
query76	2779	972	1062	972
query77	390	268	271	268
query78	10584	9907	9451	9451
query79	6137	584	584	584
query80	2404	434	427	427
query81	596	247	242	242
query82	964	117	113	113
query83	281	158	155	155
query84	289	79	79	79
query85	2275	298	284	284
query86	491	293	283	283
query87	4526	4224	4222	4222
query88	5281	2401	2420	2401
query89	577	284	281	281
query90	2269	185	181	181
query91	179	144	142	142
query92	63	49	46	46
query93	6522	537	534	534
query94	1179	289	283	283
query95	355	245	248	245
query96	632	285	287	285
query97	3371	3117	3131	3117
query98	219	196	191	191
query99	1664	1282	1275	1275
Total cold run time: 316260 ms
Total hot run time: 192689 ms

@airborne12 airborne12 merged commit 13b4c20 into apache:branch-3.0 Nov 12, 2024
21 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants