Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](arrow-flight-sql) Fix FE fetches arrow schema to BE will waiting for result sink open #43423

Closed
wants to merge 1 commit into from

Conversation

xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Nov 7, 2024

What problem does this PR solve?

If FE fetches arrow schema to BE before ResultSinkLocalState::open, will wait for register_arrow_schema to complete, otherwise it will report an error NOT_FOUND.

Fix:

get flight info statement failed, fetch arrow flight schema failed, finstId: 3573efbeb10c44a7-956531d8e15d1630, errmsg: Status [errorCode=NOT_FOUND, errorMsg=(172.16.212.191)[NOT_FOUND]FE not found arrow flight schema, maybe query has been canceled],

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No colde files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.
  • Release note

    None

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Nov 7, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/runtime/buffer_control_block.cpp Show resolved Hide resolved
be/src/runtime/buffer_control_block.h Show resolved Hide resolved
@doris-robot
Copy link

TPC-H: Total hot run time: 41354 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4d8825c0fc800c4a8873918cf40c023c8c2378d6, data reload: false

------ Round 1 ----------------------------------
q1	17619	7452	7272	7272
q2	2061	176	168	168
q3	10573	1092	1127	1092
q4	10575	872	902	872
q5	7747	3043	3005	3005
q6	235	143	144	143
q7	1023	605	604	604
q8	9364	1985	2016	1985
q9	6532	6432	6447	6432
q10	7008	2386	2426	2386
q11	451	260	244	244
q12	399	217	209	209
q13	17779	3002	2975	2975
q14	239	222	206	206
q15	557	511	532	511
q16	634	579	586	579
q17	962	638	515	515
q18	7321	6767	6749	6749
q19	1322	1009	1030	1009
q20	468	185	186	185
q21	3945	3204	3223	3204
q22	1097	1011	1009	1009
Total cold run time: 107911 ms
Total hot run time: 41354 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7379	7221	7241	7221
q2	346	259	245	245
q3	2918	2944	2984	2944
q4	2115	1907	1761	1761
q5	5730	5757	5756	5756
q6	227	142	140	140
q7	2293	1856	1785	1785
q8	3415	3514	3522	3514
q9	8902	8887	8896	8887
q10	3586	3552	3587	3552
q11	601	518	505	505
q12	818	587	595	587
q13	9339	3140	3214	3140
q14	304	274	284	274
q15	594	553	549	549
q16	706	636	629	629
q17	1847	1621	1639	1621
q18	8273	7708	7610	7610
q19	1732	1616	1424	1424
q20	2096	1890	1863	1863
q21	5563	5510	5532	5510
q22	1158	1060	1059	1059
Total cold run time: 69942 ms
Total hot run time: 60576 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194138 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4d8825c0fc800c4a8873918cf40c023c8c2378d6, data reload: false

query1	2260	2229	2089	2089
query2	6255	2197	2096	2096
query3	14993	10834	233	233
query4	33199	23530	23572	23530
query5	3477	465	439	439
query6	268	178	178	178
query7	3983	294	298	294
query8	287	230	224	224
query9	9625	2667	2647	2647
query10	464	253	246	246
query11	17860	15288	15262	15262
query12	151	106	96	96
query13	1584	415	413	413
query14	9808	6659	7751	6659
query15	261	177	185	177
query16	8103	510	418	418
query17	1834	574	596	574
query18	2494	644	652	644
query19	414	184	197	184
query20	127	115	114	114
query21	202	109	108	108
query22	4769	4241	4523	4241
query23	35228	34083	34024	34024
query24	11529	3322	3335	3322
query25	605	398	404	398
query26	1047	176	178	176
query27	2192	280	280	280
query28	7483	2424	2389	2389
query29	795	431	418	418
query30	396	317	332	317
query31	1059	802	804	802
query32	93	56	57	56
query33	761	274	273	273
query34	924	536	510	510
query35	889	745	726	726
query36	1119	950	969	950
query37	120	74	75	74
query38	4388	4235	4265	4235
query39	1507	1436	1411	1411
query40	197	100	100	100
query41	49	49	49	49
query42	108	100	99	99
query43	566	496	511	496
query44	1342	810	815	810
query45	178	170	166	166
query46	1151	723	715	715
query47	1935	1841	1843	1841
query48	421	319	321	319
query49	876	390	391	390
query50	802	395	403	395
query51	7249	7180	7052	7052
query52	102	87	87	87
query53	251	178	181	178
query54	1151	428	397	397
query55	81	75	73	73
query56	252	235	238	235
query57	1274	1186	1133	1133
query58	223	202	204	202
query59	3268	3204	3117	3117
query60	266	250	251	250
query61	103	116	113	113
query62	856	659	670	659
query63	217	187	197	187
query64	3984	643	621	621
query65	3284	3179	3221	3179
query66	889	301	307	301
query67	15994	15592	15566	15566
query68	4497	583	590	583
query69	430	257	255	255
query70	1199	1106	1151	1106
query71	312	253	241	241
query72	6302	4028	4045	4028
query73	752	370	366	366
query74	10490	8944	9099	8944
query75	3428	2660	2686	2660
query76	2751	1128	1100	1100
query77	374	280	268	268
query78	10420	9510	9402	9402
query79	1136	596	592	592
query80	870	413	436	413
query81	581	242	242	242
query82	625	121	118	118
query83	245	161	159	159
query84	213	71	82	71
query85	1282	299	290	290
query86	378	297	286	286
query87	4849	4768	4678	4678
query88	3294	2232	2190	2190
query89	397	291	288	288
query90	1925	187	188	187
query91	143	105	104	104
query92	58	46	50	46
query93	1363	534	563	534
query94	833	292	292	292
query95	349	244	249	244
query96	635	290	289	289
query97	2869	2693	2703	2693
query98	204	197	207	197
query99	1573	1315	1322	1315
Total cold run time: 304399 ms
Total hot run time: 194138 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.87 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4d8825c0fc800c4a8873918cf40c023c8c2378d6, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.03	0.02
query3	0.23	0.06	0.06
query4	1.63	0.10	0.10
query5	0.42	0.41	0.41
query6	1.13	0.64	0.64
query7	0.04	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.49	0.49
query10	0.55	0.56	0.56
query11	0.14	0.10	0.10
query12	0.13	0.12	0.11
query13	0.60	0.60	0.60
query14	2.68	2.75	2.83
query15	0.90	0.83	0.82
query16	0.40	0.38	0.39
query17	1.06	1.03	1.04
query18	0.20	0.19	0.20
query19	1.90	1.88	1.93
query20	0.01	0.01	0.01
query21	15.35	0.59	0.57
query22	3.09	2.25	2.50
query23	16.96	1.02	0.82
query24	2.89	1.71	0.86
query25	0.25	0.16	0.12
query26	0.54	0.14	0.13
query27	0.04	0.04	0.05
query28	10.35	1.11	1.07
query29	12.55	3.19	3.23
query30	0.25	0.06	0.06
query31	2.88	0.37	0.39
query32	3.29	0.46	0.45
query33	2.94	3.01	3.02
query34	17.06	4.45	4.46
query35	4.53	4.52	4.48
query36	0.67	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.02	0.02
Total cold run time: 106.85 s
Total hot run time: 32.87 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.88% (9850/26006)
Line Coverage: 29.05% (81972/282162)
Region Coverage: 28.26% (42207/149339)
Branch Coverage: 24.84% (21409/86204)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4d8825c0fc800c4a8873918cf40c023c8c2378d6_4d8825c0fc800c4a8873918cf40c023c8c2378d6/report/index.html

@xinyiZzz xinyiZzz closed this Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants