Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat](mtmv)use real snapshot instead of optional.empty() #45273

Merged
merged 1 commit into from
Dec 11, 2024

Conversation

zddr
Copy link
Contributor

@zddr zddr commented Dec 11, 2024

What problem does this PR solve?

In the previous PR, a snapshot of the table was obtained and stored in the statementContext at the beginning of the query.
The modification of this PR is to ensure that the same metadata is used during the query process. When calling the relevant interface, snapshot needs to be obtained from statementContext as a parameter and passed to the relevant method

Issue Number: close #xxx

Related PR: #44911 #44673

Problem Summary:
use real snapshot instead of optional.empty()

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zddr
Copy link
Contributor Author

zddr commented Dec 11, 2024

run buildall

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 11, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39780 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5477fd10f2b6525eb9fa38c30480879f50120af3, data reload: false

------ Round 1 ----------------------------------
q1	17606	7461	7232	7232
q2	2044	181	179	179
q3	10749	1076	1163	1076
q4	10560	696	761	696
q5	7614	2769	2689	2689
q6	238	147	143	143
q7	993	619	590	590
q8	9244	1865	1922	1865
q9	6637	6465	6415	6415
q10	7001	2257	2330	2257
q11	477	265	270	265
q12	431	220	217	217
q13	17760	2998	3044	2998
q14	251	204	211	204
q15	583	527	514	514
q16	669	592	582	582
q17	994	527	491	491
q18	7249	6757	6705	6705
q19	1334	1026	999	999
q20	470	192	185	185
q21	3995	3256	3152	3152
q22	377	329	326	326
Total cold run time: 107276 ms
Total hot run time: 39780 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7231	7285	7237	7237
q2	324	228	229	228
q3	2893	2803	2950	2803
q4	2055	1816	1813	1813
q5	5712	5636	5650	5636
q6	232	137	140	137
q7	2233	1782	1785	1782
q8	3351	3543	3460	3460
q9	8928	9120	9058	9058
q10	3582	3551	3555	3551
q11	596	507	504	504
q12	833	600	604	600
q13	12018	3327	3247	3247
q14	297	274	271	271
q15	564	522	525	522
q16	693	648	628	628
q17	1858	1599	1606	1599
q18	8307	7648	7562	7562
q19	1652	1547	1627	1547
q20	2107	1854	1909	1854
q21	5692	5445	5408	5408
q22	676	578	588	578
Total cold run time: 71834 ms
Total hot run time: 60025 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197950 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5477fd10f2b6525eb9fa38c30480879f50120af3, data reload: false

query1	1272	992	963	963
query2	6248	2144	2005	2005
query3	11024	4604	4642	4604
query4	33317	23813	23616	23616
query5	3510	485	456	456
query6	291	195	192	192
query7	3988	296	303	296
query8	293	228	242	228
query9	9618	2710	2698	2698
query10	461	248	262	248
query11	18193	15313	15655	15313
query12	151	98	96	96
query13	1570	400	394	394
query14	9604	7544	7850	7544
query15	293	218	205	205
query16	7961	456	463	456
query17	1739	640	593	593
query18	1986	332	318	318
query19	393	164	161	161
query20	119	119	117	117
query21	214	107	111	107
query22	4860	4429	4669	4429
query23	35384	34452	34678	34452
query24	10610	2484	2429	2429
query25	598	379	391	379
query26	1163	152	149	149
query27	2552	286	288	286
query28	7754	2452	2490	2452
query29	807	412	463	412
query30	243	151	153	151
query31	1051	842	863	842
query32	99	55	51	51
query33	782	295	303	295
query34	998	508	529	508
query35	930	766	797	766
query36	1121	966	947	947
query37	132	76	76	76
query38	4501	4502	4393	4393
query39	1501	1484	1469	1469
query40	211	114	104	104
query41	45	43	44	43
query42	111	100	96	96
query43	547	514	483	483
query44	1329	840	816	816
query45	198	169	172	169
query46	1229	738	719	719
query47	2003	1944	1893	1893
query48	425	322	323	322
query49	1021	390	375	375
query50	813	405	385	385
query51	7371	7310	7381	7310
query52	110	92	94	92
query53	268	196	183	183
query54	1205	410	403	403
query55	83	79	76	76
query56	253	229	245	229
query57	1267	1112	1123	1112
query58	231	209	205	205
query59	3357	3013	2821	2821
query60	265	241	246	241
query61	106	108	138	108
query62	887	686	668	668
query63	211	193	190	190
query64	3991	658	645	645
query65	3294	3179	3194	3179
query66	837	298	297	297
query67	16169	15662	15617	15617
query68	4963	543	544	543
query69	429	250	255	250
query70	1241	1130	1142	1130
query71	337	240	250	240
query72	6402	4154	4064	4064
query73	773	354	355	354
query74	10452	9110	9044	9044
query75	3417	2655	2669	2655
query76	2769	1047	1066	1047
query77	383	284	283	283
query78	10647	9479	9474	9474
query79	1505	594	592	592
query80	1193	445	425	425
query81	546	233	238	233
query82	863	117	164	117
query83	240	143	148	143
query84	225	64	73	64
query85	1511	303	307	303
query86	480	258	304	258
query87	4712	4546	4575	4546
query88	3630	2207	2162	2162
query89	417	287	293	287
query90	2076	189	182	182
query91	136	101	108	101
query92	68	49	51	49
query93	2301	539	537	537
query94	1050	362	295	295
query95	356	251	250	250
query96	616	282	280	280
query97	2854	2680	2724	2680
query98	238	193	206	193
query99	1624	1359	1292	1292
Total cold run time: 303622 ms
Total hot run time: 197950 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.53 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5477fd10f2b6525eb9fa38c30480879f50120af3, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.03
query3	0.23	0.08	0.06
query4	1.62	0.10	0.10
query5	0.43	0.38	0.40
query6	1.14	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.52	0.50
query10	0.55	0.57	0.55
query11	0.14	0.11	0.11
query12	0.15	0.13	0.12
query13	0.61	0.60	0.60
query14	2.70	2.74	2.71
query15	0.90	0.84	0.83
query16	0.40	0.37	0.38
query17	1.06	1.08	1.07
query18	0.22	0.21	0.21
query19	1.90	1.91	1.94
query20	0.01	0.01	0.01
query21	15.38	0.58	0.57
query22	2.80	2.86	1.37
query23	16.89	1.10	0.86
query24	2.46	1.47	1.19
query25	0.14	0.19	0.17
query26	0.37	0.14	0.13
query27	0.05	0.04	0.03
query28	10.57	1.11	1.07
query29	12.59	3.26	3.25
query30	0.25	0.07	0.06
query31	2.88	0.39	0.38
query32	3.26	0.48	0.47
query33	3.09	3.03	3.06
query34	16.92	4.42	4.42
query35	4.53	4.50	4.47
query36	0.66	0.48	0.47
query37	0.10	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.03	0.02
query40	0.17	0.12	0.13
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.13 s
Total hot run time: 32.53 s

@morrySnow morrySnow merged commit 47b2ddd into apache:master Dec 11, 2024
34 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants