Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLUTEN-6877][CH] Support anti/semi join with inequal join condition #6913

Merged
merged 3 commits into from
Aug 23, 2024
Merged

[GLUTEN-6877][CH] Support anti/semi join with inequal join condition #6913

merged 3 commits into from
Aug 23, 2024

Conversation

lgbo-ustc
Copy link
Contributor

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

Fixes: #6877

Support anti/semi join with inequal condition, e.g.

select t1.* from t1 left semi t2 on t1.k = t2.k and t1.a < t2.b

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

unit tests

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

@lgbo-ustc lgbo-ustc added the enhancement New feature or request label Aug 19, 2024
@lgbo-ustc lgbo-ustc force-pushed the 6877 branch 2 times, most recently from 5cbe83f to 4bac02e Compare August 19, 2024 02:30
@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

2 similar comments
@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

@github-actions github-actions bot added the CORE works for Gluten Core label Aug 21, 2024
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse C

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

1 similar comment
@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions github-actions bot added the RSS label Aug 23, 2024
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

1 similar comment
@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

1 similar comment
@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

2 similar comments
@lgbo-ustc
Copy link
Contributor Author

Run Gluten Clickhouse CI

@baibaichen
Copy link
Contributor

Run Gluten Clickhouse CI

baibaichen
baibaichen approved these changes Aug 23, 2024
View reviewed changes
Copy link
Contributor

@baibaichen baibaichen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@baibaichen baibaichen merged commit a575395 into apache:main Aug 23, 2024
8 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_08_23_2024_time.csv log/native_master_08_22_2024_d12bf1fa34_time.csv difference percentage
q1 13.85 14.71 0.853 106.15%
q2 14.92 14.38 -0.531 96.44%
q3 4.64 4.30 -0.335 92.77%
q4 70.93 72.04 1.114 101.57%
q5 7.95 7.67 -0.280 96.48%
q6 4.06 3.85 -0.202 95.02%
q7 4.82 4.98 0.163 103.38%
q8 5.57 3.36 -2.202 60.43%
q9 24.20 23.94 -0.264 98.91%
q10 10.15 9.88 -0.271 97.33%
q11 38.93 38.30 -0.630 98.38%
q12 1.42 1.36 -0.058 95.89%
q13 6.59 6.48 -0.118 98.22%
q14a 45.33 45.84 0.519 101.15%
q14b 43.07 43.62 0.546 101.27%
q15 2.51 2.51 -0.004 99.84%
q16 45.73 46.14 0.417 100.91%
q17 4.97 5.04 0.067 101.35%
q18 6.80 7.01 0.202 102.97%
q19 2.09 2.27 0.174 108.34%
q20 2.72 1.57 -1.146 57.81%
q21 1.15 1.19 0.033 102.88%
q22 8.04 7.68 -0.364 95.47%
q23a 106.31 104.47 -1.849 98.26%
q23b 129.65 129.89 0.240 100.19%
q24a 100.53 103.08 2.547 102.53%
q24b 109.93 104.81 -5.120 95.34%
q25 4.19 4.19 -0.000 99.99%
q26 3.23 2.99 -0.246 92.39%
q27 3.64 3.85 0.209 105.75%
q28 32.78 32.66 -0.116 99.65%
q29 12.79 12.07 -0.719 94.38%
q30 4.89 6.53 1.645 133.67%
q31 7.55 7.51 -0.040 99.47%
q32 2.77 1.33 -1.444 47.92%
q33 4.28 4.65 0.363 108.48%
q34 4.04 4.18 0.140 103.47%
q35 9.40 9.25 -0.152 98.38%
q36 4.72 5.00 0.281 105.95%
q37 5.73 4.90 -0.827 85.57%
q38 14.08 14.61 0.528 103.75%
q39a 3.83 3.49 -0.338 91.17%
q39b 3.09 2.80 -0.292 90.54%
q40 4.00 3.89 -0.110 97.25%
q41 0.62 0.59 -0.030 95.10%
q42 1.05 0.89 -0.166 84.26%
q43 4.66 4.83 0.167 103.59%
q44 9.93 10.00 0.076 100.77%
q45 3.31 3.36 0.052 101.59%
q46 3.71 3.77 0.058 101.56%
q47 18.78 19.10 0.323 101.72%
q48 5.26 5.39 0.131 102.49%
q49 9.23 8.54 -0.693 92.50%
q50 21.60 21.39 -0.210 99.03%
q51 10.07 9.99 -0.073 99.27%
q52 1.14 1.12 -0.025 97.83%
q53 2.42 2.50 0.085 103.51%
q54 3.81 4.09 0.279 107.32%
q55 1.00 1.08 0.078 107.79%
q56 4.21 4.08 -0.124 97.05%
q57 10.75 10.87 0.119 101.11%
q58 2.49 2.35 -0.140 94.38%
q59 11.20 11.46 0.261 102.33%
q60 4.04 4.18 0.138 103.42%
q61 4.13 4.08 -0.041 99.00%
q62 4.21 5.17 0.963 122.89%
q63 2.31 2.45 0.134 105.79%
q64 63.64 61.41 -2.225 96.50%
q65 17.92 18.63 0.715 103.99%
q66 5.54 4.33 -1.211 78.16%
q67 381.34 380.50 -0.846 99.78%
q68 3.62 3.53 -0.091 97.47%
q69 5.39 5.18 -0.209 96.13%
q70 11.44 11.74 0.297 102.60%
q71 2.37 2.36 -0.010 99.59%
q72 214.84 213.17 -1.666 99.22%
q73 2.38 2.25 -0.123 94.81%
q74 23.76 23.93 0.167 100.70%
q75 26.77 26.72 -0.050 99.81%
q76 11.57 11.39 -0.179 98.46%
q77 2.24 2.29 0.050 102.24%
q78 49.84 49.72 -0.121 99.76%
q79 3.92 3.91 -0.018 99.54%
q80 12.61 12.54 -0.066 99.47%
q81 4.94 4.95 0.004 100.09%
q82 7.76 7.96 0.195 102.51%
q83 1.52 1.79 0.274 118.04%
q84 2.93 2.87 -0.052 98.23%
q85 7.79 8.16 0.376 104.83%
q86 4.05 4.36 0.315 107.77%
q87 13.71 13.71 -0.003 99.98%
q88 21.29 21.67 0.385 101.81%
q89 3.64 3.64 0.001 100.02%
q90 3.53 3.09 -0.432 87.73%
q91 2.41 2.43 0.025 101.04%
q92 1.30 1.34 0.043 103.31%
q93 39.71 38.66 -1.042 97.38%
q94 25.05 25.33 0.279 101.11%
q9 89.76 87.36 -2.400 97.33%
q5 2.58 2.91 0.327 112.69%
q96 17.91 17.42 -0.491 97.26%
q97 1.98 1.94 -0.034 98.28%
q98 10.39 10.43 0.035 100.33%
q99 10.39 10.43 0.035 100.33%
total 2183.18 2169.14 -14.038 99.36%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_08_23_2024_time.csv log/native_master_08_22_2024_d12bf1fa34_time.csv difference percentage
q1 41.55 40.75 -0.795 98.09%
q2 30.51 29.92 -0.589 98.07%
q3 51.76 52.08 0.317 100.61%
q4 41.63 41.89 0.260 100.63%
q5 104.68 105.35 0.672 100.64%
q6 11.52 13.04 1.525 113.25%
q7 120.06 117.08 -2.979 97.52%
q8 117.51 116.07 -1.441 98.77%
q9 169.81 167.88 -1.933 98.86%
q10 65.03 65.34 0.315 100.48%
q11 27.16 26.49 -0.667 97.54%
q12 30.05 31.70 1.649 105.49%
q13 51.92 52.26 0.343 100.66%
q14 25.59 27.23 1.643 106.42%
q15 53.08 52.59 -0.488 99.08%
q16 18.56 19.24 0.684 103.68%
q17 130.08 132.20 2.118 101.63%
q18 199.91 198.60 -1.310 99.34%
q19 28.23 25.81 -2.423 91.42%
q20 41.08 41.51 0.426 101.04%
q21 394.06 390.76 -3.301 99.16%
q22 17.05 15.80 -1.256 92.64%
total 1770.83 1763.60 -7.229 99.59%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_08_24_2024_time.csv log/native_master_08_23_2024_a575395a60_time.csv difference percentage
q1 13.95 13.85 -0.092 99.34%
q2 14.11 14.92 0.807 105.72%
q3 2.47 4.64 2.173 188.14%
q4 70.88 70.93 0.045 100.06%
q5 9.27 7.95 -1.327 85.69%
q6 3.92 4.06 0.134 103.41%
q7 6.53 4.82 -1.707 73.85%
q8 5.48 5.57 0.086 101.57%
q9 24.00 24.20 0.204 100.85%
q10 9.85 10.15 0.303 103.07%
q11 38.36 38.93 0.571 101.49%
q12 1.29 1.42 0.127 109.88%
q13 6.45 6.59 0.141 102.19%
q14a 44.95 45.33 0.378 100.84%
q14b 43.69 43.07 -0.618 98.59%
q15 2.53 2.51 -0.020 99.22%
q16 45.47 45.73 0.252 100.55%
q17 4.93 4.97 0.035 100.71%
q18 7.35 6.80 -0.551 92.51%
q19 2.54 2.09 -0.450 82.30%
q20 1.47 2.72 1.246 184.79%
q21 1.03 1.15 0.119 111.46%
q22 7.65 8.04 0.396 105.18%
q23a 103.70 106.31 2.611 102.52%
q23b 127.16 129.65 2.490 101.96%
q24a 109.84 100.53 -9.305 91.53%
q24b 111.26 109.93 -1.328 98.81%
q25 4.57 4.19 -0.388 91.51%
q26 3.31 3.23 -0.080 97.60%
q27 4.09 3.64 -0.451 88.97%
q28 29.61 32.78 3.165 110.69%
q29 9.46 12.79 3.327 135.18%
q30 5.10 4.89 -0.213 95.83%
q31 7.26 7.55 0.288 103.97%
q32 1.20 2.77 1.572 230.94%
q33 4.26 4.28 0.027 100.63%
q34 4.01 4.04 0.024 100.59%
q35 7.79 9.40 1.616 120.76%
q36 4.66 4.72 0.064 101.38%
q37 4.58 5.73 1.143 124.93%
q38 15.16 14.08 -1.080 92.88%
q39a 3.08 3.83 0.744 124.14%
q39b 2.70 3.09 0.390 114.45%
q40 3.91 4.00 0.089 102.29%
q41 0.62 0.62 0.003 100.45%
q42 0.88 1.05 0.178 120.34%
q43 4.86 4.66 -0.191 96.06%
q44 9.77 9.93 0.153 101.57%
q45 3.24 3.31 0.062 101.92%
q46 3.72 3.71 -0.013 99.65%
q47 18.69 18.78 0.083 100.44%
q48 5.25 5.26 0.012 100.22%
q49 8.53 9.23 0.707 108.29%
q50 21.80 21.60 -0.199 99.09%
q51 9.48 10.07 0.587 106.19%
q52 1.17 1.14 -0.028 97.60%
q53 2.38 2.42 0.039 101.65%
q54 3.94 3.81 -0.134 96.61%
q55 1.11 1.00 -0.111 90.02%
q56 4.07 4.21 0.140 103.44%
q57 11.34 10.75 -0.589 94.81%
q58 2.50 2.49 -0.011 99.54%
q59 11.00 11.20 0.191 101.74%
q60 4.17 4.04 -0.129 96.90%
q61 4.18 4.13 -0.054 98.71%
q62 4.61 4.21 -0.406 91.20%
q63 2.36 2.31 -0.046 98.05%
q64 64.88 63.64 -1.244 98.08%
q65 18.25 17.92 -0.335 98.17%
q66 4.08 5.54 1.464 135.88%
q67 392.57 381.34 -11.224 97.14%
q68 3.62 3.62 -0.002 99.95%
q69 5.08 5.39 0.306 106.01%
q70 11.83 11.44 -0.394 96.67%
q71 2.47 2.37 -0.101 95.90%
q72 218.07 214.84 -3.233 98.52%
q73 2.21 2.38 0.169 107.67%
q74 23.84 23.76 -0.078 99.67%
q75 26.98 26.77 -0.220 99.19%
q76 11.72 11.57 -0.146 98.75%
q77 2.33 2.24 -0.089 96.18%
q78 50.33 49.84 -0.493 99.02%
q79 4.02 3.92 -0.096 97.61%
q80 12.44 12.61 0.171 101.38%
q81 4.88 4.94 0.057 101.16%
q82 7.08 7.76 0.679 109.58%
q83 1.64 1.52 -0.119 92.74%
q84 2.76 2.93 0.163 105.89%
q85 7.92 7.79 -0.133 98.33%
q86 4.17 4.05 -0.121 97.09%
q87 14.16 13.71 -0.453 96.80%
q88 21.67 21.29 -0.382 98.24%
q89 3.70 3.64 -0.060 98.37%
q90 3.17 3.53 0.360 111.36%
q91 2.50 2.41 -0.096 96.18%
q92 1.36 1.30 -0.064 95.26%
q93 39.80 39.71 -0.090 99.77%
q94 24.61 25.05 0.442 101.80%
q9 88.93 89.76 0.835 100.94%
q5 2.86 2.58 -0.280 90.20%
q96 17.64 17.91 0.266 101.51%
q97 1.85 1.98 0.131 107.10%
q98 10.11 10.39 0.279 102.76%
q99 10.11 10.39 0.279 102.76%
total 2190.11 2183.18 -6.926 99.68%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_08_24_2024_time.csv log/native_master_08_23_2024_a575395a60_time.csv difference percentage
q1 40.07 41.55 1.472 103.67%
q2 30.09 30.51 0.424 101.41%
q3 53.28 51.76 -1.519 97.15%
q4 43.54 41.63 -1.910 95.61%
q5 105.20 104.68 -0.526 99.50%
q6 12.68 11.52 -1.167 90.80%
q7 114.61 120.06 5.447 104.75%
q8 115.07 117.51 2.439 102.12%
q9 169.34 169.81 0.472 100.28%
q10 65.30 65.03 -0.270 99.59%
q11 27.88 27.16 -0.720 97.42%
q12 28.48 30.05 1.570 105.51%
q13 51.26 51.92 0.664 101.30%
q14 25.24 25.59 0.343 101.36%
q15 55.93 53.08 -2.858 94.89%
q16 17.80 18.56 0.759 104.26%
q17 131.59 130.08 -1.506 98.86%
q18 201.19 199.91 -1.284 99.36%
q19 26.05 28.23 2.179 108.36%
q20 41.12 41.08 -0.038 99.91%
q21 384.37 394.06 9.692 102.52%
q22 15.25 17.05 1.803 111.82%
total 1755.36 1770.83 15.466 100.88%

@baibaichen baibaichen mentioned this pull request Aug 26, 2024
Merged
sharkdtu pushed a commit to sharkdtu/gluten that referenced this pull request Nov 11, 2024
@lgbo-ustc
[GLUTEN-6877][CH] Support anti/semi join with inequal join condition (a…
1b0ef20 
…pache#6913)

* support anti/semi join with mixed join condition

* enable uts

* fix tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@exmy exmy exmy left review comments

@baibaichen baibaichen baibaichen approved these changes

Assignees
No one assigned
Labels
CLICKHOUSE enhancement New feature or request RSS
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CH] Inequal join support left semi/anti
4 participants
@lgbo-ustc @baibaichen @GlutenPerfBot @exmy