Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[spark] Compact procedure support where parameter #3119

Merged
merged 1 commit into from
Apr 1, 2024
Merged

[spark] Compact procedure support where parameter #3119

merged 1 commit into from
Apr 1, 2024

Conversation

Zouxxyy
Copy link
Contributor

@Zouxxyy Zouxxyy commented Mar 29, 2024 •
edited
Loading

Purpose

  • support: CALL sys.compact(table => 't', where => 'p>0 and p<3')
  • bug fix: Non-partition columns should not be in "partitions", e.g CALL sys.compact(table => 't', partitions => 'non_partition_col=1')) which will result in data loss

Tests

API and Format

Documentation

@Zouxxyy Zouxxyy requested a review from leaves12138 March 29, 2024 06:32
@Zouxxyy Zouxxyy marked this pull request as draft March 29, 2024 07:05
@Zouxxyy Zouxxyy force-pushed the dev/add-compact-filter-new branch from aa23308 to 896c5ee Compare March 29, 2024 07:20
@Zouxxyy Zouxxyy marked this pull request as ready for review March 29, 2024 07:20
leaves12138
leaves12138 approved these changes Mar 29, 2024
View reviewed changes
Copy link
Contributor

@leaves12138 leaves12138 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi merged commit 5b2ed7b into apache:master Apr 1, 2024
10 checks passed
@yuzelin
Copy link
Contributor

yuzelin commented Apr 1, 2024

Should we also modify flink compact?

@yuzelin yuzelin mentioned this pull request Apr 1, 2024
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leaves12138 leaves12138 leaves12138 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Zouxxyy @yuzelin @leaves12138 @JingsongLi