Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core] Store cardinality in the deletion vector meta #4699

Merged
merged 3 commits into from
Dec 13, 2024

Conversation

Zouxxyy
Copy link
Contributor

@Zouxxyy Zouxxyy commented Dec 13, 2024

Purpose

Store cardinality in the deletion vector meta, new meta like this:

 1. f0: the data file name corresponding to this deletion vector.
 2. f1: the starting offset of this deletion vector in the index file.
 3. f2: the length of this deletion vector in the index file.
 4. cardinality: the number of deleted rows.

new:

cardinality: It can be used for optimizing count(*) on the dv table.

compatibility:

upgrate CommitMessageSerializer to version 5
upgrate DataSplit to version 4

Tests

API and Format

Documentation

@Zouxxyy Zouxxyy changed the title [WIP][core] Store cardinality first and last in the deletion vector meta [core] Store cardinality first and last in the deletion vector meta Dec 13, 2024
@JingsongLi
Copy link
Contributor

Can you explain the usage for them separately?

@Zouxxyy Zouxxyy force-pushed the dev/update-dv-meta1 branch from 1e5c67e to 22c9b09 Compare December 13, 2024 06:26
@JingsongLi JingsongLi changed the title [core] Store cardinality first and last in the deletion vector meta [core] Store cardinality in the deletion vector meta Dec 13, 2024
@Zouxxyy Zouxxyy force-pushed the dev/update-dv-meta1 branch from 22c9b09 to e28f834 Compare December 13, 2024 07:07
@Zouxxyy Zouxxyy force-pushed the dev/update-dv-meta1 branch from e28f834 to f85c622 Compare December 13, 2024 08:29
Copy link
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi merged commit 47a8871 into apache:master Dec 13, 2024
13 checks passed
@Zouxxyy Zouxxyy deleted the dev/update-dv-meta1 branch December 13, 2024 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants