Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Predicate-size uid partitioning #65

Open
EnricoMi opened this issue Nov 16, 2020 · 0 comments
Open

Predicate-size uid partitioning #65

EnricoMi opened this issue Nov 16, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@EnricoMi
Copy link
Collaborator

Given a predicate partitioning, we can further partition each partition orthogonally by uids. Some partitions may contain more rows than others. By splitting large partitions into more parts than smaller ones, we can achieve a more even final partitioning.

The zero service provides predicate size statistics. With these, we can compute the size of each partition. This does not reflect the number of rows, but the size of the predicate values, where string, geo, password and default refer to variable size predicates. This makes estimating the number of rows difficult. Compression might also make estimation more complex. However, we can see this size as a transfer-cost estimate and make the final partitioning even-sized regarding that metric.

@EnricoMi EnricoMi added the enhancement New feature or request label Nov 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant