Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] The dynamic bucket in the primary key table does not take effect. #4633

Closed
1 of 2 tasks
chenjiehao opened this issue Dec 4, 2024 · 2 comments
Closed
1 of 2 tasks
Labels
bug Something isn't working

Comments

@chenjiehao
Copy link

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

1.0-SNAPSHOT

Compute Engine

flink-1.20

Minimal reproduce step

CREATE TABLE word_count_1 (
word STRING PRIMARY KEY NOT ENFORCED,
cnt BIGINT
)WITH (
'bucket' = '-1',
'dynamic-bucket.target-row-num' ='100000',
'write-only'= 'true'
);

CREATE TEMPORARY TABLE word_table (
word STRING
) WITH (
'connector' = 'datagen',
'fields.word.length' = '1'
);

insert into word_count_1 select word,2 from word_table;
WX20241204-145027

WX20241204-145115@2x

What doesn't meet your expectations?

The number of buckets in the primary key table is automatically expanded based on the configuration parameters.

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@chenjiehao chenjiehao added the bug Something isn't working label Dec 4, 2024
@qyfftf
Copy link

qyfftf commented Dec 4, 2024

Your datagen table configuration has fields.word.length=1, which means it can only generate 16 unique values: 1-9 and a-f. Therefore, when writing to a Paimon primary key table with word as the primary key, there will actually be only 16 rows.
image

@chenjiehao
Copy link
Author

Thank you very much. I have a deep understanding of connector as datagen. There is no problem for me to verify it again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants