Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using the MySQL CDC synchronization tool, I found that the --metadata-column parameter, specifically op_ts, cannot be properly converted from a timestamp to TIMESTAMP_LTZ(3). #2182

Closed
1 of 2 tasks
labixiaoxiaopang opened this issue Oct 26, 2023 · 1 comment · Fixed by #2234
Assignees
Labels
bug Something isn't working

Comments

@labixiaoxiaopang
Copy link

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

0.6-SNAPSHOT

Compute Engine

flink: 1.17.1
flink cdc: 2.3

Minimal reproduce step

1:use paimon-flink-action-0.6-SNAPSHOT.jar

2: use param --metadata-column op_ts,database_name,table_name

3:I've noticed that there's no tm_ms field in the org.apache.paimon.flink.action.cdc.mysql.format.DebeziumEvent file under the public class Source. I added this field, thinking I had solved the issue.

4:However, during the usage, I found that the op_ts field is using the org.apache.paimon.utils.TypeUtils utility for conversion, and I traced it to the castFromStringInternal method. It appears that TIMESTAMP_WITHOUT_TIME_ZONE cannot properly convert the timestamp.

What doesn't meet your expectations?

I would like the op_ts field to be written as either a timestamp or a bigint type.

Anything else?

1:Why isn't the data type of the op_ts field bigint?

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@labixiaoxiaopang labixiaoxiaopang added the bug Something isn't working label Oct 26, 2023
@zhuangchong
Copy link
Contributor

I will solve this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants