Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Paimon cdc synchronizes the entire database. All job names are duplicates. Prometheus monitoring job names cannot be captured. #3053

Closed
2 tasks done
wqlsdb opened this issue Mar 20, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@wqlsdb
Copy link

wqlsdb commented Mar 20, 2024

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

0.7

Compute Engine

java api

Minimal reproduce step

paimon整库同步上生产,prometheus无法抓取job name,因为所有paimon整库同步名称都一样
./bin/flink run -d -t yarn-per-job -Djobmanager.memory.process.size=2096m -Dtaskmanager.memory.process.size=2096m
-Dyarn.application.queue=spark -Dyarn.application.node-label=ingest -Dyarn.application.name=obei_contract
/opt/module/flink-1.17.0/lib/paimon-flink-action-0.6.0-incubating.jar
mysql-sync-database
--warehouse hdfs://mycluster/user/hive/warehouse/ods.db
--database ods
--mysql-conf hostname=10.xx.xx.x.x.xx
--mysql-conf username=xxxxx
--mysql-conf password='xxxxxxxx'
--mysql-conf database-name=obei-contract
--including-tables 't_dcm_contract_detail|t_dcm_contract_head'
--catalog_conf metastore=hive
--catalog_conf uri=thrift://10.80.29.39:9083
--table-conf bucket=2
--table_prefix t_ods_
--table-conf changelog-producer=input
--table-conf sink.parallelism=2
image

What doesn't meet your expectations?

所有的整库同步job name 一样,不符合

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@wqlsdb wqlsdb added the bug Something isn't working label Mar 20, 2024
@zhuangchong
Copy link
Contributor

You can customize job name, see https://paimon.apache.org/docs/master/cdc-ingestion/overview/#job-name

image

@zhuangchong zhuangchong changed the title [Bug] paimon cdc 整库同步,所有的job name都是重复的prometheus监控作业名字无法抓取 [Bug] Paimon cdc synchronizes the entire database. All job names are duplicates. Prometheus monitoring job names cannot be captured. Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants