[lineage] Introduce jdbc lineage meta implementations #2267
Conversation
FangYongs
force-pushed
the
jdbc_lineage_meta_interface
branch
from
de38dc2 to
73706cf
Compare
| .replace( | ||
| EncodingUtils.escapeIdentifier(dataField.name()), | ||
| dataField.name()) |
There was a problem hiding this comment.
Why does it need to be replaced?
| "Jdbc password for Paimon to store some information such as lineage and meta."); | ||
|
|
||
| public static final ConfigOption<Boolean> JDBC_AUTO_DDL = | ||
| key("jdbc.auto-ddl") |
There was a problem hiding this comment.
How about replacing it with jdbc.auto-create?
| public static final ConfigOption<String> JDBC_URL = | ||
| key("jdbc.url") | ||
| .stringType() | ||
| .noDefaultValue() | ||
| .withDescription( | ||
| "Jdbc url for Paimon to store some information such as lineage and meta."); | ||
|
|
||
| public static final ConfigOption<String> JDBC_USER = | ||
| key("jdbc.user") | ||
| .stringType() | ||
| .noDefaultValue() | ||
| .withDescription( | ||
| "Jdbc user name for Paimon to store some information such as lineage and meta."); | ||
|
|
||
| public static final ConfigOption<String> JDBC_PASSWORD = | ||
| key("jdbc.password") | ||
| .stringType() | ||
| .noDefaultValue() | ||
| .withDescription( | ||
| "Jdbc password for Paimon to store some information such as lineage and meta."); | ||
|
|
||
| public static final ConfigOption<Boolean> JDBC_AUTO_DDL = |
There was a problem hiding this comment.
If these options are only provided to lineage for use, I prefer to add some prefixes in front, and considering other implementations in the future, the prefix can be lineage.kind.${identifier}.xxx.
|
@FangYongs hey, this has been open for a while. are there still plans to merge it, maybe for the upcoming Paimon 1.0? |
This is a unfinished issue... We should re-design this... I close this now, feel free to re-open this if you want to drive this again. |
Purpose
Linked issue: close #1480
This PR aims to introduce jdbc lineage meta with following capabilities:
jdbc.auto-ddlis trueTests
JdbcLineageMetaTestto validate auto ddl for source/sink lineage tables.API and Format
no
Documentation
no