[Bug] Row field descriptions are ignored when creating tables with Flink SQL

[atallahade]

Search before asking

• I searched in the issues and found nothing similar.

Paimon version

a389af6

Compute Engine

Flink

Minimal reproduce step

Use sql-client.sh and create this table in a Paimon catalog:

CREATE TABLE my_table (my_row ROW<my_field INT 'my description'>);

SHOW CREATE TABLE my_table;
CREATE TABLE `my_catalog`.`my_database`.`my_table` (
  `my_row` ROW<`my_field` INT>
) WITH (
  'path' = 'file:/paimon_data/my_database.db/my_table'
)

What doesn't meet your expectations?

Descriptions of row fields are not persisted when using Flink SQL. This is a problem when manually creating tables for the KafkaSyncTableAction which uses descriptions on row fields, causing schema mismatches.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[lsyldliu]

Thanks for reporting it, do you plan to fix it?

[ChaomingZhangCN]

The main reason is that Flink UnresolvedPhysicalColumn only uses columnName, comment, dataType(DataType) to describe a physical column. That's why the inner nested comment will lost.

[atallahade]

Thanks for reporting it, do you plan to fix it?

I wasn't planning on fixing this because this isn't urgent for my use case, and as a workaround it seem that I could just edit the schema file to add the description fields (I don't actually need to do it).
I can try to understand how all of this works and propose a fix, but can't promise to do that soon.