[spark] Fix alter table column type change nullability

[ulysses-you]

Purpose

Spark catalog TableChange.UpdateColumnType should only change the data type, but paimon always changes the nullable to true. This pr keeps the original field nullability.

To fix the issue:

CREATE TABLE t (a INT, b BIGINT NOT NULL) USING paimon;
ALTER TABLE t CHANGE COLUMN b TYPE string;

SHOW CREATE TABLE t; => b STRING NOT NULL

Tests

add test

API and Format

no

Documentation

[ulysses-you]

All Spark tests passed, the failed test seems irrelevant:

2024-08-15T03:51:37.1011203Z [ERROR] Tests run: 37, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 797.345 s <<< FAILURE! - in org.apache.paimon.hive.Hive23CatalogITCase
2024-08-15T03:51:37.1078168Z [ERROR] org.apache.paimon.hive.Hive23CatalogITCase.testDropPartitionFromBranchLocationInProperties  Time elapsed: 45.84 s  <<< ERROR!
2024-08-15T03:51:37.1080812Z org.apache.flink.table.api.TableException: Could not execute ALTER TABLE my_hive.test_db.t DROP PARTITION (pt=2)

cc @JingsongLi thank you

[JingsongLi]

All Spark tests passed, the failed test seems irrelevant:

2024-08-15T03:51:37.1011203Z [ERROR] Tests run: 37, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 797.345 s <<< FAILURE! - in org.apache.paimon.hive.Hive23CatalogITCase
2024-08-15T03:51:37.1078168Z [ERROR] org.apache.paimon.hive.Hive23CatalogITCase.testDropPartitionFromBranchLocationInProperties  Time elapsed: 45.84 s  <<< ERROR!
2024-08-15T03:51:37.1080812Z org.apache.flink.table.api.TableException: Could not execute ALTER TABLE my_hive.test_db.t DROP PARTITION (pt=2)

cc @JingsongLi thank you

@ulysses-you
Your repo is passed. https://github.com/ulysses-you/paimon/tree/nullable

[JingsongLi]

+1

[ulysses-you]

Your repo is passed. https://github.com/ulysses-you/paimon/tree/nullable

oh, interesting, it is the double check!