Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema Compatibility "FULL" should consider all versions as compatible. #338

Open
ueisele opened this issue Dec 11, 2023 · 0 comments
Open

Comments

@ueisele
Copy link

ueisele commented Dec 11, 2023

Hello,

I use the S3 Sink Connector on a topic with Avro records which are fully transitive compatible. I want to achieve that records are always written in larger batches to S3 in Parquet format. This works perfectly if the schema is identical. However, I was only able to keep the connector adding records to the same batch, if either the schema version of the new record is larger or equal (FORWARD), or if the schema version of the new record is less or equal (BACKWARD). I would expect if I set schema.compatibility to "FULL", that it keeps adding to the batch independent of the schema version. However, this is not the case. It treats "FULL" exactly the same as "BACKWARD":

I would expect that "FULL" overrides "checkVersions" and always returns true.
Do you agree with this, or did I understood something wrong?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant