You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I use the S3 Sink Connector on a topic with Avro records which are fully transitive compatible. I want to achieve that records are always written in larger batches to S3 in Parquet format. This works perfectly if the schema is identical. However, I was only able to keep the connector adding records to the same batch, if either the schema version of the new record is larger or equal (FORWARD), or if the schema version of the new record is less or equal (BACKWARD). I would expect if I set schema.compatibility to "FULL", that it keeps adding to the batch independent of the schema version. However, this is not the case. It treats "FULL" exactly the same as "BACKWARD":
Hello,
I use the S3 Sink Connector on a topic with Avro records which are fully transitive compatible. I want to achieve that records are always written in larger batches to S3 in Parquet format. This works perfectly if the schema is identical. However, I was only able to keep the connector adding records to the same batch, if either the schema version of the new record is larger or equal (FORWARD), or if the schema version of the new record is less or equal (BACKWARD). I would expect if I set schema.compatibility to "FULL", that it keeps adding to the batch independent of the schema version. However, this is not the case. It treats "FULL" exactly the same as "BACKWARD":
kafka-connect-storage-common/core/src/main/java/io/confluent/connect/storage/schema/StorageSchemaCompatibility.java
Line 107 in 092297f
I would expect that "FULL" overrides "checkVersions" and always returns true.
Do you agree with this, or did I understood something wrong?
The text was updated successfully, but these errors were encountered: