From 5ce94fab21c425244b37b7978236f0e1c5a2b5bd Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 26 Nov 2024 17:02:38 +0800 Subject: [PATCH] cdc: update debezium protocol (#19447) --- ticdc/ticdc-debezium.md | 41 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 39 insertions(+), 2 deletions(-) diff --git a/ticdc/ticdc-debezium.md b/ticdc/ticdc-debezium.md index 253c3b9da6016..7092f04ba0d71 100644 --- a/ticdc/ticdc-debezium.md +++ b/ticdc/ticdc-debezium.md @@ -29,7 +29,41 @@ This section describes the format definition of the DML event output in the Debe ### DML event -TiCDC encodes a DML event in the following format: +TiCDC encodes a DML event into a Kafka message, with both the key and value encoded in the Debezium format. + +### Key format + +```json +{ + "payload": { + "a": 4 + }, + "schema": { + "fields": [ + { + "field": "a", + "optional": true, + "type": "int32" + } + ], + "name": "default.test.t2.Key", + "optional": false, + "type": "struct" + } +} +``` + +The fields in the key only include primary key or unique index columns. The fields are explained as follows: + +| Field | Type | Description | +|:------------------|:--------|:----------------------------------------------------------------------------| +| `payload` | JSON | The information about primary key or unique index columns. The key and value in each field represent the column name and its current value, respectively. | +| `schema.fields` | JSON | The type information of each field in the payload, including the schema information of the row data before and after the change. | +| `schema.name` | String | The name of the schema, in the `"{cluster-name}.{schema-name}.{table-name}.Key"` format. | +| `schema.optional`| Boolean | Indicates whether the field is optional. When it is `true`, the field is optional. | +| `schema.type` | String | The data type of the field. | + +#### Value format ```json { @@ -125,7 +159,10 @@ The key fields of the preceding JSON data are explained as follows: | payload.source.commit_ts | Number | The `CommitTs` identifier when TiCDC generates this message. | | payload.source.db | String | The name of the database where the event occurs. | | payload.source.table | String | The name of the table where the event occurs. | -| schema.fields | JSON | The type information of each field in the payload, including the schema information of the row data before and after the change. | +| schema.fields | JSON | The type information of each field in the payload, including the schema information of the row data before and after the change. | +| schema.name | String | The name of the schema, in the `"{cluster-name}.{schema-name}.{table-name}.Envelope"` format. | +| schema.optional| Boolean | Indicates whether the field is optional. When it is `true`, the field is optional. | +| schema.type | String | The data type of the field. | ### Data type mapping