TiCDC Event Merging Issue

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc事件合并问题

| username: meiliangdeng

In the same transaction, multiple update [or on duplicate key update] CDC events for the same piece of data will be merged.
If they are not in the same transaction, they will not be merged.
Here is an example:

CREATE TABLE `store_dml_test` (
  `id` bigint(20) unsigned NOT NULL COMMENT 'id',
  `store_id` bigint(20) unsigned NOT NULL COMMENT 'store id',
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) COMMENT 'creation time',
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin

Suppose there is already a record with id=5653233197006196741 in the table:

If the following 3 update statements [3 transactions] are executed consecutively, ticdc will subscribe to 3 update events:

update store_dml_test set store_id = 50 where id = 5653233197006196741;
update store_dml_test set store_id = 51 where id = 5653233197006196741;
update store_dml_test set store_id = 52 where id = 5653233197006196741;

But if the following 3 update statements [in 1 transaction] are executed, ticdc will only subscribe to the last update event:

begin;
update store_mark_dml_test set store_id = 30 where id = 5653233197006196741;
update store_mark_dml_test set store_id = 31 where id = 5653233197006196741;
update store_mark_dml_test set store_id = 32 where id = 5653233197006196741;
commit;

Is there a configuration in TiDB that supports subscribing to 3 messages from ticdc regardless of the situation, instead of just 1?

| username: tidb菜鸟一只 | Original post link

This is logical. TiCDC pulls changes from the TiKV side. If a transaction repeatedly changes the same piece of data, it will only capture the final result of the change. TiCDC 常见问题解答 | PingCAP 文档中心