Issues Arise When Downstream Fields Exceed Upstream Fields in DM Synchronization

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm同步 下游字段多于上游会出现问题

| username: TiDBer_QHSxuEa1

Currently, a DM synchronization bug has been found during testing.

Upstream fields: | id | student_id | name |
Downstream fields: | id | student_id | name | flag |

After configuring binlog-schema update according to the official documentation, the synchronization task starts running normally.
Both upstream and downstream have this data:
Upstream:
| id | student_id | name |
| 1 | 00000001 | Zhang San |

Downstream:
| id | student_id | name | flag |
| 1 | 00000001 | Zhang San | null |

The downstream data was updated to:
| id | student_id | name | flag |
| 1 | 00000001 | Zhang San | 1 |

When the upstream data is updated again:
| id | student_id | name | → | id | student_id | name |
|1 | 00000001 | Zhang San | → | 1 | 00000001 | Li Si |

The downstream data becomes:
| id | student_id | name | flag |
| 1 | 00000001 | Li Si | null |

The additional field in the downstream is reset to null.
The reason should be that the binary log format of the upstream MySQL is row, but I don’t know how to handle this issue.

| username: WalterWj | Original post link

To synchronize and generate SQL, it needs to be changed to the “insert on duplicate” syntax. The “replace” statement is a delete + insert, so it becomes null, right?

| username: Jasper | Original post link

The official website has an explanation, just look at this document: 下游存在更多列的迁移场景 | PingCAP 文档中心

| username: TiDBer_QHSxuEa1 | Original post link

Found the problem, it was caused by me configuring safe_mode later. The update was parsed into delete + replace.

| username: 这里介绍不了我 | Original post link

This is quite interesting, I learned something.

| username: TiDBer_rvITcue9 | Original post link

Checked in.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.