Data Inconsistency Between Upstream and Downstream After DM Synchronization

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: DM同步后上下游数据不一致

| username: 孤君888

【TiDB Usage Environment】Production Environment
【TiDB Version】
v7.1.2
【Reproduction Path】Operations performed that led to the issue
【Encountered Issue: Issue Phenomenon and Impact】

Background:

  1. The upstream is Percona Server 5.7.21 slave, its master is MySQL server 5.5, and the system environment is CentOS6.5.
  2. The DM component is installed with TIDB-7.1.2, and through tiup list, it shows v1.13.1. The DM configuration file specifies syncing only a certain table.
  3. The downstream is TIDB-v7.1.2.

Issue:

 1. Using the sync_diff_inspector tool to verify upstream and downstream data revealed inconsistencies, with table records not matching and some row data also inconsistent.
  1. The sync_diff_inspector configuration file is as follows:
    image

  2. After running sync_diff_inspector, the summary.txt file content is as follows:

  3. Randomly checked a repair SQL in the fix-on-tidb0 directory, as follows:

  4. The table structure of the upstream synced table is as follows:
    image

Can any expert guide me? How should I handle this issue?

【Resource Configuration】
【Attachments: Screenshots/Logs/Monitoring】

| username: Ming | Original post link

Is there a business running on the database?
The sync snapshot of 7.1 is the synchronization point of TiCDC.
Similar to the query statement:
select primary_ts, secondary_ts from tidb_cdc.syncpoint_v1 order by primary_ts desc limit 1;

Is there an issue with the comparison time point, causing inconsistency?

| username: 小龙虾爱大龙虾 | Original post link

It is not possible to compare dynamic data, right?

| username: dba远航 | Original post link

There are some version differences. Check if there are any unsupported syntax issues.

| username: 随缘天空 | Original post link

Are there any inconsistencies in the upstream and downstream table structures? It would be best to check. Also, were there any SQL errors during the synchronization process?

| username: TiDBer_小阿飞 | Original post link

Are the primary and replica databases upstream still running data?

| username: 孤君888 | Original post link

Got it. The upstream replica did not configure log_slave_updates, causing the binlog not to advance, which in turn caused DM to be unable to obtain the binlog, ultimately leading to data inconsistency.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.