DM Data Synchronization Delay

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm同步数据延时

| username: TiDBer_AxdzSBqt

【TiDB Usage Environment】Both production and testing environments
【TiDB Version】6.5.2
【Reproduction Path】Occasional occurrence
【Encountered Issue: DM synchronizes MySQL data to TiDB. At 15:30, an insert is made in MySQL, but it is not synchronized to TiDB. At 15:32, the data in MySQL is updated, and only then is the data synchronized to the downstream TiDB.

| username: 有猫万事足 | Original post link

Check the binlog to see if the recorded time is as you mentioned. If the binlog records are the same, then check the corresponding time in the DM Grafana monitoring.

| username: tidb菜鸟一只 | Original post link

Is auto-commit on or off?

| username: TiDBer_AxdzSBqt | Original post link

The upstream MySQL binlog has two entries, one insert and one update, but TiDB only has one insert, and the final result is consistent. I suspect that DM performed a merge, but the official documentation says it is disabled by default, and the configuration does not have merging enabled.

| username: TiDBer_AxdzSBqt | Original post link

All configurations are set according to the official website defaults.

| username: 有猫万事足 | Original post link

I haven’t encountered this situation; it’s basically corresponding to binlog.

You can look at this graph for binlog delay:

It’s in the standard section of Grafana. As for your current situation, I can’t reproduce it.

Please provide source.yaml, task.yaml, DM version, and the versions of upstream and downstream if possible (make sure to remove any passwords). We need a reproducible environment. Currently, it’s unclear where the problem lies.

| username: TiDBer_AxdzSBqt | Original post link

It’s not always reproducible, it happens occasionally, but it has already occurred three times online. The longest delay was an hour and a half. I’ll check the monitoring you mentioned, thanks.

| username: redgame | Original post link

When there is latency, you can check if the latency is caused by network issues or high server load.

| username: Fly-bird | Original post link

Check your resource utilization in the dashboard’s monitoring section, with a focus on KV.

| username: TiDB_C罗 | Original post link

It is recommended to separately check the data volume and load IO and other resources of the upstream MySQL and downstream TiDB.