Error of Duplicate Primary Key in DM Synchronization

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm同步出现主键重复报错

| username: Jjjjayson_zeng

[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration]
[Attachment: Screenshot/Log/Monitoring]


The DM has this primary key duplication issue, and it still occurs after resume-task. You have to keep resume-task.

| username: xingzhenxiang | Original post link

Hello, the position in relay.meta refers to the starting point from which the upstream binlog is pulled. When an error occurs during synchronization, it means that an event in the relay log encountered an error when being executed downstream, so modifying relay.meta is ineffective. Generally, when encountering issues with replication, it is highly likely that the downstream TiDB does not support it. Therefore, you can use sql-skip to skip the error or modify the binlog filter in the task configuration to filter it out in advance. If you want to specify starting synchronization from a certain position, you can modify the checkpoint information (not recommended, as incorrect modifications may lead to data loss).

| username: db_user | Original post link

Change safe-mode to true, then perform stop-task and start-task, and observe whether DM reports any errors. If it is a normal primary key conflict, it will not report an error. If there is an issue with the command execution, you can check the error logs of TiDB. Additionally, based on the pos point, retrieve the corresponding SQL statement and execute it in TiDB to see if there are any errors.

| username: Jjjjayson_zeng | Original post link

This issue occurs during the load phase when redoing the task.

| username: db_user | Original post link

Based on this, please proceed.