DM 6.6.0, TiDB 6.1.2, Data Synchronization Exception from MySQL

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm6.6.0,tidb6.1.2,从mysql同步数据异常

| username: 暗夜中独舞

[TiDB Usage Environment] Production Environment
[TiDB Version] 6.1.2
[Reproduction Path] Use TiDB to merge sharded tables from the production database into the same database and table
[Encountered Problem] New data in MySQL is not synchronized to TiDB
[Resource Configuration] New data in MySQL is not synchronized to TiDB. Each time, after stopping the task, deleting the dm_meta database, and then restarting the task, the new data gets synchronized.
Using query-status to check the status, it shows Running, but there are some logs:
“unsynced”: [
“this DM-worker doesn’t receive any shard DDL of this group”

Including using shard-ddl-lock to check locks, there are many locks. Using shard-ddl-lock unlock prompts that the lock-id does not exist.

| username: Meditator | Original post link

  1. Send the task configuration.
  2. Send the complete query-status content.
  3. Is it a scenario of merging sharded tables?
| username: 暗夜中独舞 | Original post link

  1. Supplement the query-status output
| username: 暗夜中独舞 | Original post link

$ tiup dmctl --master-addr 127.0.0.1:8261 shard-ddl-lock ai
tiup is checking updates for component dmctl …
Starting component dmctl: /home/tidb/.tiup/components/dmctl/v6.6.0/dmctl/dmctl --master-addr 127.0.0.1:8261 shard-ddl-lock ai
{
“result”: true,
“msg”: “”,
“locks”: [
{
“ID”: “ai-scriptDraft.t_push”,
“task”: “ai”,
“mode”: “pessimistic”,
“owner”: “data18000”,
“DDLs”: [
“ALTER TABLE scriptDraft.t_push ADD COLUMN studioIntersection TINYINT(1) NULL DEFAULT NULL COMMENT ‘Is it a studio intersection user’ AFTER isWxPublic
],
“synced”: [
“data18000”
],
“unsynced”: [
“data18001”,
“data18002”,
“data18003”,
“data18004”,
“data18005”,
“data18006”,
“data18007”
]
},
{
“ID”: “ai-scriptDraft.t_user_relevance”,
“task”: “ai”,
“mode”: “pessimistic”,
“owner”: “data18000”,
“DDLs”: [
“ALTER TABLE scriptDraft.t_user_relevance ADD COLUMN tags VARCHAR(255) NULL COMMENT ‘Tags’ AFTER studioId
],
“synced”: [
“data18000”
],
“unsynced”: [
“data18001”,
“data18002”,
“data18003”,
“data18004”,
“data18005”,
“data18006”,
“data18007”
]
},
{
“ID”: “ai-ai.t_user”,
“task”: “ai”,
“mode”: “pessimistic”,
“owner”: “data18002”,
“DDLs”: [
“ALTER TABLE ai.t_user ADD COLUMN config VARCHAR(128) CHARACTER SET UTF8MB4 COLLATE utf8mb4_unicode_ci NULL DEFAULT NULL COMMENT ‘User configuration’”
],
“synced”: [
“data18001”,
“data18002”,
“data18003”,
“data18005”,
“data18006”,
“data18007”
],
“unsynced”: [
“data18000”,
“data18004”
]
},
{
“ID”: “ai-aiimdata_0012.t_user”,
“task”: “ai”,
“mode”: “pessimistic”,
“owner”: “data18004”,
“DDLs”: [
“ALTER TABLE aiimdata_0012.t_user RENAME AS aiimdata_0012.t_user_1
],
“synced”: [
“data18004”
],
“unsynced”: [
“data18000”,
“data18001”,
“data18002”,
“data18003”,
“data18005”,
“data18006”,
“data18007”
]
},
{
“ID”: “ai-scriptDraft.t_user”,
“task”: “ai”,
“mode”: “pessimistic”,
“owner”: “data18000”,
“DDLs”: [
“ALTER TABLE scriptDraft.t_user DISABLE KEYS”
],
“synced”: [
“data18000”
],
“unsynced”: [
“data18001”,
“data18002”,
“data18003”,
“data18004”,
“data18005”,
“data18006”,
“data18007”
]
}

$ tiup dmctl --master-addr 127.0.0.1:8261 shard-ddl-lock unlock ai-ai.t_user
-bash: ai: command not found
-bash: t_user: command not found
tiup is checking updates for component dmctl …
Starting component dmctl: /home/tidb/.tiup/components/dmctl/v6.6.0/dmctl/dmctl --master-addr 127.0.0.1:8261 shard-ddl-lock unlock ai-.
{
“result”: false,
“msg”: “can’t find task name from lock-ID”
}

$ tiup dmctl --master-addr 127.0.0.1:8261 shard-ddl-lock unlock ai-ai.t_user
tiup is checking updates for component dmctl …
Starting component dmctl: /home/tidb/.tiup/components/dmctl/v6.6.0/dmctl/dmctl --master-addr 127.0.0.1:8261 shard-ddl-lock unlock ai-ai.t_user
{
“result”: false,
“msg”: “can’t find task name from lock-ID”

| username: Billmay表妹 | Original post link

It is advisable to use the same version for DM and TiDB whenever possible, otherwise, various issues may easily arise.