DM Error: Restoring count greater than total count for table [table_name]

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: DM报错restoring count greater than total count for table[table_name]

| username: dockerfile

【TiDB Usage Environment】Production
【TiDB Version】V5.2.2
【DM Version】dmctl/v6.1.0
【Encountered Issue】Full dump load sync of multiple upstream RDS MySQL databases, one of the database tables reported an error
【Issue Phenomenon and Impact】Error reported during the dump stage

                    "stage": "Paused",
                    "unit": "Load",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "ErrCode": 24006,
                                "ErrClass": "checkpoint",
                                "ErrScope": "internal",
                                "ErrLevel": "medium",
                                "Message": "restoring count greater than total count for table[table name xxx]",
                                "RawCause": "",
                                "Workaround": ""
                            }
                        ],

The table structure of the reported error has been compared and is consistent with the table structure of other databases.

2
| username: guominghao | Original post link

Please provide the dm-worker logs.

3
| username: h5n1 | Original post link

It seems like a bug. Have you enabled relay?

4
| username: dockerfile | Original post link

The issue mentions version V2, but I am currently using V6.1.0.
How can I check the configuration of the relay log?

5
| username: h5n1 | Original post link

The relaylog configuration file shows that this issue is still open for different versions.

6
| username: dockerfile | Original post link 
[2022/07/05 08:33:31.332 +08:00] [ERROR] [loader.go:732] ["calc load process"] [task=task-name-xxx] [unit=load] [error="[code=24006:class=checkpoint:scope=internal:level=medium], Message: restoring count greater than total count for table[table-name-xxx]"]
[2022/07/05 08:33:31.332 +08:00] [INFO] [subtask.go:315] ["unit process returned"] [subtask=task-name-xxx] [unit=Load] [stage=Paused] [status="{\"progress\":\"100.00 %\",\"metaBinlog\":\"(mysql-bin.000663, 127800)\",\"metaBinlogGTID\":\"033599ea-7b6d-11ec-80d9-b8599f37f088:1-21693957,584da0f6-2b37-11ec-b023-0c42a1df22b4:1-51405854,8c9840f4-ebf4-11ea-ab97-506b4bfece4c:1-130374091,b790db32-d621-11ea-87fe-00163e32315a:1-43242323,bc662633-ebf4-11ea-b981-506b4bfe81cc:1\"}"]
[2022/07/05 08:33:31.332 +08:00] [INFO] [loader.go:809] ["stop importing data process"] [task=task-name-xxx] [unit=load]
[2022/07/05 08:33:31.332 +08:00] [ERROR] [subtask.go:336] ["unit process error"] [subtask=task-name-xxx] [unit=Load] ["error information"="ErrCode:24006 ErrClass:\"checkpoint\" ErrScope:\"internal\" ErrLevel:\"medium\" Message:\"restoring count greater than total count for table[table-name-xxx]\" "]
7
| username: dockerfile | Original post link

I checked the configuration of task and source, and neither has relay enabled.

8
| username: guominghao | Original post link

Please provide the complete log.

9
| username: dockerfile | Original post link

I don’t want to play anymore, can I leave?

10
| username: dockerfile | Original post link

After switching to V5.3.0 dmctl, stop-task was successful.

tiup dmctl --master-addr x.x.x.x:8261 stop-task task-xxxxx
Starting component `dmctl`: /root/.tiup/components/dmctl/v6.1.0/dmctl/dmctl --master-addr x.x.x.x:8261 stop-task task-xxxxx
{
    "op": "Delete",
    "result": false,
    "msg": "[code=38041:class=dm-master:scope=internal:level=medium], Message: invalid op 7 on task",
    "sources": [
    ]
}

# At this point, switch to /root/.tiup/components/dmctl/v5.3.0/dmctl
./dmctl --master-addr x.x.x.x:8261 stop-task task-xxxxx
{
    "op": "Stop",
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": true,
            "msg": "",
11
| username: buchuitoudegou | Original post link

Could you provide the complete log?

The “invalid op” issue might be caused by a version mismatch between dmctl and DM: dm v5.4.0执行stop-task 返回 invalid op 7 on task - TiDB 的问答社区

You can specify the dmctl version or upgrade dmctl to match the DM version.

12
| username: dockerfile | Original post link

It’s too troublesome to desensitize the complete log.

How to keep the DM version consistent with dmctl? I only know how to specify the version of dmctl. The version number of DM is not consistent with dmctl and TiDB. How to adapt?

13
| username: Min_Chen | Original post link

Check if there are escape characters in the string data of the original file for the error “restoring count greater than total count for table”. It is possible that escape characters split one row of data into two or even multiple rows.

Since DM 5.4 and later, the version number changes with the TiDB release version number. Refer to DM 5.3.0 Release Notes | PingCAP 文档中心

14
| username: system | Original post link

This topic will be automatically closed 60 days after the last reply. No new replies are allowed.