After completing V6.5.3 BR backup and PITR recovery for NONCLUSTERED tables without primary keys or unique indexes, inserting data triggers ERROR 1062 (23000): Duplicate entry 'xxx' for key 'xxx.PRIMARY'

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: V6.5.3 BR备份、PITR恢复无主键、唯一索引的NONCLUSTERED表完成后,插入数据触发ERROR 1062 (23000): Duplicate entry ‘xxx’ for key ‘xxx.PRIMARY’

| username: IanWong

Bug Report
Clearly and accurately describe the issue you found. Providing any steps to reproduce the issue can help the development team address it promptly.
[TiDB Version] V6.5.3
[Bug Impact] Error when inserting new data
mysql> insert into aaa values(9000,now());
ERROR 1062 (23000): Duplicate entry ‘30001’ for key ‘aaa.PRIMARY’

[Possible Steps to Reproduce the Issue]
BR backup and restore of a NONCLUSTERED table without primary key or unique index, incorrect setting of hidden primary key _tidb_rowid causing errors when inserting new data.

  1. Original table data
    mysql> show create table aaa;
    ±------±-------------+
    | Table | Create Table |
    ±------±--------------+
    | aaa | CREATE TABLE aaa (
    id int(11) DEFAULT NULL,
    name varchar(255) DEFAULT NULL
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
    ±------±---------------+
    1 row in set (0.18 sec)

mysql> select *,_tidb_rowid from aaa;
±-----±--------------------±------------+
| id | name | _tidb_rowid |
±-----±--------------------±------------+
| 1 | 1 | 1 |
| 2 | 1 | 2 |
| 3 | 1 | 3 |
| 4 | 4 | 30001 |
| 5 | 5 | 30002 |
| 6 | 2024-03-18 11:07:54 | 30003 |
| 7 | 2024-03-18 11:08:05 | 30004 |
| 88 | 2024-03-18 11:11:58 | 30005 |
| 89 | 2024-03-18 11:12:03 | 30006 |
| 890 | 2024-03-18 11:17:20 | 30007 |
| 891 | 2024-03-18 11:17:24 | 30008 |
| 892 | 2024-03-18 11:17:27 | 30009 |
| 892 | 2024-03-18 11:22:28 | 30010 |
±-----±--------------------±------------+
13 rows in set (0.01 sec)

mysql> show table aaa next_row_id\G
*************************** 1. row ***************************
DB_NAME: test
TABLE_NAME: aaa
COLUMN_NAME: _tidb_rowid
NEXT_GLOBAL_ROW_ID: 60001
ID_TYPE: _TIDB_ROWID
1 row in set (0.00 sec)
2. Perform BR:V6.5.3 full backup
3. Perform BR restore
4. Query data after restoring BR database or table
mysql> select *,_tidb_rowid from aaa;
±-----±--------------------±------------+
| id | name | _tidb_rowid |
±-----±--------------------±------------+
| 1 | 1 | 1 |
| 2 | 1 | 2 |
| 3 | 1 | 3 |
| 4 | 4 | 30001 |
| 5 | 5 | 30002 |
| 6 | 2024-03-18 11:07:54 | 30003 |
| 7 | 2024-03-18 11:08:05 | 30004 |
| 88 | 2024-03-18 11:11:58 | 30005 |
| 89 | 2024-03-18 11:12:03 | 30006 |
| 890 | 2024-03-18 11:17:20 | 30007 |
| 891 | 2024-03-18 11:17:24 | 30008 |
| 892 | 2024-03-18 11:17:27 | 30009 |
| 892 | 2024-03-18 11:22:28 | 30010 |
±-----±--------------------±------------+
13 rows in set (0.00 sec)

mysql> show table aaa next_row_id\G
*************************** 1. row ***************************
DB_NAME: test
TABLE_NAME: aaa
COLUMN_NAME: _tidb_rowid
NEXT_GLOBAL_ROW_ID: 30001
ID_TYPE: _TIDB_ROWID
1 row in set (0.00 sec)

[Observed Unexpected Behavior]



_tidb_rowid greater than the maximum value of the original table, subsequent insertions succeed;

[Expected Behavior]
Normal data insertion and _tidb_rowid advancement after BR backup and restore

[Related Components and Specific Versions]
TIDB: V6.5.3
BR: V6.5.3

[Other Background Information or Screenshots]


Tested BR: V6.5.8, same result after restore
Such as cluster topology, system and kernel version, application app information, etc.; if the issue is related to SQL, please provide SQL statements and related table schema information; if there are critical errors in node logs, please provide relevant node log content or files; if some business-sensitive information is inconvenient to provide, please leave contact information for private communication.

| username: DBAER | Original post link

Mark it.

| username: tidb菜鸟一只 | Original post link

I’ll test it when I have time, it shouldn’t be like this.

| username: Daniel-W | Original post link

BR, there’s also this issue. I’ll give it a try.

| username: IanWong | Original post link

It is speculated that the trigger condition is related to the hidden primary key of my original table being skipped. Specifically, it depends on the logic for setting the initial value of _tidb_rowid after br backup and restore.

| username: WalterWj | Original post link

You can try the same operation on a newer version and see how it performs. Also, did br exit correctly?

| username: aytrack | Original post link

Is this restoring to the original cluster or to a new cluster? Specifically, is the operation first br backup full for a full backup, and then using br restore db or br restore table? What are the three steps of the recovery process?

  1. Perform a full backup with BR:V6.5.3
  2. Perform BR restore
  3. Query data based on the restored database or table
| username: IanWong | Original post link

Restored to the original cluster, used br restore point in the middle, it is speculated that the br restore point time point overwrote it, now it cannot be reproduced.