TiDB insert on duplicate key or replace into reports primary key conflict exception

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: Tidb insert on duplicate key 或 replace into 报主键冲突异常

| username: juecong

[TiDB Usage Environment] Production Environment
[TiDB Version]
7.1.1
[Reproduction Path]
When the program writes in multiple threads, it reports Duplicate entry ‘81432804’ for key ‘change_records.PRIMARY’, but the program controls that the same primary key will only be updated once within a few seconds.
[Encountered Problem: Problem Phenomenon and Impact]
The table that reports the primary key conflict does not have a primary key set, it is a unique key. Copying the SQL and executing it manually can insert and execute normally.

Table structure:
CREATE TABLE change_records (
id bigint(20) NOT NULL COMMENT ‘Serial number’,
u_id char(32) NOT NULL DEFAULT ‘’ COMMENT ‘Unique key’,
eid char(36) NOT NULL COMMENT ‘eid’,
seq_no int(11) NOT NULL COMMENT ‘Serial number’,
after_content mediumtext DEFAULT NULL COMMENT ‘Content after change’,
before_content mediumtext DEFAULT NULL COMMENT ‘Content before change’,
change_date varchar(255) NOT NULL DEFAULT ‘’ COMMENT ‘Change date’,
change_item varchar(2000) NOT NULL DEFAULT ‘’ COMMENT ‘Change name’,
update_time datetime(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6) COMMENT ‘Update time’,
UNIQUE KEY eid_u_id (eid,u_id),
KEY key_row_update_time (row_update_time),
KEY eid_u_tags (eid,u_tags),
KEY key_type (type)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin /*T! SHARD_ROW_ID_BITS=8 */ ;

Insert statement:
INSERT INTO change_records (seq_no, special_type, eid, update_time, change_date, source, type, u_id, id, before_content, after_content, change_item)
VALUES (-1, 0, ‘0c6c80b6-dce3-422b-9c1c’, ‘2024-04-16 01:54:51’, ‘2015-05-15’, ‘1’, ‘Personnel change’, ‘d1d75004c7261a85c09c66375082d990’, 7085492, ‘Zhang San’, ‘Li Si’, ‘Personnel’)
ON DUPLICATE KEY UPDATE seq_no = VALUES(seq_no), special_type = VALUES(special_type), eid = VALUES(eid), update_time = VALUES(update_time), change_date = VALUES(change_date), source = VALUES(source), type = VALUES(type), u_id = VALUES(u_id), id = VALUES(id), before_content = VALUES(before_content), after_content = VALUES(after_content), change_item = VALUES(change_item);

Previously using REPLACE INTO also reported the same primary key conflict.

| username: 小龙虾爱大龙虾 | Original post link

Has this table undergone any BR or Lightning restore operations? Please execute show table change_records next_row_id; and check the result.

| username: TiDBer_JUi6UvZm | Original post link

Is it just this one record conflict? Or are there many others?

| username: zhaokede | Original post link

How is the value of UNIQUE obtained, is it generated randomly or?

| username: juecong | Original post link

The result obtained

| username: juecong | Original post link

UUID + value generated according to rules

| username: juecong | Original post link

There are many conflicts, and after checking the database, this data does not exist.

| username: juecong | Original post link

Yes, it seems that I have done the lightning recovery operation before.

| username: juecong | Original post link

Moreover, some other tables are the same, all without primary keys.

| username: juecong | Original post link

Our initial full data was imported through Lightning.

| username: Jasper | Original post link

The primary key of a non-clustered table is based on _tidb_rowid. You can use this SQL query to check which data is conflicting:

select _tidb_rowid, change_records.* from change_records where _tidb_rowid = 81432804;

It might be due to an anomaly during the lightning import causing the rowid to be abnormally occupied.

| username: juecong | Original post link

Is there a solution for this? The problem now is that most tables have such operations, and several tables have the same primary key conflict.

| username: Jasper | Original post link

First, run the SQL query above to check if this rowid is already in use.

The simplest solution is to perform a rolling restart of the tidb-server, which will refresh the cached _tidb_rowid. This will most likely resolve the issue.

If the problem persists, you will have to manually set it for each table.

| username: 小龙虾爱大龙虾 | Original post link

I suspect there were some issues with the previous recovery, not sure what exactly. Currently, it seems that the hidden _tidb_rowid is conflicting. The solution now is to rebase, but you’ll need to calculate the rebase value.

| username: juecong | Original post link

Yes, I checked the other statement, and the data conflicting with _tidb_rowid is inconsistent. But the strange thing is that the statement that reported the error can be executed successfully manually. Can it be understood that when TiDB is under heavy load, _tidb_rowid didn’t advance in time? When executed manually, it had already moved forward.

| username: 小龙虾爱大龙虾 | Original post link

That’s not the reason. Each TiDB server has its own cached segment, and once an ID is used, it moves on to the next one. So inserting again might not cause a conflict.

| username: Jasper | Original post link

The principle is as follows: When your program executes, it has already attempted the conflicting _tidb_rowid, so the next execution will use the next _tidb_rowid.

For example:
If the data with _tidb_rowid 10 already exists, your program will first attempt to insert _tidb_rowid 10 and fail. When you manually execute it again, it will attempt to insert data with _tidb_rowid 11, and it will succeed.

You can also verify this by backtracking the _tidb_rowid of the successfully inserted data to see if it is different from the previously conflicting _tidb_rowid.

| username: Jasper | Original post link

I didn’t understand what you’re asking… The _tidb_rowid is duplicated.

| username: juecong | Original post link

Besides restarting kv, what command can be used to manually update the cache?

| username: Jasper | Original post link

It’s not highly recommended to do this, as you need to set it for each table, and you have to set a larger value based on the current _tidb_rowid maximum value for each table.

Additionally, it’s about restarting the tidb-server, not restarting kv.