Error Exporting Data with dumpling

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dumplingd导出数据报错

| username: TiDBer_eIQDQW2z

When I use dumpling to export data, although it can be exported successfully, TiDB reports an error.
The dumpling version is as follows:
Release version: v7.0.0
Git commit hash: 7376954cd868dbc44fc3015c9ef89c53749339a7
Git branch: heads/refs/tags/v7.0.0
Build timestamp: 2023-03-29 01:22:42Z
Go version: go version go1.20.2 linux/amd64

The error message is as follows:
tidb_192.168.10.148_4000.log (4.5 KB)

| username: tidb菜鸟一只 | Original post link

Can data be queried with this?
Is this table a clustered table?

| username: Gin | Original post link

The column _tidb_rowid, which cannot be found in the error, is a hidden column in non-index-organized tables used for automatic sharding, essentially serving as the KV key value. You can check whether the number of rows in the backup data of the table sorting_center.applet_user matches the number of records in the database to ensure that no data is lost in the backup. Generally, backup tools do not need to back up the values of hidden columns, so this might be a product bug.

| username: TiDBer_eIQDQW2z | Original post link

Cannot find:

The table is defined as follows:
CREATE TABLE applet_user (
id bigint(20) NOT NULL AUTO_INCREMENT COMMENT ‘Primary Key ID’,
openid varchar(255) DEFAULT NULL COMMENT ‘User Unique Identifier’,
user_id bigint(20) DEFAULT NULL COMMENT ‘User Table ID’,
nick_name varchar(255) DEFAULT NULL COMMENT ‘Nickname’,
gender tinyint(4) DEFAULT NULL COMMENT ‘Gender (0-Unknown 1-Male 2-Female)’,
lang varchar(255) DEFAULT NULL COMMENT ‘Language’,
city varchar(255) DEFAULT NULL COMMENT ‘City’,
province varchar(255) DEFAULT NULL COMMENT ‘Province’,
country varchar(255) DEFAULT NULL COMMENT ‘Country’,
avatar_url varchar(255) DEFAULT NULL COMMENT ‘Avatar URL’,
phone varchar(255) DEFAULT NULL COMMENT ‘Phone Number’,
login_type tinyint(4) DEFAULT NULL COMMENT ‘Login Type (1-Bidding Mini Program)’,
creator varchar(255) DEFAULT NULL COMMENT ‘Creator’,
create_time datetime DEFAULT CURRENT_TIMESTAMP,
modify_time datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
deleted tinyint(4) NOT NULL DEFAULT ‘0’ COMMENT ‘Deleted (0-Normal 1-Deleted)’,
site_id bigint(20) DEFAULT NULL COMMENT ‘Sorting Center ID’,
PRIMARY KEY (id) /*T![clustered_index] CLUSTERED */,
KEY idx_create_time (create_time),
KEY idx_modify_time (modify_time)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin AUTO_INCREMENT=90001 COMMENT=‘Mini Program User Table’

| username: TiDBer_eIQDQW2z | Original post link

There is no problem with data export, and the backup data is okay.
The issue now is that TiDB has monitoring, and this error keeps causing alerts, which is quite annoying and obscures the normal errors.

| username: caiyfc | Original post link

From the table structure, this table is indeed a clustered table. Due to internal mechanisms, clustered tables do not have the hidden field _tidb_rowid, so there will be an error when querying. I remember that non-clustered tables use the _tidb_rowid field for batch export, so dumpling may not have corresponding optimizations for clustered tables.

| username: tidb菜鸟一只 | Original post link

This table is a clustered table, and the primary key is its key. It shouldn’t automatically generate the _tidb_rowid column. It seems like it’s really an issue with the dumpling tool. Is the dumpling tool also version 7.0?

| username: TiDBer_eIQDQW2z | Original post link

From the output information, dumpling is version 7.0:
image

| username: TiDBer_eIQDQW2z | Original post link

Can the official team solve this issue with Dumpling?

| username: xingzhenxiang | Original post link

I feel that the first few minor versions of major releases should not be used in production. When it comes to MySQL, it’s only after version x.10 that it goes into production.

| username: TiDBer_eIQDQW2z | Original post link

Hello, can any official personnel respond and help solve this problem? I have raised an issue on GitHub, but no one has replied yet. :joy:

| username: 孤君888 | Original post link

Are dumpling and TIDB versions consistent?

| username: TiDBer_eIQDQW2z | Original post link

No, TiDB is 6.5.1 and Dumpling is 7.0.0. Should I try using consistent versions?

| username: TiDBer_eIQDQW2z | Original post link

I’m currently using version 7.0.0, but the latest version on GitHub is 5.1.5. This…

| username: weixiaobing | Original post link

The dumping of v6.5.1 is in the downloaded tools package.
You can directly download the installation package: wget https://download.pingcap.org/tidb-community-toolkit-v6.5.1-linux-amd64.tar.gz

| username: TiDBer_eIQDQW2z | Original post link

Currently, both TiDB and Dumpling versions are 6.5.1, but errors are still occurring.

| username: yilong | Original post link

What is the export command? I tested it and it can be exported normally.

CREATE TABLE applet_user (
id bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'Primary Key ID',
openid varchar(255) DEFAULT NULL COMMENT 'User Unique Identifier',
user_id bigint(20) DEFAULT NULL COMMENT 'User Table ID',
nick_name varchar(255) DEFAULT NULL COMMENT 'Nickname',
gender tinyint(4) DEFAULT NULL COMMENT 'Gender (0-Unknown 1-Male 2-Female)',
lang varchar(255) DEFAULT NULL COMMENT 'Language',
city varchar(255) DEFAULT NULL COMMENT 'City',
province varchar(255) DEFAULT NULL COMMENT 'Province',
country varchar(255) DEFAULT NULL COMMENT 'Country',
avatar_url varchar(255) DEFAULT NULL COMMENT 'Avatar URL',
phone varchar(255) DEFAULT NULL COMMENT 'Phone Number',
login_type tinyint(4) DEFAULT NULL COMMENT 'Login Type (1-Bidding Mini Program)',
creator varchar(255) DEFAULT NULL COMMENT 'Creator',
create_time datetime DEFAULT CURRENT_TIMESTAMP,
modify_time datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
deleted tinyint(4) NOT NULL DEFAULT '0' COMMENT 'Is Deleted (0-Normal 1-Deleted)',
site_id bigint(20) DEFAULT NULL COMMENT 'Sorting Center ID',
PRIMARY KEY (id) /*T![clustered_index] CLUSTERED */,
KEY idx_create_time (create_time),
KEY idx_modify_time (modify_time)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin AUTO_INCREMENT=90001 COMMENT='Mini Program User Table';

insert into applet_user values(1,'a',123,'sdf',0,'sdfasdfasdf','abc','asd','asdf','dsfasdf','123213',1,'dfasdf',now(),now(),1,3242);

tiup dumpling:v6.5.1 -u root -P 4913 -h 10.x.x.3 --filetype sql -t 8 -o /home/tidb/dumpling/ -r 200000 -F256MiB