Dumpling Export of a Specific Table in MySQL Keeps Getting Stuck with No Response

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dumpling导出mysql中某张表一直卡住没反应

| username: putianhui

My table has 10003 rows and is 1680MB in size, and it keeps getting stuck at
[2022/09/02 10:32:54.290 +08:00] [INFO] [config.go:617] ["detect server version"] [version=5.7.39-log]

Below is my dumpling export statement

[root@k8s-node23 my_database]# tiup dumpling:v4.0.9 -h 192.168.xxx.xx -P 3306 -u xxx -p xxx -t 8 -r 200000 -F 256MB -T gdelt.news_webpages -o /data/my_database/
Starting component `dumpling`: /root/.tiup/components/dumpling/v4.0.9/dumpling -h 192.168.99.24 -P 3306 -u root -p CNdsdis123 -t 8 -r 200000 -F 256MB -T gdelt.news_webpages -o /data/my_database/
Release version: v4.0.9
Git commit hash: 11d8d5dad31210a1ec6afef9d3c16b397f2fc9fb
Git branch:      heads/refs/tags/v4.0.9
Build timestamp: 2020-12-19 04:53:02Z
Go version:      go version go1.13 linux/amd64

[2022/09/02 10:32:54.290 +08:00] [INFO] [config.go:598] ["detect server type"] [type=MySQL]
[2022/09/02 10:32:54.290 +08:00] [INFO] [config.go:617] ["detect server version"] [version=5.7.39-log]
# It keeps getting stuck here and there is no further information

Seeking help from everyone.

2
| username: putianhui | Original post link

Now, after more than 20 minutes, this message is displayed.

[2022/09/02 10:47:54.292 +08:00] [ERROR] [main.go:74] ["dump failed error stack info"] [error="sql: FLUSH TABLES WITH READ LOCK: invalid connection"] [errorVerbose="invalid connection\
sql: FLUSH TABLES WITH READ LOCK\
github.com/pingcap/dumpling/v4/export.FlushTableWithReadLock\
\tgithub.com/pingcap/dumpling@/v4/export/sql.go:398\
github.com/pingcap/dumpling/v4/export.(*ConsistencyFlushTableWithReadLock).Setup\
\tgithub.com/pingcap/dumpling@/v4/export/consistency.go:86\
github.com/pingcap/dumpling/v4/export.(*Dumper).Dump\
\tgithub.com/pingcap/dumpling@/v4/export/dump.go:104\
main.main\
\tgithub.com/pingcap/dumpling@/cmd/dumpling/main.go:71\
runtime.main\
\truntime/proc.go:203\
runtime.goexit\
\truntime/asm_amd64.s:1357"] [stack="main.main\
\tgithub.com/pingcap/dumpling@/cmd/dumpling/main.go:74\
runtime.main\
\truntime/proc.go:203"]

dump failed: sql: FLUSH TABLES WITH READ LOCK: invalid connection
3
| username: onlyacat | Original post link

Try running it again with --loglevel debug to see what happens?

4
| username: putianhui | Original post link

Even with debug mode enabled, it still gets stuck without any logs.

base/ --loglevel debug
Release version: v4.0.9
Git commit hash: 11d8d5dad31210a1ec6afef9d3c16b397f2fc9fb
Git branch:      heads/refs/tags/v4.0.9
Build timestamp: 2020-12-19 04:53:02Z
Go version:      go version go1.13 linux/amd64

[2022/09/02 12:07:24.154 +08:00] [DEBUG] [config.go:584] ["parse server info"] ["server info string"=5.7.39-log]
[2022/09/02 12:07:24.154 +08:00] [INFO] [config.go:598] ["detect server type"] [type=MySQL]
[2022/09/02 12:07:24.154 +08:00] [INFO] [config.go:617] ["detect server version"] [version=5.7.39-log]
5
| username: putianhui | Original post link

I switched to version 6.2.0, and now there is some information, but it still gets stuck and doesn’t move.

base/ --loglevel debug
Release version: v6.2.0
Git commit hash: daf2b17cdfe30c02ce282361009fb5bdb05f2b0e
Git branch:      heads/refs/tags/v6.2.0
Build timestamp: 2022-08-22 01:40:02Z
Go version:      go version go1.18.5 linux/amd64

[2022/09/02 12:28:10.616 +08:00] [INFO] [versions.go:54] ["Welcome to dumpling"] ["Release Version"=v6.2.0] ["Git Commit Hash"=daf2b17cdfe30c02ce282361009fb5bdb05f2b0e] ["Git Branch"=heads/refs/tags/v6.2.0] ["Build timestamp"="2022-08-22 01:40:02"] ["Go Version"="go version go1.18.5 linux/amd64"]
[2022/09/02 12:28:10.618 +08:00] [WARN] [version.go:299] ["select tidb_version() failed, will fallback to 'select version();'"] [error="Error 1046: No database selected"]
[2022/09/02 12:28:10.618 +08:00] [INFO] [version.go:397] ["detect server version"] [type=MySQL] [version=5.7.39-log]
[2022/09/02 12:28:10.620 +08:00] [INFO] [dump.go:127] ["begin to run Dump"] [conf="{\"s3\":{\"endpoint\":\"\",\"region\":\"\",\"storage-class\":\"\",\"sse\":\"\",\"sse-kms-key-id\":\"\",\"acl\":\"\",\"access-key\":\"\",\"secret-access-key\":\"\",\"provider\":\"\",\"force-path-style\":true,\"use-accelerate-endpoint\":false},\"gcs\":{\"endpoint\":\"\",\"storage-class\":\"\",\"predefined-acl\":\"\",\"credentials-file\":\"\"},\"azblob\":{\"endpoint\":\"\",\"account-name\":\"\",\"account-key\":\"\",\"access-tier\":\"\"},\"AllowCleartextPasswords\":false,\"SortByPk\":true,\"NoViews\":true,\"NoSequences\":true,\"NoHeader\":false,\"NoSchemas\":false,\"NoData\":false,\"CompleteInsert\":false,\"TransactionalConsistency\":true,\"EscapeBackslash\":true,\"DumpEmptyDatabase\":true,\"PosAfterConnect\":false,\"CompressType\":0,\"Host\":\"192.168.99.24\",\"Port\":3306,\"Threads\":8,\"User\":\"root\",\"Security\":{\"CAPath\":\"\",\"CertPath\":\"\",\"KeyPath\":\"\"},\"LogLevel\":\"debug\",\"LogFile\":\"\",\"LogFormat\":\"text\",\"OutputDirPath\":\"/data/my_database/\",\"StatusAddr\":\":8281\",\"Snapshot\":\"\",\"Consistency\":\"flush\",\"CsvNullValue\":\"\\\\N\",\"SQL\":\"\",\"CsvSeparator\":\",\",\"CsvDelimiter\":\"\\\"\",\"Databases\":[],\"Where\":\"\",\"FileType\":\"sql\",\"ServerInfo\":{\"ServerType\":1,\"ServerVersion\":\"5.7.39-log\",\"HasTiKV\":false},\"Rows\":200000,\"ReadTimeout\":900000000000,\"TiDBMemQuotaQuery\":0,\"FileSize\":268435456,\"StatementSize\":1000000,\"SessionParams\":{},\"Tables\":{\"gdelt\":[{\"Name\":\"news_webpages\",\"AvgRowLength\":0,\"Type\":0}]},\"CollationCompatible\":\"loose\"}"]
6
| username: gary | Original post link

What version is your cluster?
What version is MySQL?

7
| username: putianhui | Original post link

The TiDB cluster version is 4.0.9 and the MySQL version is 5.7.39.

I see an error [error="Error 1046: No database selected"] in the logs.

8
| username: onlyacat | Original post link

That doesn’t matter, because there is no tidb_version() in MySQL, so this error is reported.

The problem is that FTWRL and other locks timed out. When it gets stuck, go into the database and run show processlist to see if there is any blocking.

9
| username: gary | Original post link

Is the TiDB cluster version 4.0.9? It is recommended to use Dumpling version 4.0.9 and try again. Different versions of tools are prone to errors.