Error Starting Migration Task When Migrating Data from Small Data Volume MySQL to TiDB

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 从小数据量 MySQL 迁移数据到 TiDB 启动迁移任务报错

| username: TiDBer_DVnkdzKV

【TiDB Usage Environment】Production\Test Environment\POC
【TiDB Version】5.4
【Encountered Issue】Unexpected termination error when using dmctl to synchronize data
【Reproduction Path】tiup dmctl --master-addr 172.21.0.5:8261 start-task --remove-meta task1.yaml
task1.yaml

image
image1262×998 101 KB

【Issue Phenomenon and Impact】
Error occurs when synchronizing around 100,000 rows
image
image1380×784 41.6 KB

Log error information
[2022/06/21 15:34:50.817 +08:00] [WARN] [writer_util.go:181] [“fail to dumping table(chunk), will revert some metrics and start a retry if possible”] [task=test] [unit=dump] [database=news] [table=totalnews] [“finished rows”=9510] [“finished size”=10625668] [error=“invalid connection”]
【Attachments】

  • Relevant logs, configuration files, Grafana monitoring (https://metricstool.pingcap.com/)
  • TiUP Cluster Display information
  • TiUP Cluster Edit config information
  • TiDB-Overview monitoring
  • Corresponding module’s Grafana monitoring (such as BR, TiDB-binlog, TiCDC, etc.)
  • Corresponding module logs (including logs one hour before and after the issue)

If the question is related to performance optimization or troubleshooting, please download the script and run it. Please select all and copy-paste the terminal output results for upload.

2
| username: TiDBer_DVnkdzKV | Original post link

I tested a smaller table, and the table with 50,000 records had no issues. However, for this table with 12 million records, the synchronization stops and reports an error after syncing around 100,000 records.

3
| username: ddhe9527 | Original post link

Judging by the error “invalid connection,” is there any kind of firewall between MySQL and TiDB?

4
| username: hey-hoho | Original post link

Please upload the complete task configuration file. It looks like the data dump is too large and has crashed MySQL.

5
| username: GreenGuan | Original post link

It should be an issue on the MySQL side. Increase the maximum idle connection duration for the MySQL user used for synchronization.

6
| username: system | Original post link

This topic will be automatically closed 60 days after the last reply. No new replies are allowed.