Error while importing CSV file to TiDB: Lost connection ERROR 2013 (HY000): Lost connection to MySQL server during query

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 向tidb导入csv文件期间，报错：连接丢失 ERROR 2013 (HY000): Lost connection to MySQL server during query

| username: TiDBer_aNVhUipd

[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version]
[Encountered Problem]
I want to import a CSV file into TiDB. After running the command for about 20 minutes, an error occurred: connection lost

ERROR 2013 (HY000): Lost connection to MySQL server during query
[Reproduction Path] What operations were performed to encounter the problem
[Problem Phenomenon and Impact]

[Attachment]

image
image1229×152 107 KB

Please provide the version information of each component, such as cdc/tikv, which can be obtained by executing cdc version/tikv-server --version.

2
| username: TiDBer_aNVhUipd | Original post link

The CSV file I need to import is 9GB, which is very large, so the import time is very long. Does TiDB have any query time limits that cause the connection to be disconnected after a certain period of time? Thank you.

3
| username: h5n1 | Original post link

It might be an OOM issue. Check TiDB memory monitoring and uptime. When performing batch imports, you can enable batch_dml for small batch submissions to control memory usage.

4
| username: wuxiangdong | Original post link

Try increasing the max_allowed_packet parameter.

5
| username: TiDBer_aNVhUipd | Original post link

Thank you, it was indeed an OOM issue. I found it by checking the server’s process logs.

6
| username: TiDBer_aNVhUipd | Original post link

Okay. I want to set it to 9G, but after restarting TiDB, I found that it only set to 1G. Perhaps the maximum value for this parameter in TiDB is 1G.

7
| username: forever | Original post link

You can manually split the file and then import it. Parallel import is also possible.

8
| username: TiDBer_aNVhUipd | Original post link

Thank you all for your answers. I have confirmed that it is an OOM issue by checking the memory logs. I increased the virtual machine memory from 16G to 32G, but it only delayed the OOM occurrence; it still eventually ran out of memory, and the import speed remains very slow. Perhaps it’s because TiDB has very high hardware requirements. My machine does not use SSDs.

9
| username: hey-hoho | Original post link

You need this article:

10
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.