Dumpling Export Stuck, High Memory Usage

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dumpling导出卡住，内存使用高

| username: 像风一样的男子

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path] When exporting data from a table using dumpling, with a data volume of approximately 170 million, TiDB’s memory usage reaches 52GB (80%) and then gets stuck, causing subsequent exports to be very slow.

image
image1511×344 20.7 KB

2
| username: 像风一样的男子 | Original post link

Is it normal to export 5 million records in half an hour?

3
| username: TiDBer_3Cusx9uk | Original post link

We need to check if there are any error logs. At present, it should be an issue with the database itself. Looking forward to an official explanation.

4
| username: 像风一样的男子 | Original post link

Increasing memory solves it.

5
| username: kkpeter | Original post link

Obviously not normal.

6
| username: 舞动梦灵 | Original post link

Wow. This is such a waste of memory. 185G of memory. If you use the dumpling tool, the memory must keep up.

7
| username: TiDBer_3Cusx9uk | Original post link

It shouldn’t be a matter of adding more memory; the memory consumption is too high.

8
| username: 濱崎悟空 | Original post link

This speed is too slow.

9
| username: zhaokede | Original post link

Memory size is king.

10
| username: 突破边界 | Original post link

It’s a bit exaggerated to require such a large memory for exporting.