Does dumpling support rate limiting?

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dumpling是否支持限速

| username: Geoffrey

[TiDB Usage Environment] Production Environment
[TiDB Version] 3.0.12
[Encountered Problem: Problem Phenomenon and Impact]
Does dumpling support rate limiting? Currently, we want to migrate some tables from TiDB. There is a large table with 3.2 billion rows. We are afraid that directly migrating the entire table using dumpling might crash the cluster. Is there any way to avoid this?

| username: zhanggame1 | Original post link

  • -t is used to specify the number of threads for export. Increasing the number of threads will increase Dumpling’s concurrency and improve export speed, but it will also increase the database’s memory consumption, so it should not be set too high. It’s fine to set the value of the -t parameter a bit lower.
| username: zhanggame1 | Original post link

In older versions of TiDB, using Dumpling requires modifying the GC time. For example:
SET GLOBAL tidb_gc_life_time = ‘720h’;

| username: 我是咖啡哥 | Original post link

While others are thinking about how to speed up, you’re still limiting the speed :joy:
I checked the documentation, and it seems that it’s not supported.

| username: Geoffrey | Original post link

I use 1, the default seems to be 4.

| username: 像风一样的男子 | Original post link

Full migration can be considered using BR backup, which basically has no impact on production environment performance. TiDB 快照备份与恢复命令行手册 | PingCAP 文档中心

| username: tidb菜鸟一只 | Original post link

3.0, I didn’t look closely just now, I really haven’t used this one…

| username: redgame | Original post link

Add the parameters -r 200000 -F256MiB, you can try it.