Expand 3 KV nodes: How to accelerate region data balancing

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 扩容3台kv节点。 如何加快region数据均衡

| username: TiDBer_BrIoQ0NO

【TiDB Usage Environment】Production Environment
【TiDB Version】v5.4.0
【Encountered Problem】KV disk space is insufficient, added 3 new KV instances. However, region migration is very slow.
Adjusted leader-schedule-limit to 400 and region-schedule-limit to 20480000, but the effect is not very noticeable.

2
| username: h5n1 | Original post link
  1. The production side also adds the following two:
    max-pending-peer-count
    max-snapshot-count
  2. The consumption side adds store limit
    pd-ctl store limit all xxxx
  3. Region migration and other operations need to produce snapshots and send them to the target TiKV over the network, which will have a significant impact on network IO and CPU.
3
| username: TiDBer_BrIoQ0NO | Original post link

The maximum value for store limit all seems to be 200. I just tried setting it to 1500 and got the message:
» store limit all 1500
rate should be less than 200.000000 for all

4
| username: h5n1 | Original post link

It prompted, so that’s it.

5
| username: TiDBer_BrIoQ0NO | Original post link

Okay, thank you. I’ll make some adjustments and observe.

6
| username: TiDBer_BrIoQ0NO | Original post link

Increasing the parameters max-pending-peer-count and max-snapshot-count, will they occupy the disk space of the old KV?

I am now adjusting the following parameters:
config set leader-schedule-limit 400
config set region-schedule-limit 20480000
config set max-pending-peer-count 64000
config set max-snapshot-count 64000
store limit all 200

The disk space of the old 3 KV instances is still increasing, and the remaining space is decreasing.

7
| username: h5n1 | Original post link

Generating snapshot files and receiving snapshots will both take up disk space. Your settings are too high.

8
| username: TiDBer_BrIoQ0NO | Original post link

What size would be more appropriate?

9
| username: system | Original post link

This topic will be automatically closed 60 days after the last reply. No new replies are allowed.