Node Scaling Down Too Slow

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 缩容节点太慢

| username: lizhuolin2020

[TiDB Usage Environment] Production Environment
[TiDB Version] v4.0.6
[Reproduction Path] Operations performed that led to the issue:

  1. Set the number of replicas to 1;
  2. Added 3 nodes on the same server;
  3. Modified variables related to migration efficiency;
  4. Decommissioned 6 original nodes.
    [Encountered Issue: Problem Phenomenon and Impact]
    Data transfer is extremely slow. The entire cluster has 2-3T of data, and only about 300G of data was transferred from the nodes to be decommissioned overnight.
    [Resource Configuration]
    image
    image1380×342 132 KB

    All on the same subnet, a total of 6 physical servers, 40C128G5T.
2
| username: WalterWj | Original post link

Try increasing the store limit.

3
| username: Kongdom | Original post link

You can refer to the SOP

4
| username: lizhuolin2020 | Original post link

Thank you very much, the speed has greatly improved after increasing it.

5
| username: WalterWj | Original post link

Remember to change it back. This can cause fluctuations in cluster performance.

6
| username: liuis | Original post link

Mark it.