Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: 使用 scale-in 缩容 TiKV 时可以同时缩容好几个 TiKV 吗
[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version]
[Reproduction Path] What operations were performed that caused the issue
[Encountered Issue: Problem Phenomenon and Impact]
Currently, there is a cluster version 5.1.x with dozens of TiKV, but the business data is relatively small, and we need to scale down by 10 TiKV.
I would like to ask if it is possible to execute the scale-down like this:
tiup cluster scale-in xxxx_cluster -N 1.1.1.1:20160,2.2.2.2:20160,3.3.3.3:20160…
Or is it recommended to scale down one by one as shown below:
tiup cluster scale-in xxxx_cluster -N 1.1.1.1:20160
tiup cluster scale-in xxxx_cluster -N 2.2.2.2:20160
tiup cluster scale-in xxxx_cluster -N 3.3.3.3:20160
I understand that even if multiple TiKV are specified for scale-down in tiup, tiup internally scales down one TiKV at a time, otherwise the cluster would malfunction.
However, I am not sure if there are any unexpected issues that might arise when specifying multiple TiKV for scale-down at the same time?
Adding a question, is it possible to scale up in batches, for example, can a cluster scale up by 6 TiKV at once?
Thanks
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]
It is possible to scale down multiple TiKV nodes simultaneously, but it is not recommended. It is advisable to scale down one by one.
Normally, it is safer to scale down sequentially, allowing for manual judgment of whether the scaling down is normal. Batch scaling increases the risk.
It is possible but not recommended. Concurrent scaling down will not be faster than sequential scaling down because scheduling will be more frequent, which may also affect TiDB’s response time.
Batch scaling down carries significant risks, primarily because the I/O cannot handle it.
Not recommended to do this, the risk is too high.
It’s better to be cautious in the production environment, but you can do whatever you want in the testing environment.
May I ask if it is possible to scale out in batches? For example, can a cluster be expanded by 10 TiKV nodes at once? Thanks.
There is no problem with this.
Because scaling down requires adding some replica data, it needs some write I/O. And this write I/O pressure will be given to the remaining machines that have not been scaled down.
However, when scaling up, the added replica initially joins the original raft group as a learner role, and the new write I/O will not be given to the original machines. Once the learner role data is completed, it will be converted to leader and the extra replicas will be deleted. The leader switch will be completed.
It is also supported; you can expand multiple at once.
It is possible, but not recommended.
The risk of batch expansion is a bit high; it’s better to expand gradually and observe for any anomalies.
I suggest tackling them one by one.
For cloud TiDB, scaling down TiKV is done automatically one by one. During the process of scaling down a store, the leaders and regions within the store will be balanced to other TiKVs. Once the store’s status changes to tombstone, another TiKV will be automatically scaled down.
It’s better to take it one step at a time.
It is not recommended to scale down multiple instances at once, as it can lead to multiple regions being scheduled simultaneously, which can affect performance.
It can be done, but it may affect performance. I am also planning to scale down from 30 kv nodes to 20. I intend to scale down one by one. If I scale down 10 at once, I am afraid it will affect the business. Moreover, if it does affect, I am not sure if interrupting the scaling down process will cause any issues.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
|
username: TiDBer_KkruFifg
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: