After Using Placement Rules in SQL, Scaling Out Will Not Balance Data

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 使用了Placement Rules in SQL 后,扩容不会balance数据

| username: 林夕一指

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version] V6.5.0
[Encountered Problem: Problem Description and Impact] After setting Placement Rules for an existing database, expanding the TiKV with the same label does not trigger region balance.

| username: caiyfc | Original post link

Please send the relevant settings and region monitoring.

| username: 林夕一指 | Original post link

Sorry, I can’t translate images. Please provide the text you need translated.

| username: caiyfc | Original post link

Could you please send the expansion file so I can take a look at the server.labels section?

| username: 林夕一指 | Original post link

10.1.38.32 has the same Placement Rules configured, but after scaling out, the regions and leaders stabilized and stopped balancing. This is a problem.

10.1.20.180 does not have Placement Rules configured and has almost no regions. I think this is not a problem at all.

| username: h5n1 | Original post link

Is 180 not configured with a Label?

| username: 林夕一指 | Original post link

180 is intentionally not configured, for comparison purposes.

| username: 林夕一指 | Original post link

Labels from pd-ctl and TiDB Dashboard can ensure consistency.

| username: h5n1 | Original post link

show placement, show placement labels

| username: 林夕一指 | Original post link

The image link you provided appears to be broken or inaccessible. Please provide the text you need translated.

| username: h5n1 | Original post link

Check the relevant scheduling content in the PD leader and this TiKV log.

| username: tidb菜鸟一只 | Original post link

Are these four TiKV machines configured the same?

| username: caiyfc | Original post link

When you really can’t find the problem, you can try restarting this TiKV. I encountered a similar situation before, and after restarting TiKV, the migration started.

| username: 林夕一指 | Original post link

From the log of this TiKV, it appears that regions are being migrated.

| username: 林夕一指 | Original post link

The disk handling capabilities are inconsistent, and the newly added 180 is slightly worse. However, the monitoring shows that the IO is not fully utilized.

| username: 林夕一指 | Original post link

I am guessing whether it could be related to the depth of the label.

| username: 裤衩儿飞上天 | Original post link

Check if there is a significant difference in the scores of several nodes.

| username: dba-kit | Original post link

By default, both add-peer and remove-peer are set to 50, which is very low. Therefore, the migration speed will be relatively slow.

| username: dba-kit | Original post link

You can take a look at this document and use pd-ctl to adjust several parameters for migrating regions.

| username: h5n1 | Original post link

Check the logs in PD to see if there is any scheduling information to store 32, such as add peer.