Why are there so many empty regions when restoring BR backup data to a new cluster, which were not present originally? What is the principle behind this?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: BR备份恢复数据到新集群中出现了非常多的empty-region原来没有这是啥原理？

| username: jaybing926

[TiDB Usage Environment] Production Environment
[TiDB Version] v4.0.9
[Reproduction Path] Operations that led to the issue:

  1. Performed a full backup of several databases using BR.
  2. Then restored the data to a newly deployed TiDB cluster using BR.
    [Encountered Issue: Symptoms and Impact]
    The new cluster has a large number of empty regions, causing severe data distribution imbalance. However, the original cluster had only a few empty regions. I would like to understand the principle behind this. Thanks~

[Attachments: Screenshots/Logs/Monitoring]

image

image
image1359×654 79.4 KB

2
| username: 普罗米修斯 | Original post link

Has the import finished? If so, increase the merge-schedule-limit parameter and observe.

3
| username: jaybing926 | Original post link

It’s not over yet. I tried it once before, and after importing, I enabled the merge configuration. In my version, it is disabled by default. Once enabled, the data will be balanced.
set config pd schedule.enable-cross-table-merge = ‘true’;

I don’t understand why importing data would cause this. What is the principle behind it? Because my original cluster didn’t have these empty regions.

4
| username: tidb狂热爱好者 | Original post link

Importing also creates a new region, it does not import a region.

5
| username: tidb狂热爱好者 | Original post link

Created too many empty ones waiting for them to merge.

6
| username: jaybing926 | Original post link

I still don’t understand why there are so many empty regions. The original cluster didn’t have them. If the original cluster initially had empty regions, they would have been merged already. Could it be that after importing data to the new cluster, it reverted to the original state and needs to be merged again?

7
| username: xfworld | Original post link

set config pd schedule.enable-cross-table-merge = ‘true’;

If this is not enabled, during import, data will be written to different regions based on the probability of data being logically dispersed, and it will not merge.

Key points:
Each interval of data writing will open a new region, load a segment of data, and then the interval will also construct continuous regions to ensure that data can be written normally. Therefore, there will be many empty regions (what you encountered should be an extreme case… :upside_down_face:)

image
image893×798 48 KB

8
| username: jaybing926 | Original post link

Okay, thank you, boss.
I’ll first look at the basic content. There are too many things to do every day, and I haven’t had the time to sit down and look at TiDB. :tired_face::tired_face::tired_face::tired_face:

9
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.