In a TiKV cluster (without TiDB) with over 400,000 keys, BR backup succeeds but cannot restore to a new TiKV cluster; invalid restore range

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: Tikv集群（无tidb）中key超过40万，br备份成功但无法恢复到新的tikv集群；invalid restore range

| username: Anthony99

[TiDB Usage Environment] Testing;
Using br txn for backup and restore, storage location is cephrgw bucket
[TiDB Version] 6.5.3

[Reproduction Path] Operations performed that led to the issue

  1. Created two TiKV clusters

  2. JuiceFS uses one of the TiKV clusters as the metadata engine and creates 400,000 empty files
    image

  3. Successfully backed up using the br tool

    image
    image2191×128 14.3 KB

  4. Error when restoring to a new cluster using the br tool
    Error: startKey > endKey, startKey: 6a66733031fd4174ff3804000000000049ff0000000000000000f7, endkey: 0000000000000000f7: [BR:Restore:ErrRestoreInvalidRange] invalid restore range

  5. Logs from the br tool

    image
    image1380×520 952 KB

[Encountered Issue: Problem Phenomenon and Impact]
Unable to restore to the new cluster. If anyone has encountered a similar issue, please provide some insights. Thank you.

[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

2
| username: xfworld | Original post link

I replied to you yesterday :rofl:

3
| username: redgame | Original post link

Did it happen again?

4
| username: Anthony99 | Original post link

Uh, you said yesterday that rxn is not supported? It can be successfully restored when the data is less than 400,000. :flushed:

5
| username: Anthony99 | Original post link

This issue is easy to reproduce, but it’s my first time posting.

6
| username: xfworld | Original post link

It’s normal for TiDB to handle several terabytes of data.

If it can be restored today but not tomorrow, the DBA would probably cry…

From the logs you posted, it seems that the region partition range is incorrect. BR will perform a checksum, and the checksum failed.

7
| username: Anthony99 | Original post link

Sure, without deploying TiDB, only TiKV and PD are deployed; you mentioned that the region partition range is incorrect, how can this be modified?

8
| username: xfworld | Original post link

This cannot be modified; the region range is allocated by PD and is also part of the metadata.

If you are using JuiceFS, just follow the documentation provided by JuiceFS.

9
| username: h5n1 | Original post link

Using tikv-br for backup and restore in a rawkv cluster

image
image1135×170 11 KB

Or use br backup raw
raw (experimental) backup a raw kv range from TiKV cluster

Refer to this article

10
| username: Anthony99 | Original post link

Okay, I’m planning to temporarily give up on the BR tool for now, thank you.

11
| username: Anthony99 | Original post link

Thank you, I’ll temporarily give up on the br tool and try raw another day.

12
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.