Full Recovery Failure

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 全量恢复失败

| username: TiDBer_sX3j5LdU

[TiDB Usage Environment] Production Environment
[TiDB Version] 6.1.3
[Reproduction Path] Operations performed that led to the issue
[Encountered Problem: Problem Phenomenon and Impact]
[Resource Configuration] Enter TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]
Primary: 230 231 232
Backup Cluster: 234 235 236
Previously, the backup cluster performed a full backup and restore operation. Yesterday, the primary cluster did another full backup, and the backup cluster restore failed. Please see the screenshot for the error.
My personal requirement is: full backup on Sunday, incremental backup from Monday to Saturday. Using the backed-up snapshots, the backup cluster will perform the same restore operation daily. Full restore on Sunday, incremental restore from Monday to Saturday.



Backup Log.log (5.4 MB)
Restore Log.log (14.3 MB)

| username: tidb菜鸟一只 | Original post link

Grep the logs for “restore checksum mismatch” to see which tables failed to restore successfully.

| username: zhanggame1 | Original post link

First, take a look at the backup logs; it might be that the backup was not successful.

| username: redgame | Original post link

Well, it’s very likely that there is an issue with the backup.

| username: TiDBer_vfJBUcxl | Original post link

Backup failed

| username: 大飞哥online | Original post link

First, take a look at the backup logs to see if there is any abnormal information.

| username: TiDBer_sX3j5LdU | Original post link

The image is not visible. Please provide the text you need translated.

| username: TiDBer_sX3j5LdU | Original post link

I think the main reason is that the default value of the tidb_distsql_scan_concurrency parameter is 15. When the concurrency is high, the CPU usage will be high. You can try to reduce the value of this parameter to see if it helps.

| username: TiDBer_sX3j5LdU | Original post link

The images you provided are not visible. Please provide the text content that you need translated.

| username: TiDBer_sX3j5LdU | Original post link

No anomalies found.

| username: 像风一样的男子 | Original post link

Is there an error like “failed in validate checksum” in the recovery logs? Is the target database for recovery an empty database, and is there any other data being written during the recovery process?

| username: TiDBer_sX3j5LdU | Original post link

You can take a look at the recently uploaded log file. Since it is a snapshot backup, I understand that it is unrelated to whether there were writes at the time or if the database was empty. In fact, there was no empty database.

| username: 像风一样的男子 | Original post link

The official documentation states that the target database must be empty.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.