TiDB BR Restore Database Error

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb br restore数据库报错

| username: jaybing926

[TiDB Version] v4.0.9
[Encountered Problem]
I failed to restore the db folder from a full backup in the production environment to the test environment (which is single-node). Is it not possible to use it this way?

./bin/br restore full --pd 127.0.0.1:2379 --storage “/root/full-2022-06-21” --ratelimit 128 --log-file restore_db.log

Detail BR log in restore_db.log
Full restore <-------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00%
Error: rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader

Below is the restore_db.log, with many errors. I have excerpted a portion as follows:
[2022/06/23 06:33:34.531 +08:00] [WARN] [domain.go:389] [“loading schema takes a long time”] [“take time”=2m51.94061665s]
[2022/06/23 06:33:34.532 +08:00] [WARN] [domain.go:521] [“reload schema in loop, schema syncer need rewatch”]
[2022/06/23 06:33:34.532 +08:00] [INFO] [domain.go:527] [“reload schema in loop, schema syncer need restart”]
[2022/06/23 06:33:34.532 +08:00] [INFO] [syncer.go:255] [“[ddl] syncer watch global schema finished”]
[2022/06/23 06:33:34.532 +08:00] [INFO] [schema_validator.go:103] [“the schema validator stops”]
[2022/06/23 06:33:34.546 +08:00] [INFO] [domain.go:596] [“domain is closed”]
[2022/06/23 06:33:34.546 +08:00] [ERROR] [domain.go:537] [“reload schema in loop, schema syncer restart failed”] [error=“context canceled”] [errorVerbose=“context canceled
github.com/pingcap/errors.AddStack
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/errors.go:174
github.com/pingcap/errors.Trace
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/juju_adaptor.go:15
github.com/pingcap/tidb/owner.contextDone
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/owner/manager.go:380
github.com/pingcap/tidb/owner.NewSession
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/owner/manager.go:148
github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Restart
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/ddl/util/syncer.go:217
github.com/pingcap/tidb/domain.(*Domain).mustRestartSyncer
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/domain/domain.go:561
github.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/domain/domain.go:535
runtime.goexit
\truntime/asm_amd64.s:1357”] [stack=“github.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/domain/domain.go:537”]
[2022/06/23 06:33:34.582 +08:00] [INFO] [domain.go:527] [“reload schema in loop, schema syncer need restart”]
[2022/06/23 06:33:34.582 +08:00] [INFO] [schema_validator.go:103] [“the schema validator stops”]
[2022/06/23 06:33:34.582 +08:00] [INFO] [domain.go:596] [“domain is closed”]
[2022/06/23 06:33:34.582 +08:00] [ERROR] [domain.go:537] [“reload schema in loop, schema syncer restart failed”] [error=“context canceled”] [errorVerbose=“context canceled
github.com/pingcap/errors.AddStack
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/errors.go:174
github.com/pingcap/errors.Trace
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/juju_adaptor.go:15
github.com/pingcap/tidb/owner.contextDone
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/owner/manager.go:380
github.com/pingcap/tidb/owner.NewSession
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/owner/manager.go:148
github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Restart
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/ddl/util/syncer.go:217
github.com/pingcap/tidb/domain.(*Domain).mustRestartSyncer
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/domain/domain.go:561
github.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/domain/domain.go:535
runtime.goexit
\truntime/asm_amd64.s:1357”] [stack=“github.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/domain/domain.go:537”]
[2022/06/23 06:33:34.583 +08:00] [INFO] [schema_validator.go:134] [“the schema validator stopped before updating”]
[2022/06/23 06:33:34.583 +08:00] [INFO] [domain.go:527] [“reload schema in loop, schema syncer need restart”]
[2022/06/23 06:33:34.583 +08:00] [INFO] [schema_validator.go:103] [“the schema validator stops”]
[2022/06/23 06:33:34.583 +08:00] [INFO] [domain.go:596] [“domain is closed”]
[2022/06/23 06:33:34.583 +08:00] [ERROR] [domain.go:537] [“reload schema in loop, schema syncer restart failed”] [error=“context canceled”] [errorVerbose=“context canceled
github.com/pingcap/errors.AddStack
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/errors.go:174
github.com/pingcap/errors.Trace
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/juju_adaptor.go:15
github.com/pingcap/tidb/owner.contextDone
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/owner/manager.go:380
github.com/pingcap/tidb/owner.NewSession
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/owner/manager.go:148
github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Restart
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/ddl/util/syncer.go:217
github.com/pingcap/tidb/domain.(*Domain).mustRestartSyncer
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/domain/domain.go:561
github.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/domain/domain.go:535
runtime.goexit
\truntime/asm_amd64.s:1357”] [stack=“github.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop
\tgithub.com/pingcap/tidb@v1.1.0-beta.0.20201214152324-ce2f365189d3/domain/domain.go:537”]
[2022/06/23 06:33:34.583 +08:00] [INFO] [domain.go:622] [“domain closed”] [“take time”=2m11.515058829s]
[2022/06/23 06:33:34.584 +08:00] [INFO] [collector.go:188] [“Full restore Failed summary : total restore files: 0, total success: 0, total failed: 0”] [“split region”=56.846093612s] [“restore ranges”=676] [Size=7848762531]
[2022/06/23 06:33:34.584 +08:00] [ERROR] [restore.go:24] [“failed to restore”] [error=“rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader”] [errorVerbose=“rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader
github.com/tikv/pd/client.(*client).ScanRegions
\tgithub.com/tikv/pd@v1.1.0-beta.0.20201209075645-beb7635d13d2/client/client.go:594
github.com/pingcap/br/pkg/restore.(*pdClient).ScanRegions
\tgithub.com/pingcap/br@/pkg/restore/split_client.go:362
github.com/pingcap/br/pkg/restore.PaginateScanRegion
\tgithub.com/pingcap/br@/pkg/restore/util.go:489
github.com/pingcap/br/pkg/restore.(*RegionSplitter).Split
\tgithub.com/pingcap/br@/pkg/restore/split.go:102
github.com/pingcap/br/pkg/restore.SplitRanges
\tgithub.com/pingcap/br@/pkg/restore/util.go:432
github.com/pingcap/br/pkg/restore.(*tikvSender).splitWorker
\tgithub.com/pingcap/br@/pkg/restore/pipeline_items.go:234
runtime.goexit
\truntime/asm_amd64.s:1357”] [stack=“github.com/pingcap/br/cmd.runRestoreCommand
\tgithub.com/pingcap/br@/cmd/restore.go:24
github.com/pingcap/br/cmd.newFullRestoreCommand.func1
\tgithub.com/pingcap/br@/cmd/restore.go:93
github.com/spf13/cobra.(*Command).execute
\tgithub.com/spf13/cobra@v1.0.0/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
\tgithub.com/spf13/cobra@v1.0.0/command.go:950
github.com/spf13/cobra.(*Command).Execute
\tgithub.com/spf13/cobra@v1.0.0/command.go:887
main.main
\tgithub.com/pingcap/br@/main.go:58
runtime.main
\truntime/proc.go:203”]
[2022/06/23 06:33:34.584 +08:00] [INFO] [batcher.go:70] [“restore canceled, cleaning in background context”]
[2022/06/23 06:33:34.628 +08:00] [ERROR] [main.go:59] [“br failed”] [error=“rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader”] [errorVerbose=“rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader
github.com/tikv/pd/client.(*client).ScanRegions
\tgithub.com/tikv/pd@v1.1.0-beta.0.20201209075645-beb7635d13d2/client/client.go:594
github.com/pingcap/br/pkg/restore.(*pdClient).ScanRegions
\tgithub.com/pingcap/br@/pkg/restore/split_client.go:362
github.com/pingcap/br/pkg/restore.PaginateScanRegion
\tgithub.com/pingcap/br@/pkg/restore/util.go:489
github.com/pingcap/br/pkg/restore.(*RegionSplitter).Split
\tgithub.com/pingcap/br@/pkg/restore/split.go:102
github.com/pingcap/br/pkg/restore.SplitRanges
\tgithub.com/pingcap/br@/pkg/restore/util.go:432
github.com/pingcap/br/pkg/restore.(*tikvSender).splitWorker
\tgithub.com/pingcap/br@/pkg/restore/pipeline_items.go:234
runtime.goexit
\truntime/asm_amd64.s:1357”] [stack=“main.main
\tgithub.com/pingcap/br@/main.go:59
runtime.main
\truntime/proc.go:203”]

| username: yilong | Original post link

You can try expanding the cluster resources, reducing the concurrency during the restore, and enabling the ratelimit settings.

| username: jaybing926 | Original post link

Okay, thank you~

| username: system | Original post link

This topic was automatically closed 1 minute after the last reply. No new replies are allowed.