Backup failed, error during checksum: Error: other error: Coprocessor task terminated due to exceeding the deadline

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: br备份失败,checksum的时候报错 Error: other error: Coprocessor task terminated due to exceeding the deadline

| username: TiDBer_BwNZ5U9X

[TiDB Usage Environment] Testing
[TiDB Version] 6.5.0
[Reproduction Path] br backup, 1.2T data volume, failed after two hours
[Encountered Issue: Problem Phenomenon and Impact]

Using the br tool to write backups to shared storage, failed after 2 hours
tiup br backup full --pd 192.168.40.93:2379 -s /share/mountpoint/s_tidb/tidb-test/87B21CECE46F4974943567BBC53253DE/

br log
<-------------------------------------------------------------------------------> 99.71% Full Backup <------------------------------------------------------------------------------> 99.71% Full Backup <------------------------------------------------------------------------------|> 99.85% Full Backup <------------------------------------------------------------------------------/> 99.85% Full Backup <-------------------------------------------------------------------------------> 99.85% Full Backup <------------------------------------------------------------------------------> 99.85% Full Backup <------------------------------------------------------------------------------|> 99.85% Full Backup <------------------------------------------------------------------------------/> 99.85% Full Backup <------------------------------------------------------------------------------> 100.00% Checksum <…> 0.00% Checksum <…> 0.00% Checksum <…> 0.00% Checksum <…> 0.00% Checksum <–…> 1.46% Checksum ← .…> 1.46% Checksum <-|…> 1.46% Checksum <-/…> 1.46% Checksum <–…> 1.46% Checksum ← .…> 1.46% Checksum <-|…> 2.05% Checksum <-/…> 2.05% Checksum <–…> 2.05% Checksum ← .…> 2.05% Checksum <-|…> 2.05% Checksum .…> 2.05% Checksum <-|…> 2.05% [2024/06/11 20:19:10.909 +08:00] [INFO] [collector.go:73] [“Full Backup failed summary”] [total-ranges=21408] [ranges-succeed=21408] [ranges-failed=0] [backup-total-ranges=679]
Checksum <-/…> 2.05% Error: other error: Coprocessor task terminated due to exceeding the deadline

Testing Environment Information


br log.txt (581.3 KB)

| username: WalterWj | Original post link

Is the port open? It seems like calling the TiKV API to perform checksum calculation :thinking:: TiKV | Checksum
Then it timed out, maybe it’s not open.

| username: 像风一样的男子 | Original post link

Is the server configuration too low?

| username: 小于同学 | Original post link

How about upgrading the version?

| username: TiDBer_BwNZ5U9X | Original post link

Are you talking about upgrading the database version?

| username: ziptoam | Original post link

The business side will have a timeout error (Coprocessor task terminated due to exceeding the deadline). Usually, this situation is caused by an unoptimized execution plan, such as missing indexes, leading to a full table scan.