Backup Error: TiKV Server Timeout and Context Deadline Exceeded

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 备份报错 TiKV server timeout和context deadline exceeded

| username: LBX流鼻血

[TiDB Usage Environment] Production Environment
[TiDB Version] 6.1.0
[Encountered Problem: Problem Phenomenon and Impact] Two types of errors occurred during backup
TiKV server timeout and context deadline exceeded
The parameter --kill-long-queries=360000 has already been added during the backup, so it shouldn’t timeout
Is there something wrong somewhere?

| username: ShawnYan | Original post link

What anomalies are there in the TiKV log?
Can you copy the text from the image?

| username: redgame | Original post link

We still need to start by investigating the timeout issue. The prompt indicates that network problems are causing communication timeouts with the TiKV server.

| username: TiDBer_vfJBUcxl | Original post link

You need to check the detailed log information; these two errors do not reveal the issue.

| username: 昵称想不起来了 | Original post link

  1. It is possible that the TiKV node is overloaded, causing a timeout. You can check the monitoring to confirm.
  2. It is possible that the network accessing the TiKV node is unstable or has insufficient bandwidth.
| username: hey-hoho | Original post link

First, check if there are any restarts of the TiKV nodes. You can determine this by looking at the instance start time in the Dashboard or the uptime in Grafana.

| username: 大飞哥online | Original post link

  1. The monitoring turntable information at that time
  2. Backup logs
  3. Node logs
    Take a look at all of them. With only this information, it’s hard to tell anything.