There are 3 TiKV nodes with 0 leaders

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 3个tikv上有个leader数量为0

| username: zhanggame1

[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version] 7.5
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
Today, I checked the monitoring and found that 1 out of 3 TiKV nodes has no leader. Let’s see what the problem is.

image
image1185×517 32.1 KB

image
image1157×486 31.2 KB

2
| username: 小龙虾爱大龙虾 | Original post link

Take a look at the PD => scheduler panel.

3
| username: zhanggame1 | Original post link

Sorry, I cannot translate images. Please provide the text you need translated.

4
| username: 小龙虾爱大龙虾 | Original post link

This is not correct. There is an evict-leader-scheduler. You can remove this scheduler using pd-ctl.

5
| username: 小龙虾爱大龙虾 | Original post link

I don’t know what those two schedulers with witnesses are for. Were they added in a higher version? Waiting for an expert to answer.

6
| username: 路在何chu | Original post link

Have you experienced a node crash?

7
| username: zhanggame1 | Original post link

There were 2 TiKV nodes that went down abnormally before, how can this be recovered?

8
| username: zhanggame1 | Original post link

How to operate specifically?

9
| username: 小龙虾爱大龙虾 | Original post link
  1. Enter pd-ctl interactive mode
tiup ctl:v6.5.5 pd -u http://192.168.1.1:2381 -i
  1. Check the current scheduler status
scheduler show
  1. Remove the scheduler with the evict-leader-scheduler keyword
scheduler remove <scheduler with evict-leader keyword from the previous command result>
10
| username: zhanggame1 | Original post link

Back to normal.

11
| username: dba远航 | Original post link

Learned.

12
| username: oceanzhang | Original post link

That means it has been scheduled. If there are few regions, I think it’s fine. It will allocate itself later.

13
| username: dockerfile | Original post link

The leader is 0, but there is a region?
Restart this TiKV, and the leader will come back instantly.

14
| username: 小于同学 | Original post link

Take a look at the scheduler panel.

15
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.