Why does one of the three TiDB storage nodes have no leader?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb三个存储节点,其中一个节点没有leader,请问是什么原因?

| username: TiDBer_FMWXa7ja

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path]
[Encountered Problem: Problem Phenomenon and Impact] One of the three TiDB storage nodes does not have a leader. What could be the reason?
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

| username: xfworld | Original post link

What version are you using? Please provide more information.

| username: TiDBer_FMWXa7ja | Original post link

Version: TiDB 6.5

| username: xfworld | Original post link

There is too little information. It would be best if the running logs also contain key information to make a judgment…

| username: Kongdom | Original post link

This is normal. Check the node score to see if it is particularly low or if a leader eviction policy has been added.

| username: 考试没答案 | Original post link

Please show the config of PD.

| username: 考试没答案 | Original post link

Enter the PD interactive interface and execute the store command to take a look.

| username: TiDBer_FMWXa7ja | Original post link

The weight of all three nodes is 1.

| username: 考试没答案 | Original post link

leader_weight and leader_count? leader_score is also related to scheduling. You’d better post all of them.

| username: TiDBer_FMWXa7ja | Original post link

The image you provided is not visible. Please provide the text you need translated.

| username: Kongdom | Original post link

You can refer to this for troubleshooting

| username: 考试没答案 | Original post link

From the screenshot, it was found that their weight configurations are the same. However, the scores are indeed very different.

| username: Kongdom | Original post link

The default weights are the same, so differences in scores are normal. Usually, if the leader of a certain node is directly 0, it indicates an issue with the node, such as resource constraints or slow writes. You can refer to the documentation to troubleshoot step by step.

| username: db_user | Original post link

Check if the nodes without a leader have evict-leader configured, and also check the remaining disk space on the three nodes.

| username: WalterWj | Original post link

Check pd-ctl schedule show to see if there is an evict-leader.

| username: TiDBer_FMWXa7ja | Original post link

Does this mean there is an evict-leader?

| username: WalterWj | Original post link

Remove it.

| username: 考试没答案 | Original post link

The default value of tidb_gc_life_time is 10m, which means that the data deleted 10 minutes ago will be cleaned up. If you want to restore the data, you need to set the tidb_gc_life_time to a larger value before deleting the data.

| username: 考试没答案 | Original post link

The image you provided is not visible. Please provide the text content for translation.

| username: TiDBer_FMWXa7ja | Original post link

Thank you for personally helping to solve the problem.