Store Leader Frequently Changes

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: store leader 经常出现变化

| username: beacoolkid

【TiDB Usage Environment】Production Environment
【TiDB Version】
【Reproduction Path】Frequent leader changes causing balance migration
【Encountered Issues: Symptoms and Impact】
【Resource Configuration】
【Attachments: Screenshots/Logs/Monitoring】

Has anyone encountered this situation? Why is this happening?

| username: Kongdom | Original post link

Encountered this before, it happens when the server resources are insufficient.

| username: zhanggame1 | Original post link

Check the logs of various database components when the leader count fluctuates drastically in the graph to see if there are any issues.

| username: beacoolkid | Original post link

Which resources are insufficient? CPU? What parameters can be adjusted?

| username: beacoolkid | Original post link

It’s hard to determine where the problem is. It doesn’t seem to be an issue with the components.

| username: tidb菜鸟一只 | Original post link

Check the resource monitoring page of the corresponding machine during the relevant time period.

| username: Kongdom | Original post link

It is unbalanced. For example, if server A lacks resources and migrates to server B, as a result, server B also becomes insufficient due to the migration, and then it migrates back to server A.

| username: beacoolkid | Original post link

Which aspect of imbalance are you referring to?

| username: h5n1 | Original post link

Check the monitoring tikv detail -errors.

| username: beacoolkid | Original post link


| username: h5n1 | Original post link

Take a look at the CPU, IO, and network latency monitoring during this time period.

| username: Kongdom | Original post link

What I usually encounter are disk space, IO, and network issues. You can check if the store score has changed.

| username: 昵称想不起来了 | Original post link

Encountered this when the IO network was not good.

| username: beacoolkid | Original post link

There should be no problem with network I/O.

| username: redgame | Original post link

Insufficient resources and uneven load