False Alarms in Alertmanager After Scaling Down Compute Nodes in TiDB

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiDB 缩容计算节点后 alertmanager 误报

| username: GreenGuan

After normally scaling in the compute nodes, there are still a lot of alarms. When checking the Prometheus interface, it considers the scaled-in nodes as server down. How can I eliminate this alarm in Prometheus (not alertmanager)?

Database version v5.4.2

2
| username: WalterWj | Original post link

Is it that Prometheus hasn’t rolled once? :thinking:

3
| username: GreenGuan | Original post link

My operation is to use the tiup tool to scale-in the TiDB compute node. Do I need to reload Prometheus separately?
tiup cluster reload xxx -N prometheus

4
| username: WalterWj | Original post link

I think you can give it a try.

5
| username: GreenGuan | Original post link

It seems we need to manually reload.

6
| username: WalterWj | Original post link

Done?

7
| username: GreenGuan | Original post link

Okay, the Prometheus interface no longer reports server down.

8
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.