How to clear Prometheus monitoring after taking TiKV offline

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 下线tikv后怎么清除prometheus监控

| username: 普罗米修斯

[TiDB Usage Environment] Production Environment
[TiDB Version] V5.2.4
[Encountered Problem: Problem Phenomenon and Impact]
After scaling down a TiKV, how to delete the monitoring configuration in Prometheus;

| username: Fly-bird | Original post link

Restart Grafana.

| username: 普罗米修斯 | Original post link

Restarting Grafana didn’t work. There’s also something in the Prometheus configuration.
Image

| username: tidb菜鸟一只 | Original post link

reload Prometheus

| username: Miracle | Original post link

If Prometheus data has an expiration policy, you don’t need to worry about it; it will be automatically deleted when it expires.

| username: ti-tiger | Original post link

  • If the TiDB cluster is deployed using tiup, you can use the tiup cluster edit-config command to edit the Prometheus configuration file, delete the corresponding TiKV node information, and then use the tiup cluster reload -R prometheus command to reload the Prometheus configuration.
  • If the TiDB cluster is manually deployed, you can directly edit the Prometheus configuration file, delete the corresponding TiKV node information, and then restart the Prometheus service.
| username: 普罗米修斯 | Original post link

The default data expiration is 30 days, and data older than 30 days will be deleted. The configuration will not change, and 30 days have already passed.

| username: 普罗米修斯 | Original post link

Reloading Prometheus encountered an error, possibly due to network issues. However, the Prometheus configuration for TiKV 192.168.80.202 is missing on the server.


Image

But the web version of Prometheus monitoring still has the configuration for TiKV 192.168.80.202, and Grafana still displays it.

Image

| username: 普罗米修斯 | Original post link

After reloading Prometheus and restarting it, the removed TiKV nodes were excluded after clearing the cache.

| username: TiDBer_小阿飞 | Original post link

Re-edit this file, delete the corresponding TIKV, and then restart.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.