Added a TiDB node, but Grafana does not display the node information

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 扩了1个tidb的节点，但grafana没有显示该节点信息

| username: fly4310862

[TiDB Usage Environment] Production Environment / Testing / POC
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Problem Phenomenon and Impact]

Help needed: Added one TiDB node, but Grafana does not display the monitoring information for that node.

[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

2
| username: Kongdom | Original post link

If the monitoring information is not timely, you can try restarting the cluster or the monitoring node.

3
| username: fly4310862 | Original post link

–Cluster Information
tiup cluster display cluster-xxx

ID Role Host Ports OS/Arch Status Data Dir Deploy Dir

10.10.1.1:9093 alertmanager 10.10.1.1 9093/9094 linux/x86_64 Up /data/deploy/data.alertmanager /data/deploy
10.10.1.1:3000 grafana 10.10.1.1 3000 linux/x86_64 Up - /data/deploy
10.10.1.1:9090 prometheus 10.10.1.1 9090 linux/x86_64 Up /data/deploy/prometheus2.0.0.data.metrics /data/deploy

–Is it like the following statement to restart the grafana node?
tiup cluster reload cluster-xxx -N 10.10.1.1:3000

4
| username: Kongdom | Original post link

Yes, it does. Additionally, does the tiup cluster display command include information about the scaled-out nodes?

5
| username: WalterWj | Original post link

-R prometheus also

6
| username: fly4310862 | Original post link

The nodes that have been scaled out are displayed in tiup cluster display, and the nodes that have been scaled out are also shown in the Dashboard.

7
| username: 裤衩儿飞上天 | Original post link

Try restarting the Grafana and Prometheus nodes.

8
| username: tidb菜鸟一只 | Original post link

It should be that the configuration in Grafana has not been updated, or the information in Prometheus has not been updated. Try restarting these two nodes.

9
| username: fly4310862 | Original post link

Restarted the Grafana and Prometheus nodes, still not working…

10
| username: Kongdom | Original post link

Refer to this for processing:

11
| username: fly4310862 | Original post link

I just executed tiup cluster prune, and the newly added node information appeared in the monitoring. I guess the cluster was affected by the Tombstone status. Thank you all, love you.

12
| username: 裤衩儿飞上天 | Original post link

I’m trembling with fear. I just took over an online environment, also version 4.0.8, and the monitoring is still integrated into a unified Prometheus. Upgrading definitely won’t be easily approved, and adding a monitoring node definitely won’t be approved by the leadership either. :joy::joy::joy::joy::joy:

13
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.