Ng-monitor port fails to start

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ng-monitor端口起不来

| username: TiDBer_OzwiOwPc

[TiDB Usage Environment] Production Environment
[TiDB Version] Cluster upgraded from old version 7.1.1
[Encountered Issue: Problem Phenomenon and Impact] Monitoring indicates that ng is not enabled. After confirming the configuration file and restarting Prometheus, the ng_port port fails to start, with continuous errors in tsdb.log and no significant anomalies in ng.log.

2
| username: 像风一样的男子 | Original post link

Go to the server and find the scripts directory, there is a startup script inside, try starting it manually and see.

3
| username: TiDBer_OzwiOwPc | Original post link

bin/ng-monitoring-server --config /opt/tidb-deploy/prometheus-9090/conf/ngmonitoring.toml? No response.

4
| username: 像风一样的男子 | Original post link

Check if the process is still running with ps -ef | grep monitoring.

5
| username: TiDBer_OzwiOwPc | Original post link

No, there’s no response when I type the command. There’s nothing except the logs in the picture.

6
| username: TiDBer_OzwiOwPc | Original post link

It should be caused by this, not sure how to fix it.

7
| username: tidb菜鸟一只 | Original post link

Memory is running low, check the memory usage of the node’s machine.

8
| username: dba远航 | Original post link

The write exceeds the limit, see the last highlighted yellow part.