Although configured according to the official documentation, it still prompts: The necessary component `NgMonitoring` is not started in the cluster, some functions will be unavailable

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 已经按照官方文档配置了还是提示:集群中未启动必要组件 NgMonitoring,部分功能将不可用。

| username: 玩具果冻

[TiDB Usage Environment] Production Environment
[TiDB Version] Upgraded from 5.3 to 6.5
[Reproduction Path] Online upgrade of TiDB cluster
[Encountered Problem: Symptoms and Impact] Followed the official documentation for online upgrade of TiDB cluster, both tiup and tiup cluster are the latest versions. After upgrading to 6.5, the dashboard still prompts: System health check failed, necessary component NgMonitoring not started in the cluster, some functions will be unavailable. Followed the steps in this post: 已经配置了还是提示:集群中未启动必要组件 `NgMonitoring`,部分功能将不可用。 - TiDB 的问答社区, deleted all data in the directory /mysqldata/tidb-data/prometheus-8249, and reloaded the prometheus node, but it still shows unavailable.
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

| username: wzf0072 | Original post link

Try deploying the TidbNGMonitoring CR according to this: 访问 TiDB Dashboard | PingCAP 文档中心

| username: xingzhenxiang | Original post link

Monitoring can be completely rebuilt by scaling down and then scaling up again.

| username: 玩具果冻 | Original post link

Scaled down and up Prometheus, but the issue remains unresolved. It still indicates that the necessary component NgMonitoring is not started in the cluster, and some functions will be unavailable.

| username: 小王同学Plus | Original post link

Hello, does this affect the use of the cluster?
If ngm is successfully deployed, after entering the TiDB Dashboard, the top right corner will no longer display the message System health check failed: necessary component NgMonitoring not started in the cluster, some features will be unavailable.

| username: ljluestc | Original post link

If the system health check in the TiDB dashboard fails after upgrading to version 6.5, it may be because the NgMonitoring component did not start properly during the upgrade.

One possible solution is to manually start the NgMonitoring component using the following command:

tiup cluster start <cluster-name> --node <node-name> -R prometheus

Replace <cluster-name> with the name of your TiDB cluster and <node-name> with the name of the node running the Prometheus component. The -R flag specifies the component to start.

If this does not resolve the issue, you can try the following steps:

  1. Check the logs of the Prometheus component to see if there are any error messages or warnings that might indicate the cause of the problem. The logs are usually located in the directory /mysqldata/tidb-deploy/prometheus-8249/log/.

  2. Check the configuration of the Prometheus component to ensure it is configured correctly and pointing to the correct data sources. The configuration files are generally located in the directory /mysqldata/tidb-deploy/prometheus-8249/conf/.

  3. Restart the Prometheus component and check the TiDB dashboard again to see if the issue is resolved.

If these steps do not resolve the issue, you can try opening a support ticket with TiDB for further assistance.

| username: caiyfc | Original post link

If you are upgrading the cluster using tiup version 1.12.0, you can upgrade the tiup version to 1.12.1 and then restart Prometheus.
You can check out this article for specific bugs:
Column - Step-by-Step Guide to Becoming a TiDB Contributor | TiDB Community