Dashboard unable to monitor ext4 disk usage on TiKV nodes

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dashboard 监控 不到tikv节点的ext4磁盘使用情况

| username: xiaohetao

TiDB production environment dashboard monitoring does not show the ext4 disk usage of TiKV nodes. I posted about this issue last year.
dashboard monitoring - :ringer_planet: TiDB technical issues / deployment & operation management - TiDB Q&A community (asktug.com)

Is there a solution to this problem now?

2
| username: xiaohetao | Original post link

Our TiDB cluster version is v4.0.6, but it is not available in the version options.

3
| username: 像风一样的男子 | Original post link

My previous version was 4.0.9, and the disk was ext4, there was no such issue.

4
| username: xiaohetao | Original post link

@[Billmay’s cousin] Could you please ask the experts if they can solve this? Thank you :pray:

5
| username: Billmay表妹 | Original post link

You can try upgrading the version or consider using system-level monitoring tools such as Prometheus and Grafana. TiKV provides various metrics that can be collected and visualized using Prometheus and Grafana.

6
| username: xiaohetao | Original post link

Is this phenomenon present in version v4.0.6?

7
| username: xiaohetao | Original post link

If upgrading, which version can v4.0.6 be directly upgraded to?

8
| username: Billmay表妹 | Original post link

The upgrade documentation is available. You can consider upgrading to versions 6.5.5, 6.1.7, or 7.1.2.

9
| username: Fly-bird | Original post link

Check the metrics on port 9100 yourself to see if there is any disk information.

10
| username: 随缘天空 | Original post link

Try the following methods:

  1. Ensure that the monitoring metrics for TiKV nodes are correctly configured. In the TiDB Dashboard, make sure that the relevant metrics for monitoring TiKV nodes have been added.

  2. Check if the disk monitoring of TiKV nodes is working properly. You can check the disk usage by running the command df -h. If you cannot obtain the disk usage, it might be due to permission issues preventing the reading of disk information. Ensure you have sufficient permissions to read disk information.

  3. Verify that the monitoring configuration of TiKV nodes is correct. Ensure that Prometheus monitoring and Grafana dashboards are correctly configured in the TiDB cluster, and that the relevant metrics for monitoring TiKV nodes are properly set.

  4. Check if the network connection of TiKV nodes is normal. Ensure that TiKV nodes can connect to the monitoring system normally so that the metric data can be sent to the monitoring system.