Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: tidb的报警存储200g文件 我能删除吗
【TiDB Usage Environment】Production Environment / Testing / PoC
【TiDB Version】5.40
【Reproduction Path】What operations were performed when the issue occurred
Monitoring reports disk full.
du: cannot read directory ‘prometheu’: Permission denied
0 prometheu
218G prometheus-9090
229G tikv-20160
[dbadmin@zy-tw-zf-k8s-uat-dbvm-tikv01 data]$
【Encountered Issues: Issue Phenomenon and Impact】
【Resource Configuration】
【Attachments: Screenshots/Logs/Monitoring】
The Prometheus folder should contain monitoring data, and it is not recommended to delete it directly. You might consider adjusting the retention period.
I just delete them directly.
May I ask which command sets the retention period?
Try using tiup cluster edit-config
to modify the storage_retention
configuration. The default should be 30 days.
In the Prometheus startup script, there is a setting for the number of days to retain logs. I have tried directly modifying the script and then killing the Prometheus process, which can change the log retention duration.
Log location: /tidb-deploy/prometheus-9090/scripts/run_prometheus.sh
Parameter: --storage.tsdb.retention=“30d”
After manually killing the process, the system will automatically restart Prometheus.
Official documentation:
TiDB Monitoring FAQ | PingCAP Documentation Center