How to Adjust the Data Retention Period for monitoring_servers

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: monitoring_servers数据保留时间怎么调整

| username: MSGJC

Add the following to the monitoring_servers configuration:
storage_retention: 2b
Found that it has no effect, the monitoring data is still increasing.

2
| username: ddhe9527 | Original post link

Run ps -ef | grep prometheus to check if the Prometheus process includes the --storage.tsdb.retention option and see how many days this option is set to.

3
| username: 啦啦啦啦啦 | Original post link

If you want to retain it for 2 days, it should be 2d, right?

4
| username: MSGJC | Original post link

I made a mistake and wrote “d” as “b”.

5
| username: MSGJC | Original post link

Yes, thank you.

6
| username: MSGJC | Original post link

When will the data trigger recycling?

7
| username: Raymond | Original post link

Did the parameters set not take effect after restarting Prometheus?

8
| username: MSGJC | Original post link

Restarted Prometheus with --storage.tsdb.retention=2d, but the storage space has not been reclaimed. Now the storage space usage has reached 95%, so I want to know when the reclamation will be triggered.

9
| username: Raymond | Original post link

I experimented myself and found that setting -storage.tsdb.retention takes effect immediately. There might be two reasons why it doesn’t work on your side:

  1. The parameter was not set successfully. Go directly to the Prometheus server and execute ps -ef | grep prometheus to check if --storage.tsdb.retention is set correctly.
  2. Most of your storage space is occupied by data from the last 2 days, so setting --storage.tsdb.retention=2d won’t make a difference. Check the directory where Prometheus monitoring data is stored and see when the earliest data directory was created.
10
| username: MSGJC | Original post link

I set it to retain for one day
ps -ef|grep prometheus
tidb 22701 1 4 Jul15 ? 03:04:46 bin/prometheus/prometheus --config.file=/sdata/usr/local/tidb/monitoring_servers1/conf/prometheus.yml --web.listen-address=:9091 --web.external-url=http://172.18.3.251:9091/ --web.enable-admin-api --log.level=info --storage.tsdb.path=/sdata/data/tidb/monitoring_servers1 –storage.tsdb.retention=1d

Directory with large monitoring data storage: docdb

11
| username: Raymond | Original post link

In the picture you showed, those are not Prometheus data files, but Pump data files. Did you enter the wrong directory? Check if Prometheus data files and Pump data files are placed on the same file system.

12
| username: ablewang_xiaobo | Original post link

The data retention time for Prometheus should be adjustable. “storage.tsdb.retention.time=10000d”

13
| username: system | Original post link

This topic will be automatically closed 60 days after the last reply. No new replies are allowed.