After reloading Prometheus using tiup, it was found that Prometheus lost half an hour of data. Could you explain the mechanism of Prometheus WAL persistence?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 通过tiup reload prometheus，发现prometheus 丢失了半小时的数据，想请问下prometheus wal的落盘机制？

| username: Raymond

After restarting Prometheus, I found that Prometheus lost half an hour of data. I would like to ask about the disk persistence mechanism of Prometheus WAL. Based on the results, it seems that Prometheus WAL logs are not written to disk for every piece of data. Is it possible that they are written to disk after a certain period of time or after reaching a certain amount?

2
| username: hey-hoho | Original post link

How was it determined that Prometheus lost half an hour of data? Is there a blank segment in the monitoring graph?

3
| username: Raymond | Original post link

Yes, it’s very obvious.

4
| username: hey-hoho | Original post link

Are all the monitoring graphs blank, or just some of them?

5
| username: Raymond | Original post link

Missing half an hour of monitoring data.

6
| username: yilong | Original post link
  1. How was it deployed? Physical machine or operator?
  2. Why did Prometheus restart, and how long was the restart?