What is the appropriate setting for tidb_gc_life_time?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb_gc_life_time设置多大合适

| username: Christophe

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version] 5.0.0
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
What is the appropriate value to set for tidb_gc_life_time? Thank you!
[Resource Configuration]
[Attachments: Screenshots / Logs / Monitoring]

2
| username: tidb菜鸟一只 | Original post link

It depends on the business and the actual situation. If it’s too long, it will take up too much space. If it’s too short, a lot of historical change data might not be retrievable.

3
| username: Fly-bird | Original post link

Try adjusting according to the business, but don’t make it too long, as it may cause performance issues.

4
| username: 像风一样的男子 | Original post link

What is your requirement for changing this parameter? If there is no need, don’t change it.

5
| username: 有猫万事足 | Original post link

Overall, if there are more deletions and the focus is on recording the state, the duration should be shorter.
If there are more insertions and the focus is on recording operation logs, the duration can be longer.

It also depends on your machine’s performance.
Anyway, my machine’s performance is not great, so I set it to 10 minutes. Since I’m the only one managing this database, having a 10-20 minute flashback is generally enough to cover my own operational mistakes.

6
| username: TiDBer_小阿飞 | Original post link

If it’s for importing data into a new database, it’s recommended to set the time longer, around 20-30 minutes. After importing the data, change it back! If the business changes frequently, 5-10 minutes should be fine. It also depends on the module node configuration, etc.

7
| username: Christophe | Original post link

Preventing the right person from accidentally deleting data can also allow for data restoration.

8
| username: TiDBer_vfJBUcxl | Original post link

tidb_gc_life_time Introduced from Version v5.0

  • Scope: GLOBAL
  • Persisted to Cluster: Yes
  • Type: Duration
  • Default Value: 10m0s
  • Range: [10m0s, 8760h0m0s]
  • This variable is used to specify the retention period for data during each garbage collection (GC). The variable value is in Go’s Duration string format. During each GC, the current time minus the value of this variable is used as the safe point.

Note

  • In scenarios with frequent data updates, setting the tidb_gc_life_time value too high (e.g., several days or even months) may lead to potential issues such as:
    • Occupying more storage space.
    • A large amount of historical data may affect system performance to some extent, especially for range queries (e.g., select count(*) from t).
  • If the duration of a transaction exceeds the value configured for tidb_gc_life_time, during GC, to ensure the transaction can continue to run normally, the system will retain the data from the start time start_ts of the transaction. For example, if the tidb_gc_life_time value is configured as 10 minutes, and during a GC, the earliest running transaction in the cluster has been running for 15 minutes, then this GC will retain the most recent 15 minutes of data.
9
| username: TiDBer_vfJBUcxl | Original post link

The default is fine.

10
| username: 像风一样的男子 | Original post link

That depends on your response time. Setting this time too long can affect database performance.

11
| username: Billmay表妹 | Original post link

I see that most community users generally set it to 10 minutes.

12
| username: 大飞哥online | Original post link

For example:
If the maintenance personnel accidentally delete data and the business side only discovers it 2 hours later, if tidb_gc_life_time is set to 3 hours, the data can be flashed back. In this case, there will be more old versions of the data, which may result in some performance loss during queries.

It mainly depends on the business and recovery time, etc.

13
| username: 啦啦啦啦啦 | Original post link

We set it to 1 hour. 10 minutes might result in accidental data deletion, and there might not be enough time to react or even notice it. However, it mainly depends on the data update frequency and the disk, which can somewhat impact performance. The longest I’ve seen it set to is 1 day.

14
| username: zhanggame1 | Original post link

Based on the requirements, I plan to set it to 24 hours or 48 hours. Our database is mainly for inserts, with few updates and no deletes, so having more MVCC copies won’t have an impact.

15
| username: 大飞哥online | Original post link

24 hours, that’s quite a lot. Don’t you use select? If you frequently perform range queries, it will affect performance.

16
| username: zhanggame1 | Original post link

The impact on select depends on the number of MVCC versions caused by update and delete. Ensuring fewer updates and deletes can solve this problem.

17
| username: 大飞哥online | Original post link

Sure, that works. It still depends on the business. :blush:

18
| username: xingzhenxiang | Original post link

I mistook it for TiKV.

tikv_gc_life_time | 1h

19
| username: TiDB_C罗 | Original post link

I think 24 hours is appropriate, but the default is 10 minutes. It’s hard to find problems in 10 minutes.

20
| username: Kongdom | Original post link

We’ve always used the default values. When the hardware can’t keep up, setting it too long can affect performance.