Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: TiDB binlog 日志开启后，是否需要关注保留周期及存储空间？
[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Problem Phenomenon and Impact]
During the process of searching through TiDB’s official documentation or forums, most of the information treats binlog as a tool, without paying attention to whether the binlog retention period and storage space issues need to be considered after enabling TiDB binlog logging. Could any forum members help answer this question?
The default retention period for TiDB binlog is 7 days. This tool will be explicitly deprecated in the future. Switch to using TiCDC.
In the column/Principle Interpretation: # drainer binlog cleaning mechanism source code analysis:
In addition to being able to delete drainer binlog (hereinafter referred to as binlog) using the Linux operating system command rm, it actually also maintains an automatic binlog cleaning mechanism. The function mainly involved in the automatic cleaning of binlog is GCByTime, which is defined at line 384 in ./pkg/binlogfile/binlogger.go. The following code is the definition of this function (the code version referenced in this article is tidb-binlog version 4.0.13).
To enable the automatic cleaning mechanism of drainer binlog, drainer actually provides a configuration option syncer.to.retention-time. By configuring this parameter under the drainer tag, binlog can be automatically cleaned.
The drainer configuration can be referenced at https://github.com/pingcap/tidb-binlog/blob/v4.0.9/cmd/drainer/drainer.toml#L153. If you want to add the binlog automatic cleaning function to drainer, you can use tiup cluster edit-config cluster_name and add the following content under the drainer configuration (assuming the binlog retention period is set to 4 days). Of course, this value can only be written as an integer type, as the definition of this variable in the code is of type int.
drainer: syncer.to.retention-time: 4
After setting this parameter, the initial value of the retentionTime parameter in the GCByTime function comes from this value. However, syncer.to.retention-time needs to be converted to the time.Duration type by calculating the corresponding number of hours for the days. The following code is about the time conversion part (source code in the drainer/sync/pb.go NewPBSyncer function).
However, this is not mentioned in the official documentation.
Original text address:
Pay attention to disk space.
If disk space is insufficient, it will affect your normal operations. You must pay attention to it.
Configuration items related to log files.
filename
- General log file name.
- Default value: “”
- If set, general logs will be output to this file.
max-size
- Size limit of the log file.
- Default value: 300
- Unit: MB
- Maximum setting limit is 4096.
max-days
- Maximum number of days to retain logs.
- Default value: 0
- By default, logs are not cleaned; if a value is set, TiDB will clean up expired log files after
max-days.
max-backups
- Maximum number of retained log files.
- Default value: 0
- By default, all logs are saved; if set to 7, a maximum of 7 old log files will be retained.
Pay attention to disk space.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
|
username: 麻烦是朋友
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: