The Issue of Log Backup Latency in TiDB

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb的日志备份延时问题

| username: zhanggame1

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version] 7.5.0
[Encountered Problem: Problem Phenomenon and Impact]

In the log backup:
checkpoint [global]: Data in the cluster earlier than this checkpoint has been saved to the backup storage, and it is also the most recent point in time that the backup data can be restored.

From the actual situation, the delay is quite significant. The delay for ticdc to synchronize to the downstream is generally about 1 second. Is there any way to shorten the delay for this log backup?

| username: wfxxh | Original post link

Run the br log start command to start the log backup task, which will continuously run on each TiKV node and periodically back up TiDB change data to the specified storage in small batches.

Note that it is in small batches and periodic, so there will definitely be a delay.
It is different from the usage scenario of ticdc.

| username: zhanggame1 | Original post link

I understand this. Is there a place to adjust the parameters for this periodic small batch?

| username: wfxxh | Original post link

I am using v6.5.3 and have not found any parameters related to controlling intervals.

| username: cassblanca | Original post link

According to the introduction, this usage scenario involves refreshing all the changed data records generated after the last refresh to the backup storage within 5 to 10 minutes. It is used for backup recovery with an RPO of less than 10 minutes. A high refresh frequency is likely to have a significant impact on cluster performance.

| username: zhanggame1 | Original post link

I use NFS with an interval of about 1 minute.

| username: Jayjlchen | Original post link

The current BR log-based PITR solution has an RPO of less than 5 minutes, with the gap generally being around 2 minutes. It depends on your scenario, as this is not a solution for real-time recovery of backup databases.

| username: dba远航 | Original post link

The mechanisms are different, it’s a bit difficult.

| username: zhang_2023 | Original post link

Using a remote disk, 2 minutes…

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.