How to Perform Incremental Backup and Data Recovery in TiDB Production Environment?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiDB生产环境的增量备份以及恢复数据怎么做？

| username: TiDBer_superwu

【TiDB Usage Environment】Production Environment
【TiDB Version】4.0.12
【Encountered Problem】How to perform incremental backup and point-in-time data recovery
【Reproduction Path】What operations were performed that caused the problem
【Problem Phenomenon and Impact】

  1. Currently using TiDB in the production environment, TiDB (4.0.12) On K8S (1.19)
  2. How to perform incremental backup? Incremental data uploaded to S3
  3. How to perform point-in-time recovery with full and incremental data?

Key Point: Production environment, not a test environment
After searching for a long time, it is currently recommended not to use in production environments. So, what components are recommended for incremental backup in production environments?

【Attachments】

  • Relevant logs, configuration files, Grafana monitoring (https://metricstool.pingcap.com/)
  • TiUP Cluster Display information
  • TiUP Cluster Edit config information
  • TiDB-Overview monitoring
  • Corresponding module Grafana monitoring (if any, such as BR, TiDB-binlog, TiCDC, etc.)
  • Corresponding module logs (including logs one hour before and after the problem)

If the question is related to performance optimization or fault troubleshooting, please download the script and run it. Please select all and copy-paste the terminal output results for upload.

2
| username: BraveChen | Original post link

TiDB Binlog can do this, and there is also a best practice Directly Addressing the Pain Points of Backup and Recovery: Fast Point-in-Time Recovery Based on TiDB Binlog | PingCAP

3
| username: HACK | Original post link

Currently, achieving point-in-time recovery is still a bit troublesome.

4
| username: TiDBer_superwu | Original post link

What about TiCDC? Can it be used for incremental backups in production? Also, I see that the official BR tool for data recovery is not recommended for use in production.

5
| username: ddhe9527 | Original post link

TiCDC used to be able to do incremental backups, but now it can’t. Previously, br store had a cdclog option, which was removed in v5.4.

Dingtalk_20220629105358
Dingtalk_20220629105358812×570 123 KB

6
| username: BraveChen | Original post link

Currently, the official implementation by PingCAP at some customer sites is based on TiDB binlog backup. There are two drainers: one drainer generates binlog files locally and uploads them to S3 at regular intervals, while the other drainer synchronizes to the downstream standby database. In the future, it can be used for master-slave switching.

7
| username: system | Original post link

This topic was automatically closed 1 minute after the last reply. No new replies are allowed.