TiDB Incremental Backup

TiDB
1

Application environment:

Production

TiDB version: v4.0.14

Problem:

We are using TiDB v4.0.14 and we have around 100 TB of data to be replicated. Is there a way to enable incremental backup for this particular version of TiDB? Has anyone done it earlier?
Please help with the steps/process for the same.

2

To enable incremental backup in TiDB, follow this detailed step-by-step guide. Incremental backups are a useful strategy to reduce backup volume by only capturing data changes since the last backup. Here’s how you can set it up:

Step 1: Prepare Your Environment

  1. Ensure Compatibility: Verify that your TiDB version supports incremental backups. Note that development for some incremental backup features may have stopped, and alternatives like log backup and Point-in-Time Recovery (PITR) might be recommended. Check the TiDB Incremental Backup and Restore Guide for the latest recommendations.

  2. Set tidb_gc_life_time: Adjust the tidb_gc_life_time system variable to ensure that Multi-Version Concurrency Control (MVCC) data is not garbage collected before the incremental backup is completed. For example, if you plan to perform hourly backups, set this variable to a value greater than 1 hour.

Step 2: Perform a Full Backup

Before you can perform an incremental backup, you need a baseline full backup. Use the br tool to back up your entire cluster or specific databases/tables.

br backup full --pd "${PD_IP}:2379" \
--storage "s3://backup-101/snapshot-202209081330?access-key=${access-key}&secret-access-key=${secret-access-key}" \
--ratelimit 128

Step 3: Configure Incremental Backup

  1. Determine the Last Backup Timestamp: Use the br validate command to decode the last backup timestamp (lastbackupts).

    LAST_BACKUP_TS=`tiup br validate decode --field="end-version" --storage "s3://backup-101/snapshot-202209081330?access-key=${access-key}&secret-access-key=${secret-access-key}" | tail -n1`

  2. Run the Incremental Backup Command: Use the br backup command with the --lastbackupts option to back up only the data changes since the last backup.

    tiup br backup full --pd "${PD_IP}:2379" \
--storage "s3://backup-101/snapshot-202209081330/incr?access-key=${access-key}&secret-access-key=${secret-access-key}" \
--lastbackupts ${LAST_BACKUP_TS} \
--ratelimit 128

Step 4: Restore Incremental Data

  1. Restore Full Backup First: Ensure that all data from the full backup is restored before applying incremental backups.

    tiup br restore full --pd "${PD_IP}:2379" \
--storage "s3://backup-101/snapshot-202209081330?access-key=${access-key}&secret-access-key=${secret-access-key}"

  2. Apply Incremental Backups: Restore the incremental data to the target cluster.

    tiup br restore full --pd "${PD_IP}:2379" \
--storage "s3://backup-101/snapshot-202209081330/incr?access-key=${access-key}&secret-access-key=${secret-access-key}"

Important Considerations

  • Limitations: Be aware of limitations such as unsupported batch renaming of tables during incremental backups. Deleted tables during the backup process might still exist after restoration and need manual deletion.
  • Alternative Solutions: Consider using log backup and PITR for more robust backup and recovery solutions, as recommended in the TiDB Incremental Backup and Restore Guide.

By following these steps, you can effectively set up and manage incremental backups in TiDB, ensuring efficient use of storage and faster backup processes. For more detailed information, refer to the official TiDB documentation.