Full Data Migration from TiDB to TiDB

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TIDB全量数据迁移到TIDB

| username: Johnpan

Hello, TiDB version 5.4.0

In the production environment, we need to migrate the full data from TiDB 5.4.0 to another new TiDB cluster and keep the incremental data synchronized. The data volume is about 1TB. What is the best and fastest method to achieve this?

【TiDB Usage Environment】Production/Test Environment/POC
【TiDB Version】
【Encountered Issues】
【Reproduction Path】What operations were performed that led to the issue
【Issue Phenomenon and Impact】
【Attachments】

  • Relevant logs, configuration files, Grafana monitoring (https://metricstool.pingcap.com/)
  • TiUP Cluster Display information
  • TiUP Cluster Edit config information
  • TiDB-Overview monitoring
  • Corresponding module Grafana monitoring (if any, such as BR, TiDB-binlog, TiCDC, etc.)
  • Corresponding module logs (including logs from 1 hour before and after the issue)

If the question is related to performance optimization or troubleshooting, please download the script and run it. Please select all the terminal output and copy-paste it for upload.

2
| username: Kongdom | Original post link

BR backup is the fastest, after all, it is a physical backup. ticdc can perform incremental transmission.

3
| username: forever | Original post link

Use BR for full backup and TiCDC for incremental backup.

4
| username: HACK | Original post link

First, perform logical export and import, then use CDC for subsequent incremental synchronization.

5
| username: Kongdom | Original post link

:wink: Logical import and export should not be as fast as physical backup and restore, right?

6
| username: HACK | Original post link

BR is the fastest :smile:

7
| username: Hacker_92quIrMM | Original post link

When using TiCDC for synchronization, pay attention to the timestamp issue. If the time interval is too long, synchronization failures may occur.

8
| username: alfred | Original post link

BR full backup with Ticdc incremental synchronization

9
| username: HACK | Original post link

Pay attention to the GC duration settings.

10
| username: Johnpan | Original post link

Okay, thank you.

11
| username: Johnpan | Original post link

Okay, thank you for the suggestion.

12
| username: Johnpan | Original post link

After the migration is completed, the account passwords of the old cluster have not been migrated. Does BR backup not include account passwords?
If that’s the case, do the account passwords of the old cluster need to be manually dumped?

13
| username: 啦啦啦啦啦 | Original post link

The image is not visible. Please provide the text you need translated.