Data Migration

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 数据迁移

| username: TiDBer_Y2d2kiJh

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version] v5.4.0
I would like to ask how to migrate data from database A to database B. Database B currently has data that cannot be deleted, and the table structures are the same.

2
| username: luoscorn | Original post link

For large data, use Dumpling to export a full data backup, TiDB Lightning to import into a new database, and DM for incremental data.

3
| username: luoscorn | Original post link

No problem at all. You can directly back up to S3 or other places, and then restore it when you come back.

4
| username: dba-kit | Original post link

It shouldn’t work. Currently, rename operations are not supported. During recovery, it seems to use the same table name. Additionally, BR cannot restore to the same cluster, right?

5
| username: TiDBer_Y2d2kiJh | Original post link

Can I use BR to fully back up database A and then restore it to database B? They are different clusters.

6
| username: dba-kit | Original post link

You can check the usage recommendations in the official documentation:

7
| username: luoscorn | Original post link

The official documentation 从 TiDB 集群迁移数据至另一 TiDB 集群 | PingCAP 文档中心 recommends backing up to S3.

8
| username: TiDBer_Y2d2kiJh | Original post link

Error: cannot find rewrite rule: [BR:Restore:ErrRestoreInvalidRewrite] invalid rewrite rule. Now using restore to recover, this error is prompted at the end.

9
| username: dba-kit | Original post link

This cluster refers to TiKV nodes. BR can be compared to MySQL’s physical backup, which skips the tidb-server and restores directly to the TiKV nodes. Therefore, if you restore directly to the current cluster, it will directly overwrite the TiKV files of the current cluster.

10
| username: GreenGuan | Original post link

If I remember correctly, ticdc (incremental) and br in version 5.4.x do not support rewrite rules. If incremental data is not considered, it is recommended to use dumping and then lighting (rewrite rule).

11
| username: dba-kit | Original post link

If it’s within the same cluster from db_A to db_B, you can use Dumpling + TiDB Lightning for the full backup, which is very fast. For incremental data, you can use TiCDC.

12
| username: TiDBer_Y2d2kiJh | Original post link

They are different clusters.

13
| username: 胡杨树旁 | Original post link

How much data is there? For the full amount, you can use Dumpling + Lightning. For incremental data, you can use the Binlog tool, or you can directly extract it using Kettle.

14
| username: db_user | Original post link

If the table structure is the same, should the data be placed in the original table, or should the table name be changed in the original database? If it’s the latter, then use the dumpling+lightning routing method.

15
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.