TiDB Data Migration - Table Data Synchronization Between Two Clusters (Non-Full Database Synchronization)

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiDB数据迁移 - 两个集群之前的表数据同步(非全库同步)

| username: 超7成网友

Request for Data Migration
We currently have two sets of TiDB:
The version in use is v3.0.12;
The newly built version is 6.5, which has not yet been put into production.
Now we need to migrate some tables’ data from the old version (v3.0.12) to the v6.5 version.
The largest table is about 200G. Is there any good method for this? I see that the documentation mentions dumpling and lightning. Does using dumpling have a significant impact on the read and write operations of the cluster in use?

| username: WalterWj | Original post link

Concurrent 1 backup will be fine.

| username: 超7成网友 | Original post link

Does dumpling use a single thread to back up slowly?

| username: WalterWj | Original post link

200GB of data itself is not much.

| username: 这里介绍不了我 | Original post link

Yes, 200G. Use Lightning with a smaller number of threads and observe the load.

| username: Soysauce520 | Original post link

Dumpling splits the export based on rowid, so it doesn’t put much pressure on the cluster. However, you need to pay attention to the GC time during the export. If it exceeds the GC time, an error will occur.

| username: TIDB-Learner | Original post link

BR cannot be used across major versions. If you need to migrate a large volume of data, is there really no good tool other than physical backup and restore?

| username: zhanggame1 | Original post link

Can only use dumpling, the BR version is too different.

| username: onlyacat | Original post link

There should be no major issues with controlling concurrency in Dumpling.

| username: TiDBer_rvITcue9 | Original post link

Got it! Here is the translation:


| username: 源de爸 | Original post link

In addition to paying attention to pressure issues, maintenance window time must also be considered.