How to Synchronize Data from a Table in TiDB 3.0 to TiDB 6.1 in Real-Time?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb3.0 的某表数据如何实时同步到 tidb6.1环境?

| username: 海石花47

Old environment: TiDB 3.0
New environment: TiDB 6.1
Question: There is a table (700 million rows of data) in the old environment that needs to be migrated to the new environment. Is there any way to achieve real-time data synchronization first, and then modify the connection domain name once to complete the migration operation?

| username: 裤衩儿飞上天 | Original post link

DM …

| username: magic | Original post link

It’s too difficult to rely entirely on tools for seamless switching;
Our previous approach was to first import historical data offline into 6.1, then dual-write to both clusters, and switch to the new cluster after data synchronization.

| username: 海石花47 | Original post link

Dual writing is also an option, but it requires cooperation from the development team… it’s a bit troublesome.

| username: 海石花47 | Original post link

Isn’t the upstream of DM MySQL?

| username: magic | Original post link

It seems this is the only way, after all, the version difference is too significant.

| username: 海石花47 | Original post link

I want to try using dumpling + lightning now. First, dump all the current historical data and import it into the new environment. Then, when switching, do another incremental dump (based on the update time column). Is this feasible?

| username: 海石花47 | Original post link

Help~ Is there any expert who can assist?

| username: TiDBer_pkQ5q1l0 | Original post link

The version is too old, it feels difficult to achieve real-time without dual writing.

| username: 啦啦啦啦啦 | Original post link

DM doesn’t work, using TiDB Binlog is fine.

| username: WalterWj | Original post link

binlog

| username: dockerfile | Original post link

Obviously dumpling + ticdc

| username: 啦啦啦啦啦 | Original post link

There is no TiCDC in version 3.0, but I have personally tested that binlog works fine.

| username: 海石花47 | Original post link

The old version of ticdc does not have it.

| username: 海石花47 | Original post link

Dual writing, the R&D side can’t handle it…

| username: 海石花47 | Original post link

How should binlog be configured? Set up a binlog in the old TiDB, and then directly synchronize it to the new TiDB? Can I synchronize just a single table? What are the specific steps?

| username: 啦啦啦啦啦 | Original post link

Yes, we also have this requirement. We just tested it a few weeks ago. You can follow the official documentation:

| username: 海石花47 | Original post link

Is it possible to synchronize only one table between the old and new TiDB?

| username: 啦啦啦啦啦 | Original post link

Sure, you can configure it in the drainer configuration file.

| username: 裤衩儿飞上天 | Original post link

DM really doesn’t work, sorry~