How to Achieve Data Synchronization Between Identical Tables in Different Databases?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 不同库相同表间数据同步如何实现?

| username: Kongdom

【TiDB Usage Environment】Production Environment
【TiDB Version】v6.5.3

I originally wanted to use TiCDC for data synchronization between the production database and the analysis database. After deployment, I found that it does not support synchronization between databases with different names. If I want to synchronize from db1.table1 to db2.table1 and from db1.table1 to db2.table2 in real-time or near real-time, do you have any good suggestions?

| username: Jasper | Original post link

DM supports table routing, you can give it a try.

| username: tidb菜鸟一只 | Original post link

You can also first send the data from TiCDC to Kafka, and then handle the table names when reading Kafka messages. However, this is a bit more troublesome and it’s better to directly use DM’s table routing feature.

| username: Kongdom | Original post link

I looked at the documentation, and it seems that it’s not supported. My scenario involves different schemas and different table structures across databases. In DM, it’s a scenario of merging sharded tables and databases.

| username: Kongdom | Original post link

:thinking: It seems that it is not supported, right? Do I have to configure hundreds of routes for hundreds of tables? :joy:

| username: TiDBer_vfJBUcxl | Original post link

Different database names or table names data validation

| username: Kongdom | Original post link

I haven’t used it yet. Is this for verification or transmission?

| username: zhanggame1 | Original post link

DM supports different databases.

| username: TiDBer_vfJBUcxl | Original post link

When you use synchronization tools like TiDB DM, you can set route-rules to synchronize data to a specified downstream table. The sync-diff-inspector provides the function of verifying tables with different database names and table names by setting rules.

| username: tidb菜鸟一只 | Original post link

If it’s from A.A to B.A, you can directly write a rule like this:
rule-2:
schema-pattern: “A”
target-schema: “B”
If it’s from A.B to B.A, then you would need a rule for each table, right?
rule-1:
schema-pattern: “A”
table-pattern: “B”
target-schema: “B”
target-table: “A”
Otherwise, how would it know which table you want to sync to which table?

| username: Kongdom | Original post link

I think it’s necessary to support it~ :yum:

| username: redgame | Original post link

TiDB Data Migration

| username: Kongdom | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.