How to Achieve Data Synchronization Between Two TiDB Clusters Deployed on Kubernetes

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 如何将在k8s上部署的两个tidb集群实现数据同步

| username: panqiao

[TiDB Usage Environment] Testing
[TiDB Version] v6.5.0
[Reproduction Path] Operations performed that led to the issue
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

There is a requirement to synchronize data between a specific database in two TiDB clusters deployed on k8s, such as synchronizing database ‘a’ in cluster A with database ‘b’ in cluster B. It seems that there is no relevant technology mentioned in the official documentation, so I would like to seek help from the experts here.

2
| username: 裤衩儿飞上天 | Original post link

You can consider DM
TiDB Data Migration Overview | PingCAP Docs

3
| username: xfworld | Original post link

TiCDC can be used, but BR must be enabled first to initialize the data (to achieve a complete data copy). TiCDC only supports incremental data.

Cluster A → Cluster B

4
| username: 裤衩儿飞上天 | Original post link

Does TiCDC support schema and table routing? I’m not quite sure.

5
| username: panqiao | Original post link

Can it be synchronized to a single database and a single table, or does it have to be the entire database?

6
| username: panqiao | Original post link

Boss, can this DM tool be used on K8s? I see that the articles all use physical deployment methods, not container deployment.

7
| username: panqiao | Original post link

I saw the k8s documentation, I’ll give it a try.

8
| username: panqiao | Original post link

I took a look, and it seems that the DM tool is mainly used for data migration, not for real-time master-slave data synchronization.

9
| username: 裤衩儿飞上天 | Original post link
  1. First, determine if TiCDC can meet your schema routing requirements. If it can, use TiCDC.
    If your upstream and downstream database names are consistent, you can directly use CDC. CDC supports database and table filtering. For details, refer to:
    Changefeed Log Filter | PingCAP Docs
  2. Both TiCDC and DM can synchronize, but both are asynchronous replication and cannot achieve real-time synchronization as you mentioned.
  3. However, personally, I feel that DM is heavier than TiCDC. If high availability is not a strict requirement, DM can also work with fewer nodes in the cluster.
10
| username: tidb菜鸟一只 | Original post link

Just use TiCDC.

11
| username: panqiao | Original post link

Okay, I’ll test ticdc first. Thanks for the support.

12
| username: panqiao | Original post link

Okay, I’ll try TiCDC first. Thanks for the support.

13
| username: yiduoyunQ | Original post link

Refer to 在 Kubernetes 上部署 TiCDC | PingCAP 文档中心

14
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.