How do you currently synchronize data from databases other than MySQL?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 请教下大家,目前数据mysql以外的数据库同步是怎么做的

| username: xmlianfeng

Currently, there is a need to synchronize data from MongoDB to TiDB. Could you please share how everyone is doing this~!

| username: ShawnYan | Original post link

mongo to kafka, then sink to TiDB.

| username: Billmay表妹 | Original post link

Here are a few practical articles from community users, take a look first~

Using Gravity to Achieve Data Replication from MongoDB to TiDB

Online Migration of MongoDB Data to TiDB at Banyu Database

Why We Abandoned MongoDB and MySQL and Chose TiDB

| username: 大飞哥online | Original post link

Need to use an intermediary, such as Kafka.

| username: ShawnYan | Original post link

Take a look at these two articles, both focus on collecting oplog to Kafka and then writing into TiDB:

Using Gravity to Achieve Data Replication from MongoDB to TiDB

Online Migration of MongoDB Data to TiDB in Panyu Database

| username: 普罗米修斯 | Original post link

Try CloudCanal for data synchronization.

| username: ShawnYan | Original post link

Where is the original image?

| username: Kongdom | Original post link

Synchronized through the program interface, without database-level synchronization.

| username: oceanzhang | Original post link

Using Kafka to collect and synchronize optlog, but some data needs to be handled manually.

| username: 普罗米修斯 | Original post link

CloudCanal Homepage Introduction
https://www.clougence.com/cc-doc/intro/product_intro

| username: 随缘天空 | Original post link

Many practical solutions have already been mentioned above. If it’s just about synchronizing data from MongoDB to TiDB, you can also use some ETL tools, which are very convenient, such as Kettle, StreamSets, etc.

| username: tidb菜鸟一只 | Original post link

CloudCanal is the simplest and also offers a free trial.

| username: 有猫万事足 | Original post link

I came across a demonstration on Bilibili about using Tapdata to transfer data from MongoDB to MySQL, but I haven’t used it myself.

If you don’t want to go through the hassle, CloudCanal is still a good option.

| username: zhanggame1 | Original post link

You can use ETL tools, such as DataX.

| username: tidb狂热爱好者 | Original post link

CloudCanal

| username: come_true | Original post link

DG deployment

| username: 春风十里 | Original post link

What are the synchronization requirements? If it’s real-time, you might need to consider commercial synchronization software. You can check with domestic companies like Info, DSG, and Nine Bridges, or foreign ones like OGG. If it’s non-real-time synchronization or scheduled synchronization, you can consider Kettle or DataX.

| username: oceanzhang | Original post link

Kafka, and then you need to transform the data, right?

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.