Data Synchronization

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 同步数据

| username: TiDBer_CkS2lbTx

How can I use CDC to synchronize data from other types of databases into TiDB in real-time?

| username: WalterWj | Original post link

Simply put, just search on Baidu for something like: “hive data migration to MySQL.” Methods that work for MySQL are generally suitable for TiDB.

| username: TiDBer_CkS2lbTx | Original post link

The production business uses PostgreSQL data, and some functions may not be supported. How should this be handled?

| username: WalterWj | Original post link

Convert the PG xx function to MySQL syntax.
Or use AI, any AI tool will do, the accuracy of SQL rewriting is very high.

| username: buddyyuan | Original post link

Applying your own developed functions?

| username: WalterWj | Original post link

Custom configurations would be troublesome; TiDB does not support them.

| username: TiDBer_CkS2lbTx | Original post link

There are many things that are not supported. Just the fact that foreign keys are not supported makes it difficult.

| username: zhaokede | Original post link

TiDB supports defining foreign keys, but they do not work. This needs to be controlled in the business code.

| username: WalterWj | Original post link

Some are just syntax support without actual functionality. It depends on your expected modifications.

| username: CuteRay | Original post link

You can take a look at this tool, TurboDX for MySQL, but I’m not quite sure about the custom function part, it still needs to be modified.

| username: 像风一样的男子 | Original post link

There are many methods to migrate PostgreSQL data to TiDB: datax, navicat, TurboDX for MySQL, TiDB-for-PostgreSQL (GitHub - DigitalChinaOpenSource/TiDB-for-PostgreSQL: PgSQL compatible on distributed database TiDB), etc. Currently, I haven’t seen any methods to achieve real-time synchronization.

| username: tidb菜鸟一只 | Original post link

Try CloudCanal.

| username: gcworkerishungry | Original post link

Similar to incremental migration tools. Not many, domestically there are DSG, Yingfang, etc.; internationally there is OGG. It seems they are all paid.

| username: ffeenn | Original post link

If it’s a public network, you can try Alibaba Cloud’s DTS, or use ETL tools for real-time synchronization.

| username: TiDBer_aaO4sU46 | Original post link

Migration basically relies on manual effort.

| username: Kongdom | Original post link

:man_technologist: What solution did you end up using? We are migrating from SQL Server to TiDB and also have concerns about migrating functions.

| username: 这里介绍不了我 | Original post link

Has it been resolved, boss?

| username: Demo二棉裤 | Original post link

Otherwise, it depends on the upstream. If it’s Oracle, use TMS. If it’s MySQL, use DM. If neither works, OGG is suitable for everything.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.