Data Synchronization

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 数据同步

| username: TiDBer_Y2d2kiJh

【TiDB Usage Environment】Production Environment
【TiDB Version】v5.4.0
【Reproduction Path】The TiDB cluster has been set up and is already online.
【Encountered Problem: Problem Description and Impact】A standby database has been created, and the backup is mysql5.7.34. How to migrate the entire TiDB database to MySQL and then perform incremental synchronization.
【Resource Configuration】
【Attachment: Screenshot/Log/Monitoring】

| username: 啦啦啦啦啦 | Original post link

ticdc can meet

| username: TiDBer_Y2d2kiJh | Original post link

I would like to ask, what method can be used to migrate TiDB data to MySQL first? The SQL files exported by Dumpling are too many, with thousands of SQL files. Executing them one by one will cause many issues, such as primary and foreign key problems.

| username: 啦啦啦啦啦 | Original post link

You can use any MySQL-compatible logical export tools, such as mysqldump and mydumper.

| username: TiDBer_Y2d2kiJh | Original post link

May I ask, if using mysqldump to export, how to confirm the start-ts value of ticdc?

| username: TiDBer_pkQ5q1l0 | Original post link

If you can’t get the exact start-ts, you still need to use Dumpling to export the snapshot and use Loader to import it. Additionally, there shouldn’t be any foreign key issues with the data exported from TiDB.

| username: 啦啦啦啦啦 | Original post link

Forgot about the binlog position issue… In that case, use dumpling to export and lightning to import into MySQL, using the Logical mode. Lightning also supports MySQL.

| username: TiDBer_Y2d2kiJh | Original post link

Is loader import referring to the myloader tool? Previously, using source to import always reported primary and foreign key issues, as shown in the image. After disabling the foreign key, it still reported errors.

| username: TiDBer_pkQ5q1l0 | Original post link

Loader is the import tool developed by PingCAP. This error looks like a CDC synchronization error and has nothing to do with the import. Also, may I ask if there are foreign keys in the downstream MySQL table? Doesn’t TiDB upstream not support foreign keys? How can there be foreign key constraints downstream?

| username: xingzhenxiang | Original post link

Foreign key constraint error has nothing to do with TiDB.

| username: TiDBer_Y2d2kiJh | Original post link

Is it because TiDB does not support foreign keys, so creating a foreign key does not take effect and therefore has no impact, while MySQL’s foreign keys take effect, causing continuous errors?

| username: TiDBer_pkQ5q1l0 | Original post link

It is possible. If there are foreign keys in MySQL, they will definitely take effect. It is recommended to remove them uniformly.