DM Configuration for Upstream Database

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm配置上游数据库

| username: TiDBer_mGBoAnW9

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration]
[Attachments: Screenshots / Logs / Monitoring]

Can tables from different databases within the same TiDB cluster be synchronized using DM? There is an error in configuring the data source.

Message: database driver error, RawCause: Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your TiDB version for the right syntax to use line “SLAVE HOSTS”

| username: 有猫万事足 | Original post link

It is recommended to clearly explain the upstream and downstream situations and the details of the error.

I understand your problem as follows: The upstream consists of tables with the same structure from different database instances, and the downstream is a single table in a TiDB cluster. DM can import tables with the same structure from different upstream database instances into a single table in the downstream TiDB cluster. However, attention should be paid to write hotspots and primary key conflicts.

Additionally, regarding the configuration data source error, you have provided too little information to determine the cause. For beginners, it is strongly recommended to use the WebUI for configuration.

The probability of encountering issues is relatively low when you can get things done with just a few mouse clicks.

Also, pay attention to the permission settings for upstream and downstream users, and you should be good to go.

| username: redgame | Original post link

Logs or monitoring data

| username: tidb菜鸟一只 | Original post link

DM can synchronize data from sharded tables and databases to downstream.

| username: zhanggame1 | Original post link

The original poster seems to be asking if it’s okay for both upstream and downstream to be the same database, but different databases.

| username: TiDBer_mGBoAnW9 | Original post link

Yes, the upstream database is still TiDB.

| username: TiDB_C罗 | Original post link

You can use TiCDC.

| username: TiDBer_mGBoAnW9 | Original post link

Can’t the upstream for DM synchronization be TiDB?

| username: tidb菜鸟一只 | Original post link

Just configure the routing rules:

routes:                           # Define the routing rules for migrating source tables to target TiDB tables, multiple rules can be defined
  route-rule-1:                   # Rule name
    schema-pattern: "test1"       # Match the source database name, supports wildcards "*" and "?"
    table-pattern: "*"            # Match the source table name, supports wildcards "*" and "?"
    target-schema: "test"         # Target TiDB database name
    target-table: "*"             # Target TiDB table name
| username: Hacker007 | Original post link

DM is for synchronizing MySQL.

| username: TiDBer_mGBoAnW9 | Original post link

The official documentation states that it uses the MySQL protocol, and TiDB itself is based on the MySQL protocol.

| username: Hacker007 | Original post link

DM synchronization also reads MySQL’s binlog. The binlog format of TiDB is different from that of MySQL. DM supports synchronization from MySQL or MariaDB to TiDB.

| username: TiDBer_mGBoAnW9 | Original post link

Then we can only use TiCDC.

| username: TiDB_C罗 | Original post link

MySQL → DM → TiDB
TiDB → TiCDC → MySQL/TiDB

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.