The route is not effective when synchronizing data with DM

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm 同步数据时路由不生效

| username: TiDBer_AxdzSBqt

[Test Environment] Testing environment
[TiDB Version] 6.5.1
[Reproduction Path] Scenario 1: The upstream MySQL data source has a test database, which is synchronized to the downstream and changed to the test1 database, but both test and test1 databases will exist downstream (in the case of one source and multiple tasks). Scenario 2: The downstream TiDB has a test database, and the test database of a data source is synchronized to the downstream as the test1 database using DM. When the test database of the upstream MySQL is deleted, both the test and test1 databases downstream will be deleted.
[Encountered Problem: Problem Phenomenon and Impact]
[Resource Configuration]
routes:

rule-1:
schema-pattern: “test”
table-pattern: “aaa”
target-schema: “test1”
target-table: “aaa”

block-allow-list:
rule-1:
do-dbs: [
“test”
]

| username: Billmay表妹 | Original post link

It can be seen that this is a question about DM data synchronization, which requires knowledge of DM to answer.

First, for scenario one, you can use DM’s route-rules to solve it. In route-rules, you can specify the mapping relationship between the source and target databases, for example:

route-rules:
  - name: "test to test1"
    schema-pattern: "test"
    target-schema: "test1"

In this way, when synchronizing data, DM will synchronize the data from the source database to the target database, and will not synchronize the data to the same-named database that already exists in the source database.

For scenario two, you can use block-allow-list to solve it. In block-allow-list, you can specify which databases are not allowed to be deleted, for example:

block-allow-list:
  rule-1:
    do-dbs: ["test1"]

In this way, when deleting the test database in the upstream MySQL, DM will check whether the test1 database in the downstream TiDB is in the block-allow-list. If it is, it will not be deleted.

As for the issue of routing not taking effect, you can check whether the route-rules and block-allow-list are correctly configured in the DM configuration file and whether DM is correctly started. If the problem persists, you can check the DM log files for more information.

| username: TiDBer_AxdzSBqt | Original post link

I didn’t see the configuration “- name: ‘test to test1’” in the official documentation.

| username: 有猫万事足 | Original post link

tiup dmctl config task <task-name>

Check what the task configuration looks like after the task is submitted.

| username: Billmay表妹 | Original post link

# Starting from DM v2.0.2, Binlog event filter can also be configured in the upstream database configuration file
# case-sensitive: false
# filters:
# - schema-pattern: dmctl
#   table-pattern: t_1
#   events: []
#   sql-pattern:
#   - alter table .* add column `aaa` int
#   action: Ignore

Take a look at this content

| username: TiDBer_AxdzSBqt | Original post link

The routing rules are normal, but I suspect it might be a bug. I see that others in the community have encountered this situation as well.

| username: TiDBer_AxdzSBqt | Original post link

I looked into it, and this has nothing to do with the phenomenon encountered.

| username: redgame | Original post link

Check the mapping relationships of the database.

| username: TiDBer_AxdzSBqt | Original post link

I have checked using the method tiup dmctl config task, and the mapping relationship is normal.

| username: TiDBer_AxdzSBqt | Original post link

After testing, there is an issue when setting table-pattern and target-table. I don’t know why this happens, whether it’s a bug or by design.

| username: okenJiang | Original post link

Although the downstream has a test database and will synchronize the deletion of the test database, it will not create tables or synchronize data. Is that correct?

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.