Configuration Issues in the Routes Section of DM

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: DM的routes部分配置问题

| username: TiDBer_8rWAgqMU

When using DM for migration, if the source table names are as follows:
s1.t1
s1.t2
s1.t3
The desired table names after migration are:
s1_bak.t1_bak
s1_bak.t2_bak
s1_bak.t3_bak
May I ask if the following configuration is supported:
routes:
route-rule-1:
schema-pattern: “s1”
table-pattern: “t*”
target-schema: “${schema_name}_bak”
target-table: “${table_name}_bak”

In the above configuration, ${schema_name} and ${table_name} are variables, and the variable names are arbitrarily chosen by me. Does your company have similar configurations, and what are the actual variable names?

2
| username: songxuecheng | Original post link

Support for inconsistent database and table names, variable syntax should not be supported, support for regular expression matching.

3
| username: TiDBer_8rWAgqMU | Original post link

Is it possible to arrange it later? :heart_eyes:

4
| username: buchuitoudegou | Original post link

I don’t understand what this requirement is :face_with_spiral_eyes: Currently, regular expressions or postfix expressions can both perform table name mapping, and it seems more straightforward and easier to understand…

5
| username: TiDBer_8rWAgqMU | Original post link

What I mean is using variables in target-schema or target-table.

6
| username: buchuitoudegou | Original post link

Are there any problems that can only be solved using variables?

7
| username: TiDBer_8rWAgqMU | Original post link

For example, in my case above, although it can be achieved without using variables, the task configuration would be a bit verbose. Using variables makes the configuration much clearer.

8
| username: buchuitoudegou | Original post link

That might be an issue with your usage habits. However, too many ways of expression can also confuse other users, so it is unlikely to be introduced as a feature. Thank you for your support and feedback!

9
| username: TiDBer_8rWAgqMU | Original post link

So how should the route be configured for the following?

s1.t1
s1.t2
s1.t3

The desired table names after migration are:

s1_bak.t1_bak
s1_bak.t2_bak
s1_bak.t3_bak

Should it be configured like this? (If you have a better configuration, please share it. Mainly because there are too few examples of target-schema and target-table configurations in your documentation, only schema-pattern and table-pattern examples are more abundant.)

routes:                          
  route-rule-1:                  
    schema-pattern: "s1"  
    table-pattern: "t1"
    target-schema: "s1_bak"    
    target-table: "t1_bak"
  route-rule-2:                  
    schema-pattern: "s1"  
    table-pattern: "t2"
    target-schema: "s1_bak"    
    target-table: "t2_bak"
  route-rule-3:                  
    schema-pattern: "s1"  
    table-pattern: "t3"
    target-schema: "s1_bak"    
    target-table: "t3_bak"
10
| username: buchuitoudegou | Original post link

Yes. The route in DM requires at least one rule to be written for each target table downstream.

In your example, each database table name is mapped one-to-one to the downstream database table, so each upstream database table needs to have a rule written; whereas most other users need to merge multiple upstream tables into a single downstream database table.

This design does indeed cause some inconvenience for you. We will record this requirement in a GitHub issue to see if we can make some optimizations in the future.

11
| username: TiDBer_8rWAgqMU | Original post link

Sure! Thanks for the clarification!

12
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.