DM Creates Redundant Empty Databases

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm创建冗余的空库

| username: TiDBer_hwEZA4rV

When DM performs full synchronization, the configuration is set to synchronize the tdehr database to the tidb_tdehr database. However, an extra empty tdehr database is always created at the target address, but the synchronization result is correct.

Configuration:

name: tdehr
task-mode: all
is-sharding: true
shard-mode: pessimistic
ignore-checking-items: []
meta-schema: dm_meta
enable-heartbeat: false
heartbeat-update-interval: 0
heartbeat-report-interval: 0
timezone: ""
case-sensitive: false
collation_compatible: loose
target-database:
  host: tidb-bigdata.hz.td
  port: 4000
  user: root
  password: Rk2s49@
  max-allowed-packet: null
  session: {}
  security: null
mysql-instances:
- source-id: tdehr
  meta: null
  filter-rules: []
  column-mapping-rules: []
  route-rules:
  - route-01
  expression-filters: []
  black-white-list: ""
  block-allow-list: balist-01
  mydumper-config-name: dump-01
  mydumper:
    mydumper-path: ./bin/mydumper
    threads: 4
    chunk-filesize: "64"
    statement-size: 0
    rows: 0
    where: ""
    skip-tz-utc: true
    extra-args: ""
  mydumper-thread: 0
  loader-config-name: load-01
  loader:
    pool-size: 16
    dir: ./dumped_data
    import-mode: sql
    on-duplicate: replace
  loader-thread: 0
  syncer-config-name: sync-01
  syncer:
    meta-file: ""
    worker-count: 16
    batch: 100
    queue-size: 1024
    checkpoint-flush-interval: 30
    compact: false
    multiple-rows: false
    max-retry: 0
    auto-fix-gtid: false
    enable-gtid: false
    disable-detect: false
    safe-mode: false
    enable-ansi-quotes: false
  syncer-thread: 0
  continuous-validator-config-name: validator-01
online-ddl: true
shadow-table-rules: []
trash-table-rules: []
online-ddl-scheme: ""
routes:
  route-01:
    schema-pattern: tdehr
    table-pattern: '*'
    target-schema: tidb_tdehr
    target-table: ""
filters: {}
column-mappings: {}
expression-filter: {}
black-white-list: {}
block-allow-list:
  balist-01:
    do-tables:
    - db-name: tdehr
      tbl-name: '*'
    do-dbs: []
    ignore-tables: []
    ignore-dbs: []
mydumpers:
  dump-01:
    mydumper-path: ./bin/mydumper
    threads: 4
    chunk-filesize: "64"
    statement-size: 0
    rows: 0
    where: ""
    skip-tz-utc: true
    extra-args: ""
loaders:
  load-01:
    pool-size: 16
    dir: ./dumped_data
    import-mode: sql
    on-duplicate: replace
syncers:
  sync-01:
    meta-file: ""
    worker-count: 16
    batch: 100
    queue-size: 1024
    checkpoint-flush-interval: 30
    compact: false
    multiple-rows: false
    max-retry: 0
    auto-fix-gtid: false
    enable-gtid: false
    disable-detect: false
    safe-mode: false
    enable-ansi-quotes: false
validators:
  validator-01:
    mode: none
    worker-count: 4
    validate-interval: 10s
    check-interval: 5s
    row-error-delay: 30m0s
    meta-flush-interval: 1m0s
    batch-query-size: 100
clean-dump-file: false
ansi-quotes: false
remove-meta: false
experimental:
  async-checkpoint-flush: false
2
| username: buchuitoudegou | Original post link

This is a table-level routing rule (although * already matches all tables in this database, DM doesn’t know that). If there are other tables in this database that do not hit this table-level routing rule, and we have not created this “redundant” database, the migration will report an error.

If you don’t want to create this redundant database, you can add a database-level routing rule:

schema-pattern: tdehr
target-schema: tidb_tdehr

I feel this is an area that can be optimized, let me help you log an issue.

3
| username: Hacker007 | Original post link

If you want to synchronize the entire database, you can synchronize at the database level. There’s no need to use table-pattern: ‘*’.

4
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.