Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: DM备份
During DM migration, an error was reported that the target table does not exist. After manually creating the table and restarting the task, the error still indicates that the table does not exist.
“Message”: “[Lightning:Restore:ErrSchemaNotExists]table ailearn_test.intell_third_poll_data schema not found”
DM version: v1.10.3
TiDB version: v5.4.0
【TiDB Usage Environment】Production/Test Environment/POC
【TiDB Version】
【Encountered Problem】
【Reproduction Path】Operations performed that led to the issue
【Problem Phenomenon and Impact】
【Attachments】
- Relevant logs, configuration files, Grafana monitoring (https://metricstool.pingcap.com/)
- TiUP Cluster Display information
- TiUP Cluster Edit config information
- TiDB-Overview monitoring
- Corresponding module Grafana monitoring (if any, such as BR, TiDB-binlog, TiCDC, etc.)
- Corresponding module logs (including logs from 1 hour before and after the issue)
If the question is related to performance optimization or fault troubleshooting, please download the script and run it. Please select all and copy-paste the terminal output results for upload.
Has this DDL file been generated?
Could you please share the steps for DM migration?
Can you show the specific configuration?
name: “ailearn-test”
task-mode: “all”
shard-mode: “pessimistic”
meta-schema: “dm_meta” # A schema will be created in the downstream database to store metadata
ignore-checking-items: [“auto_increment_ID”] # In this example, there is an auto-increment primary key in the upstream, so it needs to be ignored
target-database:
host: “10.10.100.14”
port: 4000
user: “dm_worker”
password: “QInjI6TsmA+POyB7yKhauEAAqAx336rCMMgmzw==”
mysql-instances:
- source-id: “mysql-ailearn-test”
route-rules:
[
“route-rule-03”,
“route-rule-10”,
]
block-allow-list: “listA”
routes:
route-rule-03:
schema-pattern: “ailearn”
table-pattern: “global_id”
target-schema: “ailearn_test”
target-table: “global_id”
route-rule-10:
schema-pattern: “ailearn”
table-pattern: “intell_third_poll_data”
target-schema: “ailearn_test”
target-table: “intell_third_poll_data”
block-allow-list:
listA:
do-tables:
- db-name: “ailearn”
tbl-name: “global_id”
- db-name: “ailearn”
tbl-name: “intell_third_poll_data”
tiup dmctl --master-addr 127.0.0.1:8261 start-task task-ailearn-test.yaml
Error
I deleted the target database and tables, restarted the task, and it automatically created the database and tables, but then it reported that the table does not exist.
Could you please send the logs? We can take a look and see what’s going on.
What do you think? Did you notice any issues? Our TiDB is version 5.4.0 and DM is version 6.1.0. One of the nodes was restarted this morning.
The main reason is that the default value of the tidb_distsql_scan_concurrency parameter is 15, which means that each query will use 15 threads to scan data. If there are many concurrent queries, it will cause a large number of threads to be created, resulting in high CPU usage. You can try to reduce the value of this parameter to alleviate the problem.
The image you provided is not visible. Please provide the text you need translated.
Can manually created tables be queried normally? Are all tables unable to be successfully migrated during DM migration?
Is the DM version 1.10.3? Does this exist? Is it 1.0.3?
For version 1.0.3,
if the DM version is <= v1.0.6, use black-white-list instead of block-allow-list.
shard-mode: “pessimistic” I didn’t see this under version 1.0, but it’s there under version 2.0. I suggest you can remove it. I’m not sure if this affects anything, but you can try using the complete configuration file of the standard version first.
The image is not visible. Please provide the text you need translated.
Another log file in the same directory is needed.
|
username: Johnpan
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: