Data Migration from MySQL to TiDB v5 Using DM: Mismatch in Table Count

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 使用DM从MySQL迁移数据只TiDB v5，表数量对不上

| username: TiDBer_iLonNMYE

[Test Environment for TiDB] Testing
[TiDB Version] v5.4.3
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Phenomenon and Impact] 111
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]
After deploying the DM 5.4.3 cluster, I migrated from the source MySQL 5.7.34 to the TiDB V5.4.3 cluster. The check passed, so I started the data migration task. The status showed as Finished, and the progress was 100%. However, the source database had 1473 tables, while the target database had 1441 tables, missing 32 tables. How can I troubleshoot this?

During the check phase, there was a type of Warning stating that foreign keys could be parsed but would be ignored by TiDB. After the migration, I found that these tables did exist. There were more than 40 tables without primary keys and unique indexes, which I manually added before the migration. Random checks on a few tables also showed that they were successfully migrated.

2
| username: 考试没答案 | Original post link

Was it exported using the dumpling tool??? Or was it exported after starting the task???

3
| username: TiDBer_iLonNMYE | Original post link

Migration using DM FULL mode

4
| username: 考试没答案 | Original post link

Save the data exported by dumpling, change it to false, and do not clear it.

5
| username: 考试没答案 | Original post link

Is the data volume large? Can it be exported again?

6
| username: TiDBer_iLonNMYE | Original post link

You can try again. Additionally, the stored procedures, triggers, and functions in the MySQL source database did not migrate along with the database. How can they be migrated?

7
| username: 考试没答案 | Original post link

Stored procedures are not supported by TiDB.

8
| username: TiDBer_iLonNMYE | Original post link

What about triggers and functions?

9
| username: 考试没答案 | Original post link

Please familiarize yourself with this document.

10
| username: 考试没答案 | Original post link

None of these are supported. Compare them yourself.
For large MySQL databases, none of what you mentioned would be used. Our entire company does not have a single stored procedure or trigger.

11
| username: TiDBer_iLonNMYE | Original post link

Re-migrate, error message: subtasks with name mysql_2_tidb for sources[xxx] already exists
Do I need to start a new task?

12
| username: 考试没答案 | Original post link

Delete the incorrect task.

13
| username: TiDBer_iLonNMYE | Original post link

There are start and stop, but I didn’t see drop or destroy in dmctl --help?
I started a new task again.

14
| username: TiDBer_iLonNMYE | Original post link

It was found that the missing 32 tables are views. It seems that DM v5.4.3 does not migrate views, so they need to be created manually.

15
| username: 考试没答案 | Original post link

Yes. Export it and take a look, then you’ll know what’s missing… Hahaha.

16
| username: 小王同学Plus | Original post link

You can check here: TiDB Data Migration 简介 | PingCAP 文档中心

image
image1380×590 97.2 KB

17
| username: TiDBer_iLonNMYE | Original post link

Xiao Wang has dug deep! Thank you.

18
| username: 小王同学Plus | Original post link

:thinking: If you use views, let me add one more thing (in case you need to use dumpling for data export later). By default, dumpling does not export views. However, dumpling has parameters that can be modified -W or --no-views set to false to include views.

19
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.