TiDM Synchronization Task, DDL Execution Blocked, Wait for Latest Schema Version Change

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiDM 同步任务,DDL执行阻塞,wait latest schema version changed

| username: 大飞飞呀

【TiDB Usage Environment】Production Environment
【TiDB Version】7.1.0
【Reproduction Path】Operations performed that led to the issue

【Encountered Issue: Phenomenon and Impact】
DM blocks when synchronizing DDL executed on upstream MySQL
alter table table_a add index idx_taxpayer_number(CH_NUMBER);

[2024/05/09 10:52:50.361 +08:00] [INFO] [ddl_worker.go:1204] ["[ddl] wait latest schema version changed(get the metadata lock if tidb_enable_metadata_lock is true)"] [ver=821386] ["take time"=12m57.081431959s] [job="ID:1610598, Type:add index, State:running, SchemaState:delete only, SchemaID:6313, TableID:7240, RowCount:0, ArgLen:6, start time: 2024-05-09 10:39:53.158 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0, UniqueWarnings:0"]
[2024/05/09 10:52:50.369 +08:00] [INFO] [ddl_worker.go:980] ["[ddl] run DDL job"] [worker="worker 5, tp add index"] [job="ID:1610598, Type:add index, State:running, SchemaState:delete only, SchemaID:6313, TableID:7240, RowCount:0, ArgLen:0, start time: 2024-05-09 10:39:53.158 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0, UniqueWarnings:0"]
[2024/05/09 10:52:50.385 +08:00] [INFO] [domain.go:240] ["diff load InfoSchema success"] [currentSchemaVersion=821386] [neededSchemaVersion=821387] ["start time"=2.597749ms] [gotSchemaVersion=821387] [phyTblIDs="[7240]"] [actionTypes="[7]"]
[2024/05/09 10:52:50.390 +08:00] [INFO] [domain.go:833] ["mdl gets lock, update to owner"] [jobID=1610598] [version=821387]
[2024/05/09 10:52:50.433 +08:00] [INFO] [ddl_worker.go:1204] ["[ddl] wait latest schema version changed(get the metadata lock if tidb_enable_metadata_lock is true)"] [ver=821387] ["take time"=52.453221ms] [job="ID:1610598, Type:add index, State:running, SchemaState:write only, SchemaID:6313, TableID:7240, RowCount:0, ArgLen:6, start time: 2024-05-09 10:39:53.158 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0, UniqueWarnings:0"]
[2024/05/09 10:52:50.441 +08:00] [INFO] [ddl_worker.go:980] ["[ddl] run DDL job"] [worker="worker 4, tp add index"] [job="ID:1610598, Type:add index, State:running, SchemaState:write only, SchemaID:6313, TableID:7240, RowCount:0, ArgLen:0, start time: 2024-05-09 10:39:53.158 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0, UniqueWarnings:0"]

Code location

【Resource Configuration】Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
【Attachments: Screenshots/Logs/Monitoring】

| username: xfworld | Original post link

TiDB’s DDL is a dynamic switching process, so you need to wait for a while…

Unless it is confirmed that the DDL operation is locked or dead and hasn’t moved to the next step.

There are two methods:

  1. Manually skip this DDL step using DM.
  2. Manually stop the DDL job, and handle the DDL process manually during idle time.

There are some minor bugs in version 7.1.0, so it is recommended to upgrade to the latest minor version, 7.5.1…

| username: 大飞飞呀 | Original post link

I don’t dare to upgrade. Last time, when I upgraded from 5.X to 7.1.0, I found several compatibility issues.

| username: xfworld | Original post link

I made a mistake, it’s 7.1.5.

Major versions have some changes (mainly new features, which will definitely have some impact), while minor versions are mainly focused on bug fixes.

| username: Billmay表妹 | Original post link

Minor version upgrades can be done directly!

| username: 小龙虾爱大龙虾 | Original post link

Have you checked the metadata lock? 元数据锁 | PingCAP 文档中心

| username: 大飞飞呀 | Original post link

It’s completely unsearchable, the information_schema.ddl_jobs table is too large and doesn’t even have an index.

| username: yytest | Original post link

You can try a minor version upgrade, no need to upgrade to the latest version.

| username: TiDBer_RjzUpGDL | Original post link

The metadata lock is occupied.