Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: tidb 升级到6.5.0 tidb无法启动，‘mysql.tidb_ttl_table_status’ doesn’t exist"
【TiDB Usage Environment】Production Environment
【TiDB Version】6.1.2 to 6.5.0
【Reproduction Path】Version Upgrade
【Encountered Problem: Issue Phenomenon and Impact】 After the upgrade, the TiDB component fails to start, and the log contains [WARN] [job_manager.go:144] [“fail to update table status cache”] [ttl-worker=manager] [error=“[schema:1146]Table ‘mysql.tidb_ttl_table_status’ doesn’t exist”]
【Resource Configuration】
【Attachments: Screenshots/Logs/Monitoring】
The issue has been resolved. The reason might be that the DDL statements were not cleared during the upgrade, causing the table structure of the MySQL table to fail to upgrade successfully, and TiDB could not start normally.
The solution is to manually patch a lower version of TiDB, start one node, and manually synchronize the table structure. After completion, restart TiDB.
What does it mean to not clear the DDL statement?
Check if there are any unfinished DDL statements using ADMIN SHOW DDL jobs. If there are, you need to cancel the corresponding DDL job using ADMIN CANCEL DDL JOBS job_id. Otherwise, it may lock the MySQL table, preventing the execution of MySQL-related table structure statements during the upgrade. For example, if tidb_ttl_table_status is not created, it will cause the TiDB server to fail to start.
Why upgrade to a lower version of TiDB?
Because the new version of TiDB requires the tidb_ttl_table_status table to start, and you didn’t create it successfully due to DDL blocking, the new version cannot start. You need to first start an older version of TiDB, create the tidb_ttl_table_status table, synchronize it to other nodes, and then start TiDB with the new version.
“The solution is to manually patch a lower version of TiDB, start one node, and manually synchronize the table structure. After completion, restart TiDB.”
Can you explain this in more detail?
Is it enough to start just one TiDB instance? Since the table structure is not stored on the KV instance, which KV instances need to be started? Also, does this TiDB instance need to be the DDL owner node?
There is currently an ongoing DDL operation, causing the updated DDL operation to be blocked.
Due to the failure in upgrading the table structure during the upgrade process, the new version of TiDB cannot be started, and only the previous version of TiDB can be used to start.
At that time, the phenomenon was that all TiKV nodes had started, but none of the TiDB nodes could start. My operation was to patch one of the nodes, then scale down all other TiDB nodes (to avoid manual DDL operations being transferred to the faulty TiDB node). After deploying a simple TiDB of the same version, I manually synchronized the MySQL table structure using Navicat. Finally, after scaling up all the TiDB nodes, they started normally.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
Recently, I handled some cases where TiDB could not start after upgrading from a lower version to version 6.5.x or above, reporting the error “Table ‘mysql.tidb_ttl_table_status’ doesn’t exist.”
There could be several scenarios:
First scenario:
- As the moderator mentioned above, you need to check thoroughly before upgrading to ensure there are no unfinished DDL executions. It is also not recommended to execute DDL during the upgrade, as it may cause unknown issues.
Second scenario:
Third scenario:
|
username: guoyanliang
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 