DS Scheduling ETL Frequently Reports _mysql_connector.MySQLInterfaceError: table doesn't exist

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ds 调度ETL,经常报_mysql_connector.MySQLInterfaceError: table doesn’t exist

| username: TiDBer_vFs1A6CZ

[TiDB Usage Environment] Production Environment
[TiDB Version] 6.5.2
[Reproduction Path] truncate, multi-table join, insert
[Encountered Problem: Phenomenon and Impact] During the ETL process, the error _mysql_connector.MySQLInterfaceError: table doesn’t exist frequently occurs. Sometimes the ETL succeeds, and sometimes this error appears.
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]


| username: xfworld | Original post link

This is not a query, right? Also, are the database and table names correct? Does the user have sufficient permissions?

| username: tidb菜鸟一只 | Original post link

Is it possible that this table is frequently dropped and then recreated? The report says the table doesn’t exist, but there are times when it executes successfully. This should be the issue…

| username: TiDBer_vFs1A6CZ | Original post link

The ETL scheduled by ds sometimes executes successfully, and sometimes it fails. When it fails, it reports that the table doesn’t exist. Considering the successful executions, the possibility of the error direction you provided is very small.

| username: TiDBer_vFs1A6CZ | Original post link

No, drop and create operations need to go through the DBA. ETL will only involve truncate, multi-table join, and insert operations.

| username: xfworld | Original post link

DS strictly distinguishes between query and not query.
image


Is there no problem with this part?

| username: TiDBer_vFs1A6CZ | Original post link

We use ds to load our self-developed Python packages through Python to achieve ETL scheduling. We don’t have the scenario you mentioned.

| username: xfworld | Original post link

Then you need to carefully check the ETL process. The issues are not as described.

50% success, 50% failure? Then there must be a logical problem.

In short, the DDL operation truncate table will not cause the table to not exist.