Upstream MySQL executes DDL, DM synchronization to downstream TiDB reports an error

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 上游mysql 执行DDL,DM同步下游到TIDB报错

| username: LKLK

V5.4.1 cluster version, upstream MySQL executes DDL, DM synchronization to downstream TiDB reports an error, how can I solve it?
{
“result”: true,
“msg”: “”,
“sources”: [
{
“result”: true,
“msg”: “”,
“sourceStatus”: {
“source”: “xxxx”,
“worker”: “xxx”,
“result”: null,
“relayStatus”: null
},
“subTaskStatus”: [
{
“name”: “pay”,
“stage”: “Running”,
“unit”: “Sync”,
“result”: null,
“unresolvedDDLLockID”: “”,
“sync”: {
“totalEvents”: “15928709”,
“totalTps”: “49777”,
“recentTps”: “0”,
“masterBinlog”: “(mysql-bin.000021, 14947773)”,
“masterBinlogGtid”: “0-1453306-3023561378”,
“syncerBinlog”: “(mysql-bin.000019, 677578533)”,
“syncerBinlogGtid”: “0-1453306-3022111539”,
“blockingDDLs”: [
“ALTER TABLE snspaycenter.t_paymethod CHANGE COLUMN pm_show_name pm_show_name VARCHAR(60) CHARACTER SET UTF8 COLLATE utf8_general_ci NOT NULL DEFAULT _UTF8MB4’’ COMMENT ‘External Name’”
],
“unresolvedGroups”: [
{
“target”: “snspaycenter.t_paymethod”,
“DDLs”: [
“ALTER TABLE snspaycenter.t_paymethod CHANGE COLUMN pm_show_name pm_show_name VARCHAR(60) CHARACTER SET UTF8 COLLATE utf8_general_ci NOT NULL DEFAULT _UTF8MB4’’ COMMENT ‘External Name’”
],
“firstLocation”: “position: (mysql-bin.000019, 677578575), gtid-set: 0-1453306-3022111539”,
“synced”: [
snspaycenter.t_paymethod
],
“unsynced”: [
]
}
],
“synced”: false,
“binlogType”: “remote”,
“secondsBehindMaster”: “74983”
}
}
]
}
]
}

[Attachment]

Please provide the version information of each component, such as cdc/tikv, which can be obtained by executing cdc version/tikv-server --version.

| username: db_user | Original post link

Could you please clarify what this change is about? Is it changing the length of the varchar (shortening it) or changing the character set? You can execute the corresponding SQL on the downstream TiDB to see what error it reports. TiDB is not 100% compatible with MySQL at the moment.

| username: LKLK | Original post link

Changed several fields to not null
ALTER TABLE snspaycenter.t_paymethod CHANGE COLUMN pm_alias pm_alias VARCHAR(60) CHARACTER SET UTF8 COLLATE utf8_general_ci NOT NULL DEFAULT _UTF8MB4’’ COMMENT ‘Backend Name’;
ALTER TABLE snspaycenter.t_paymethod CHANGE COLUMN pm_show_name pm_show_name VARCHAR(60) CHARACTER SET UTF8 COLLATE utf8_general_ci NOT NULL DEFAULT _UTF8MB4’’ COMMENT ‘External Name’;

Error message when executing in TiDB

ERROR 1265 (01000): Data truncated for column ‘pm_show_name’ at row 1

| username: db_user | Original post link

Did this statement execute without errors downstream? It looks like an error-prone statement. If it executed successfully, it might be because a deadlock situation caused the execution to fail. You can try stopping and starting the task again to see if it reports an error. If it reports that the corresponding binary file cannot be found, it means the binary file retention time is relatively short, and you need to resynchronize.

| username: LKLK | Original post link

There was an execution error. How should we handle it now? Should we resynchronize this table? How do we resynchronize a single table? The entire database has a large amount of data, and it’s for online business.

| username: jerry | Original post link

If this table is a combination of multiple tables, should we consider whether some of the sub-tables do not have indexes, causing the issue?

| username: xiaohetao | Original post link

Check if it is related to precision.

| username: db_user | Original post link

You can skip the failed DDL statements

However, this will cause inconsistencies between the upstream and downstream table structures. It is best to pause the task, adjust the downstream table structure, and then skip the failed DDL.

| username: Hacker007 | Original post link

DDL statements can be skipped. After skipping, you can manually execute them downstream.

| username: LKLK | Original post link

I used handle-error test skip, and it prompted no errors… Strange.

| username: Hacker007 | Original post link

Still getting an error? Skip can only bypass DDL statements. If there are still exceptions, try using resume.

| username: LKLK | Original post link

Tried everything, still not working.

| username: LKLK | Original post link

I would like to ask if I rebuild the cluster, but the full synchronization of DM still uses the default character set and does not inherit the MySQL character set. Has this bug been fixed by the official team?

| username: Hacker007 | Original post link

The full amount will not be inherited.

| username: Hacker007 | Original post link

Each skip will only bypass one exception. Please check if there are multiple statement errors and try executing it several times.

| username: LKLK | Original post link

How should this be handled?

| username: LKLK | Original post link

The official fix has been made.

colation_compatible: “loose” # The method for synchronizing the default Collation in CREATE statements. Options are “loose” and “strict”, with “loose” being the default. The “loose” mode does not explicitly supplement the upstream default Collation, while “strict” explicitly supplements the upstream default Collation. When using the “strict” mode, if the downstream does not support the upstream default Collation, the downstream may report an error.

| username: Hacker007 | Original post link

No, incremental is possible. I’ve tested it on my side.

| username: Hacker007 | Original post link

Moreover, inheriting the collation from upstream will have many issues.

| username: system | Original post link

This topic was automatically closed 1 minute after the last reply. No new replies are allowed.