Upstream RDS MySQL Specification Change Causes Downstream DM Synchronization Error

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 上游RDS MySQL变更规格,下游DM同步出错

| username: TiDB_C罗

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]

"result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "ErrCode": 36069,
                                "ErrClass": "sync-unit",
                                "ErrScope": "upstream",
                                "ErrLevel": "high",
                                "Message": "get binlog event error: ERROR 1236 (HY000): Client requested master to start replication from position \u003e file size",
                                "RawCause": "",
                                "Workaround": "Please check if the binlog file could be parsed by `mysqlbinlog`."
                            }
                        ],
                        "detail": null

After the upstream change, I tried resume-task but it did not recover. Since the table is not large, I directly stopped the task with stop-task, then renamed the synchronized table in TiDB, and restarted the task with start-task. The question is, why can’t the synchronization continue after an upstream change (specification, master-slave switch) based on GTID?

| username: DBAER | Original post link

Check if the upstream binlog is still there.

| username: zhaokede | Original post link

It looks like the binlog file does not exist.

| username: Hacker007 | Original post link

The fastest way is to resynchronize the entire dataset.

| username: tidb菜鸟一只 | Original post link

You need to follow the installation documentation:

| username: TiDBer_QYr0vohO | Original post link

The binlog is gone, right?

| username: 呢莫不爱吃鱼 | Original post link

The error doesn’t mean that the binlog file no longer exists.

| username: 啦啦啦啦啦 | Original post link

RDS specification changes generally involve automatically creating a new replica with the new specifications. Once the data is synchronized, a master-slave switch is performed, and the original master is destroyed. You can handle it according to the master-slave switch scenario mentioned in the link above. If that doesn’t work, you can redo the synchronization.

| username: TiDBer_RjzUpGDL | Original post link

The binlog is gone.

| username: TiDB_C罗 | Original post link

My understanding is that since it is based on GTID, it should be able to find the correct Binlog file corresponding to the GTID when restarting the task, unless the binlog containing the GTID to be pulled is lost.

| username: TiDBer_JUi6UvZm | Original post link

Failed to obtain binlog, the reason is unknown.

| username: 舞动梦灵 | Original post link

If GTID is not used and there is a master-slave switch, then DM definitely needs to recreate the task and specify the binlog and pos values. With GTID, during a master-slave switch, it will find the corresponding binlog based on the GTID value. However, the binlog names will be different unless the master and slave binlog names are the same.

| username: TiDB_C罗 | Original post link

Sure enough, GTID is not enabled in the data source.

enable-gtid determines whether to use GTID to pull binlog from the upstream. The default value is false. Generally, manual configuration is not required. If the upstream database has GTID support enabled and master-slave switch is needed, set this configuration item to true.