Error "get binlog event error: ERROR 1236 (HY000): Could not find first log file name in binary log index file" when using DM to synchronize from upstream MySQL to downstream TiDB

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb 上游mysql 下游tidb 使用dm同步,报错"get binlog event error: ERROR 1236 (HY000): Could not find first log file name in binary log index file

| username: 初来乍到

【TiDB Usage Environment】Production Environment
【TiDB Version】V7.5.0
【Reproduction Path】Operations performed that led to the issue
【Encountered Issue: Phenomenon and Impact】
【Resource Configuration】Enter TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
The issue is as follows: using DM to synchronize MySQL data to TiDB. The error message screenshot is as follows:
},
“subTaskStatus”: [
{
“name”: “order-migrate-other”,
“stage”: “Paused”,
“unit”: “Sync”,
“result”: {
“isCanceled”: false,
“errors”: [
{
“ErrCode”: 36069,
“ErrClass”: “sync-unit”,
“ErrScope”: “upstream”,
“ErrLevel”: “high”,
“Message”: “get binlog event error: ERROR 1236 (HY000): Could not find first log file name in binary log index file”,
“RawCause”: “”,
“Workaround”: “Please check if the binlog file could be parsed by mysqlbinlog.”
}
],
“detail”: null
},
“unresolvedDDLLockID”: “”,
“sync”: {
“totalEvents”: “12”,
“totalTps”: “0”,
“recentTps”: “0”,
“masterBinlog”: “(mysql-bin.087559, 1048413401)”,
“masterBinlogGtid”: “”,
“syncerBinlog”: “(mysql-bin.086999, 47836715)”,
“syncerBinlogGtid”: “”,
“blockingDDLs”: [

Checking the upstream MySQL, since MySQL’s binlog automatically clears binlogs older than one day, it should indeed have been cleared. Now I want to recover from the existing binlog in the upstream MySQL, such as mysql-bin.087449.
Attempted method, in the task configuration file


Specify the master’s binlog
Information after starting

【Attachments: Screenshots/Logs/Monitoring】

| username: 像风一样的男子 | Original post link

DM has a Relay Log feature, try enabling it

| username: 有猫万事足 | Original post link

The gist is that the corresponding binlog file name was not found in the index file under the upstream MySQL binlog directory.

You can check upstream to see if this is the case.

| username: WalterWj | Original post link

The upstream binlog has been cleaned up…

| username: 初来乍到 | Original post link

But it is indeed gone.

| username: 初来乍到 | Original post link

Yes.

| username: rebelsre | Original post link

To continue from the latest binlog position, the simplest way is to change the task name, or there is a parameter to clear the cache when starting the task. You can check the prompt when you start the task.
The problem is that doing this will result in data inconsistency between the two sides. It is recommended to perform a full + incremental synchronization again.

| username: TIDB-Learner | Original post link

I haven’t done it before, gaining experience!

| username: tidb菜鸟一只 | Original post link

No, your binlog is already missing in the middle, does this synchronization still make sense? Just do a full + incremental synchronization again…

| username: 小龙虾爱大龙虾 | Original post link

The best suggestion is to redo the full + incremental.

| username: vcdog | Original post link

In this situation, it’s better to redo it.

| username: DBAER | Original post link

The upstream binlog probably no longer exists. You can enable the DM relay log.

| username: Jellybean | Original post link

Since the binlog that needs to be synchronized has already been cleaned up, it is no longer possible to use the old binlog for synchronization. The solution is to re-establish the master-slave setup, export the full data to restore to TiDB, and then use the latest binlog to set up real-time synchronization.

| username: 连连看db | Original post link

Keeping binlogs for only 1 day is too short; at least 3 days are needed in production.

| username: TiDBer_rvITcue9 | Original post link

Resync it.

| username: Hacker007 | Original post link

Deleting data and resynchronizing is the only way to avoid data loss.