DM Task Shows Status Over 1000

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm任务显示状态1000多

| username: Jjjjayson_zeng

【TiDB Usage Environment】Production Environment / Testing / PoC
【TiDB Version】
【Reproduction Path】What operations were performed that led to the issue
【Encountered Issue: Issue Phenomenon and Impact】
【Resource Configuration】
【Attachments: Screenshots / Logs / Monitoring】
image


However, the task shows as normal.

| username: db_user | Original post link

Take a look at the source, execute SHOW MASTER STATUS on the source database, and see what it looks like.

| username: Jjjjayson_zeng | Original post link

The image you uploaded is not visible. Please provide the text you need translated.

| username: Jjjjayson_zeng | Original post link

Looking at it like this now

| username: Jjjjayson_zeng | Original post link

Moreover, this error value keeps increasing. For example, it was 1150 before, and now it’s 1160.

| username: db_user | Original post link

Is the binary log for DM synchronization still 946? If it is, it’s possible that the primary database is processing a large transaction that hasn’t completed yet. You can analyze the binlog of 047 to check, or see how large the binlog of 947 is. If it’s too large, this process could also take a long time.

| username: Jjjjayson_zeng | Original post link

The image you provided is not visible. Please provide the text you need translated.

| username: dba-kit | Original post link

Did you previously enable the relay_log and then disable it? This caused the relay log pulled by DM to remain unchanged while the MySQL binlog kept increasing?

| username: Jjjjayson_zeng | Original post link

I don’t quite understand what you mean by relay_log. I only know start-relay.

| username: dba-kit | Original post link

Did you initially start the relay and let it run for a while, but then stop the relay for some reason?

| username: dba-kit | Original post link

Please take a few more screenshots and post the complete task-status.

| username: Jjjjayson_zeng | Original post link

It seems so… but I’m not very sure.

| username: Jjjjayson_zeng | Original post link

Sorry, I cannot translate images. Please provide the text content you need translated.

| username: dba-kit | Original post link

Sure enough, as I expected, you logged into this worker. Check the ${deploy-dir} directory, is there a relay_log directory?

| username: dba-kit | Original post link

If there are files in the directory, then it is as I said. There are two solutions to fix it:

  1. Delete the relay_log directory on the worker, then execute start-relay -s mysql-replica-09, and then restart the task.
  2. Use transfer-source to schedule the source to a new worker.

Both solutions will refresh the memory information and can resolve this type of alert.

| username: Jjjjayson_zeng | Original post link

It seems I didn’t see it.

| username: Jjjjayson_zeng | Original post link

The image is not visible. Please provide the text you need translated.

| username: Jjjjayson_zeng | Original post link

However, the relay is indeed not enabled.

| username: dba-kit | Original post link

What is the current task status? DM sometimes has bugs, and starting the relay during task execution may cause errors. You may need to stop-task & start-task to restart the task.

| username: Jjjjayson_zeng | Original post link

It looks normal now.