DM startup has no response

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm启动没有任何响应

| username: 宸凡_22

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration] Enter TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]
Synchronizing a database, starting DM, no response at all, not sure where to start? Both start-task and check-task are the same.
tiup dmctl --master-addr 10.10.110.173:8261 check-task dtudm-task.yaml

| username: db_user | Original post link

You can first telnet 10.10.110.173 8261 to check if both upstream and downstream are accessible, and then check the DM logs for any issues.

| username: 宸凡_22 | Original post link

Where can I view the DM logs?

| username: 宸凡_22 | Original post link

On this dm-work machine, other databases are running normally.

| username: 小龙虾爱大龙虾 | Original post link

Is the command stuck, or what’s the situation?

| username: tidb菜鸟一只 | Original post link

tiup dm display dm-test
Then go to the corresponding host at /u01/tidb/dm/deploy, and check the logs of the corresponding worker in the deploy directory.

| username: 宸凡_22 | Original post link

This state

| username: Hacker007 | Original post link

Check the dm-work.log log of node 173.

| username: okenJiang | Original post link

How long is it stuck? It’s possible that the check-task is running, and this command is synchronous, so it might take a bit longer.

Let’s check the DM logs first.

| username: andone | Original post link

Please attach the specific logs.

| username: 小龙虾爱大龙虾 | Original post link

It feels like waiting for a network timeout. The program shouldn’t take that long to run. Try executing the previous telnet command on all dm-work nodes again.

| username: 小龙虾爱大龙虾 | Original post link

No, what did you telnet earlier? You need to telnet the upstream MySQL address and the downstream TiDB address.

| username: 宸凡_22 | Original post link

The above is the upstream database, and the below is the downstream database.

| username: 宸凡_22 | Original post link

I checked the logs, and there is no log for this task. It seems that the log has not been written yet.

| username: 小龙虾爱大龙虾 | Original post link

Wait a little longer and see.

| username: 宸凡_22 | Original post link

I’ve been waiting for several days. Normally, there would definitely be feedback soon.

| username: 小龙虾爱大龙虾 | Original post link

How about using strace to check?

| username: 宸凡_22 | Original post link

The image you provided cannot be processed for translation. Please provide the text content directly for translation.

| username: 小龙虾爱大龙虾 | Original post link

Run the following command, don’t use tiup to call dmctl, write the full path directly:

strace -ttt -T -f -o /tmp/strace.out [full path to dmctl] --master-addr 10.10.110.173:8261 check-task dtudm-task.yaml

Then check the contents of the strace.out file and analyze it.

| username: 宸凡_22 | Original post link

strace.out (871.4 KB)