Error When Stopping the Cluster

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 停掉集群报错

| username: TiDBer_Y2d2kiJh

[TiDB Usage Environment] Production Environment
[TiDB Version] v5.4.0
[Reproduction Path] Operations performed that led to the issue
tiup cluster stop whsmrx
[Encountered Issue: Issue Phenomenon and Impact]
When shutting down the cluster, it prompts Error: failed to stop: 10.212.16.100 node_exporter-9100.service, please check the instance’s log() for more detail.: timed out waiting for port 9100 to be stopped after 2m0s
This node is where I expanded ticdc
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

| username: 胡杨树旁 | Original post link

Check the specific logs, what do they show?

| username: TiDBer_Y2d2kiJh | Original post link

| username: TiDBer_Y2d2kiJh | Original post link

Is this log okay?

| username: 胡杨树旁 | Original post link

The image is not visible. Please provide the text you need translated.

| username: wakaka | Original post link

Manually killing the monitoring process is also fine, no problem.

| username: tidb菜鸟一只 | Original post link

Check if the node_exporter process is still running on the machine with IP 10.212.16.100 by using the command: netstat -anp | grep 9100

| username: DBRE | Original post link

Check if the SSH trust relationship from the TiUP machine to 10.212.16.100 is normal.

| username: TiDBer_Y2d2kiJh | Original post link

The monitoring on the entire cluster server is like this.

| username: WalterWj | Original post link

The first process on port 9100 appears to be occupied by a process started by root. It is likely that someone else started a node monitoring process, causing this issue.

To resolve it, either kill the process or change the TiDB monitoring port.

| username: tidb菜鸟一只 | Original post link

Change the port for node_exporter in the configuration file. The default port 9100 is occupied by the Zstack monitoring process.