Stopping a TiKV Node with TiUP but the tikv-server on the Node Keeps Running

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tiup 停止一个tikv 节点 但是节点上tikv-server 一直不停止

| username: TiDBer_ZsnVPQB4

[TiDB Usage Environment] Production Environment
[TiDB Version] v6.5.0
[Reproduction Path]

[Encountered Problem: Phenomenon and Impact]
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]
Using tiup to stop a TiKV node, but the tikv-server on the node does not stop. It has been half an hour and it still hasn’t stopped. tiup display always shows the status as up. Is there any way to see what operations TiKV is performing and approximately how long it will take to shut down by itself? Otherwise, I can only use systemctl stop tikv, which is quick but seems to cause the TiDB node to get stuck, making it impossible to connect to TiDB using the mysql command.

| username: 裤衩儿飞上天 | Original post link

  1. Check the monitoring to see if the region leaders have been moved away from the nodes that are going to be stopped.
  2. Check the logs of the relevant nodes.
| username: tidb菜鸟一只 | Original post link

Check in Grafana to see if all the regions on the TiKV that is about to be taken offline have been transferred.

| username: TiDBer_ZsnVPQB4 | Original post link

It seems that after initiating the tiup cluster stop command, there was no region movement at all. Even after waiting for more than 10 minutes, it still wasn’t disconnected. Directly using systemctl stop on the machine immediately shut it down. After shutting down, upgrading the CPU and memory configuration, and restarting, the same operation was performed on another machine. Again, using systemctl stop on the machine immediately shut it down. After shutting down, upgrading the CPU and memory configuration, and restarting, it was only at 15:04 and 15:20 that the manual command caused movement.