Upgrade from TiUP 5.1.1 to 6.5.1 Fails Due to node_exporter Stop

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tiup 升级5.1.1到6.5.1 报node_exporter stop导致失败

| username: magdb

[TiDB Usage Environment] Production Environment
[TiDB Version] v5.1.1
[Reproduction Path] tiup cluster upgrade
[Encountered Problem: Phenomenon and Impact] During the upgrade of the TiDB version from v5.1.1 to v6.5.1 using tiup, an error occurred: Error: failed to start: 192.168.1.xxx node_exporter-9100.service timed out waiting for port 9100 to be started after 2m0s. After investigation, the issue was found to be that the node_exporter on one node was not upgraded. After copying the node_exporter from other nodes and replacing it on the problematic node, the node_exporter could be used normally. Currently, pd/tidb/tikv have all been upgraded to version v6.5.1, but prometheus/grafana are still at lower versions, and the cluster version is displayed as Cluster version: v5.1.1. How should I proceed to upgrade all nodes in the cluster to a unified version?
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

| username: Jellybean | Original post link

You can use the replay function of tiup, which allows you to continue upgrading the cluster in a manner similar to breakpoint resumption. Alternatively, you can upgrade the monitoring module separately.

| username: tidb菜鸟一只 | Original post link

You can refer to this: tiup cluster replay | PingCAP 文档中心

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.