After adding a new PD node and removing the old one, the cluster status is normal and data writes are normal, but an error occurs when adding an index through the TiDB server. The error message indicates that it is still connecting to the removed PD node

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: pd添加了新节点后，剔除了老节点，集群状态正常，数据写入正常，但是发现通过tidb server添加索引报错，错误信息显示还是链接已剔除的pd节点

| username: shunzi

[TiDB Usage Environment] Production Environment
[TiDB Version] v7.1.3
[Reproduction Path] The original TiDB cluster had 3 old PD nodes. Added 3 new PD nodes and removed the original 3 old PD nodes. The cluster status is normal, and data writing is normal. However, an error occurs when adding an index through the TiDB server. The error message shows that it is still connected to the removed PD nodes.

CREATE INDEX licence_upload_log_code_IDX USING BTREE ON wac_heb_test.licence_upload_log (code);
[Encountered Problem: Phenomenon and Impact] Data writing is normal, but an error occurs when adding an index. The error message shows that it is still connected to the removed old PD nodes.

SQL Error [1105] [HY000]: pd address (192.168.212.32:2379,192.168.212.33:2379,192.168.212.37:2379) not available, error is Get “http://192.168.212.37:2379/pd/api/v1/config/cluster-version”: dial tcp 192.168.212.37:2379: connect: connection refused, please check network

Supplement: After adding the new PD nodes, the old PD leader node was removed first. After the PD leader node automatically transferred to the new PD, all the old PD nodes were removed.
[Resource Configuration] New PD nodes

image
image950×789 32.6 KB

[Attachment: Screenshot/Log/Monitoring]

2
| username: zhanggame1 | Original post link

Check the logs to see which component is reporting the error.

3
| username: shunzi | Original post link

The error reported when adding the index using the tool can be seen in the TiDB server.

4
| username: tidb狂热爱好者 | Original post link

Restart it.

5
| username: shunzi | Original post link

Restart the TiDB server node or the entire cluster?

6
| username: 小毛毛虫 | Original post link

Using tiup display to check the status of each component can be more accurate than the dashboard. For example, some components may be in a pending state after scaling in, and you need to execute the prune command to completely clean them up.

7
| username: shunzi | Original post link

The status is all normal, and it has been running stably for a week. Currently, the only issue is that adding an index doesn’t work.

8
| username: 我是人间不清醒 | Original post link

Try restarting the TiDB server first.

9
| username: zhanggame1 | Original post link

Just restart the TiDB server node.

10
| username: TiDBer_小阿飞 | Original post link

The TiDB server probably didn’t recognize it again, right? Did you restart the TiDB server? What changes were made to the configuration file?

11
| username: 小毛毛虫 | Original post link

Hmm, then try restarting it.

12
| username: 江湖故人 | Original post link

Analyze the error-level logs from the past week.

13
| username: shunzi | Original post link

Restarting the TiDB server role worked, thank you.

14
| username: wangccsy | Original post link

Can restarting solve it?

15
| username: 这里介绍不了我 | Original post link

Restarting solves the problem.

16
| username: 不想干活 | Original post link

Try restarting the TiDB server node.

17
| username: TIDB-Learner | Original post link

It seems that restarting is a universal remedy.

18
| username: dba远航 | Original post link

Try restarting.

19
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.