Connection Issues After Data Warehouse Crash Recovery

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 数仓宕机恢复后连接问题

| username: Jolyne

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
After the cluster experienced an abnormal crash due to storage issues and recovered, all nodes are running normally, but there is an error when connecting to TiDB.
[Encountered Issue: Problem Phenomenon and Impact]
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]
This is when connecting with haproxy

1693563099095
16935630990951380×226 50.7 KB

This is when directly connecting to port 4000
1693563512734
1693563512734959×73 18.6 KB

Using Navicat to connect reports this error
1693563561685

2
| username: TiDB_C罗 | Original post link

Post the HAProxy configuration file.

3
| username: Jolyne | Original post link

The image you provided is not accessible. Please provide the text you need translated.

4
| username: wenyi | Original post link

Directly connect to a specific TiDB instance without connecting to HAProxy.

5
| username: Jolyne | Original post link

It will report this error.

6
| username: TiDB_C罗 | Original post link

Check if haproxy is starting normally.

7
| username: wenyi | Original post link

It has nothing to do with HA, direct connection doesn’t work either.

8
| username: wenyi | Original post link

Which node failure caused the issue?

9
| username: TiDB_C罗 | Original post link

It should report the same error, but now it’s a different error.

10
| username: Jolyne | Original post link

Now direct connection works, but HA doesn’t.

11
| username: Jolyne | Original post link

The storage of the physical machine in the data center is broken, and the TiKV and TiDB virtual machines are on this physical machine.

12
| username: wenyi | Original post link

Remove the Chinese comments from ha and restart it.

13
| username: Jolyne | Original post link

Solved it by adding this to the configuration file, but it’s strange that it didn’t report an error before without it.

14
| username: redgame | Original post link

It’s good that it’s resolved, everyone has learned something.

15
| username: 大飞哥online | Original post link

Mine is empty too, but it doesn’t affect usage. I can access it normally.

16
| username: 大飞哥online | Original post link

It’s quite strange, hahaha.