Client Connection Error

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 客户端连接报错

| username: Jolyne

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
Using tools and external programs to connect constantly results in this error: 2013 - Lost connection to MySQL server at 'reading initial communication packet, system error: 0 “Internal error check (Not system error)”, sometimes it works, sometimes it doesn’t. However, using the command line has no issues (I personally feel it might be an issue with HAProxy).
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page



[Attachments: Screenshots/Logs/Monitoring]

| username: ealam_小羽 | Original post link

Refer to this:

| username: Jolyne | Original post link

At first, I didn’t configure the send-proxy parameter. Later, I configured it, and there were no issues for a while. Suddenly, there was a problem today, which is very strange.

| username: Jolyne | Original post link

I didn’t change anything, just restarted haproxy and the error stopped occurring :sweat_smile:

| username: tidb狂热爱好者 | Original post link

What kind of malfunction is this?

| username: ealam_小羽 | Original post link

Did someone shorten the connection time, and it got overwritten after the restart? :sweat_smile:

| username: caiyfc | Original post link

The almighty reboot

| username: zhanggame1 | Original post link

Sure enough, relying on a restart. It’s better to invest in F5.

| username: Jolyne | Original post link

It seems there isn’t any, so it’s very strange. It was configured before and hasn’t been touched, but this kind of problem still occurs.

| username: 我是咖啡哥 | Original post link

This morning I also encountered an issue with HAProxy…
ERROR 2013 (HY000): Lost connection to MySQL server at ‘reading authorization packet’, system error: 2
It was also resolved after a restart.

| username: cassblanca | Original post link

Restarting is good, but you shouldn’t casually restart the TiDB cluster.

| username: zhanggame1 | Original post link

HAProxy can be configured with several machines for high availability, and restarting should not be an issue.

| username: Jolyne | Original post link

Brother Coffee, how did you solve it? Are you still getting this error over there? I’m still occasionally getting this error on my end.

| username: 我是咖啡哥 | Original post link

I encountered this issue only once in the test environment, no problems in production.

| username: Jolyne | Original post link

:rofl: I’m cracking up. I encountered this issue in production, and it breaks once or twice from time to time. Whenever it happens, the ETL process reports an error.

| username: 有猫万事足 | Original post link

It should be related to the version of HAProxy, especially if your HAProxy was installed directly via yum or apt. In the past, I forgot which OS it was, but using the automatic installation would occasionally cause disconnections, and I was troubled by this issue.

The solution later was to recompile the latest version from GitHub. I strongly recommend checking the version of HAProxy. Look on GitHub to see if there are any corresponding issues.

| username: Jolyne | Original post link

Okay, thank you, I’ll give it a try.

| username: Jolyne | Original post link

I tried both 2.9 and 2.6, but the error still occurs. By the way, does this correspond to the cluster version? My cluster is 7.1.

| username: 有猫万事足 | Original post link

I checked with the person involved, and he doesn’t remember which version of HAProxy it was, but he is sure the OS was Ubuntu 20. There were issues with the apt installation, but they disappeared after compiling it.

Also, in your configuration, the maximum connection count of 1000 seems a bit low. You might want to try increasing it.

| username: Jolyne | Original post link

Mine is indeed an OS after Ubuntu 20 :rofl:. I’ll try to change it. Thanks.