TiDB Cluster Error: Too Many Connections

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiDB集群报错:too many connections

| username: seiang

[TiDB Usage Environment] Production Environment
[TiDB Version] V6.5.1
[Reproduction Path]
[Encountered Problem: Problem Phenomenon and Impact]
[Resource Configuration]
The business side reported that they couldn’t connect to the TiDB cluster, with the error “too many connections.” Checking the TiDB Server node logs, there were also many “too many connections” errors, as shown below:

From the error information, the max_connections parameter is set to 15, but when I logged into the TiDB Server node and checked the parameter, it was not set. The variable value is 0, which means there is no limit on the number of client connections.
image

image

Can any experts help me figure out the reason? Is this a bug?

| username: 啦啦啦啦啦 | Original post link

There are several TiDB nodes, and the TiDB node that might report an error is not the same as the one you used for show variables. I remember that setting this parameter only takes effect on the current TiDB node.

| username: caiyfc | Original post link

Try editing the config to see if there are any settings.

| username: liyuepeng123 | Original post link

Check the max-server-connections parameter.

| username: oceanzhang | Original post link

Is the max connections setting on that server set too low?

| username: oceanzhang | Original post link

I remember this parameter should follow the node, not a global parameter.

| username: 啦啦啦啦啦 | Original post link

It looks like this TiDB instance has a limit of 15 connections. Try connecting to this TiDB instance separately and check the max_connections setting to see if it’s set to 15. Previously, were you connecting through a load balancer? You might have been seeing the parameters of other nodes.

| username: seiang | Original post link

When checking the config with edit config, it also shows that no settings have been made. The error is only occurring on that one node, while the other nodes are functioning normally.

| username: seiang | Original post link

The node I logged into reported “too many connections.” When I logged into each TiDB Server node to check, the default value was 0.
image

| username: seiang | Original post link

It seems that the max-server-connections parameter is no longer available in version 6.5.1.

| username: 啦啦啦啦啦 | Original post link

There is this parameter, but it seems that you can’t directly check it with show config. Try using tiup cluster edit-config to see if max-server-connections is set. If this is the reason, you can directly set a very high value for max_connections on this TiDB node, such as 10000. This might solve the issue because max_connections has a higher priority. You can give it a try.

| username: 路在何chu | Original post link

Does this affect the business?

| username: seiang | Original post link

There are no connection-related configuration parameters when viewing with tiup cluster edit-config, so this issue is strange.

| username: seiang | Original post link

It will definitely affect the business, and the business will also report an error: too many connections.

| username: 啦啦啦啦啦 | Original post link

Setting max_connections=10000, will this work?

| username: seiang | Original post link

Currently, I haven’t made any adjustments, and now the number of connections to this node can exceed 15.

| username: 春风十里 | Original post link

Besides the error log, will the actual connection fail?

| username: seiang | Original post link

Yes, not only does the TiDB Server node report an error, but the business application is also unable to connect.

| username: 小龙虾爱大龙虾 | Original post link

Check the operating system logs of the problematic node for any information.

| username: seiang | Original post link

From the operating system logs of the problematic node, no anomalies were found.