TiDB client disconnected, connection cannot be released

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb 客户端断掉,连接无法释放

| username: TiDBer_OJg7HeFu

The TiDB connections cannot be released, causing the connections to be full and the service to crash.

| username: xfworld | Original post link

You can track the top SQL through the dashboard to see which SQL queries are slow and consuming resources, and then optimize them.

| username: zhanggame1 | Original post link

What exactly does it mean that the connection cannot be released?

| username: TiDBer_OJg7HeFu | Original post link

Sorry, I can’t translate images. Please provide the text you need translated.

| username: TiDBer_OJg7HeFu | Original post link

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:836)
at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:456)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance$original$cIghTEv0(ConnectionImpl.java:246)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance$original$cIghTEv0$accessor$piZyFsI3(ConnectionImpl.java)
at com.mysql.cj.jdbc.ConnectionImpl$auxiliary$3DwVSHHx.call(Unknown Source)
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.StaticMethodsInter.intercept(StaticMethodsInter.java:83)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect$original$n4kNkdPl(NonRegisteringDriver.java:197)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect$original$n4kNkdPl$accessor$ODCq0Wvr(NonRegisteringDriver.java)
at com.mysql.cj.jdbc.NonRegisteringDriver$auxiliary$8pSE0BcZ.call(Unknown Source)
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:86)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java)
at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1657)
at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1723)
at com.alibaba.druid.pool.DruidDataSource$CreateConnectionThread.run(DruidDataSource.java:2838)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.GeneratedConstructorAccessor89.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91)
at com.mysql.cj.NativeSession.connect(NativeSession.java:144)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:956)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:826)
… 14 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:155)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65)
… 17 common frames omitted

| username: zhanggame1 | Original post link

While you can’t connect here, try connecting with other clients.

| username: TiDBer_OJg7HeFu | Original post link

The tool cannot connect either.

| username: Hacker007 | Original post link

Restart the TiDB node.

| username: TiDBer_OJg7HeFu | Original post link

After restarting, high concurrency comes in and it doesn’t work again.

| username: Hacker007 | Original post link

Find a way to prevent requests from coming in. For example, change the password of the user initiating the request, and change it back after optimization.

| username: TiDBer_OJg7HeFu | Original post link

TiDB can support how many connections? It seems like there’s no connection pool reuse effect, and the connections get maxed out quickly. It’s quite disappointing.

| username: zhanggame1 | Original post link

When you can’t connect, use netstat to check how many connections there are in the system. Also, check the memory and CPU status.

| username: TiDBer_OJg7HeFu | Original post link

| username: Hacker007 | Original post link

The default maximum connections are 151, you can try setting it higher.

| username: Hacker007 | Original post link

show variables like '%max_connections%';
| username: MrSylar | Original post link

Intuitively, your tidb-server’s memory usage is too high, causing the tidb-server to not establish new connections.

| username: TiDBer_OJg7HeFu | Original post link

Isn’t the default 0 supposed to mean unlimited connections?

| username: Hacker007 | Original post link

Change it, it’s also my first time seeing it as 0.

| username: Hacker007 | Original post link

Was it changed by someone?

| username: zhanggame1 | Original post link

The default is 0 unless someone has changed it.