TiDB Connection Passive Closure

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TIDB连接被动关闭

| username: TiDBer_YQ5s65BA

The application occasionally encounters connection exceptions: reporting “Connection must be valid and open.”
The TiDB server-side exception information is as follows:
[2023/12/21 16:46:14.535 +08:00] [WARN] [simple.go:2549] [“Kill failed: Received a 32bits truncated ConnectionID, expect 64bits. Please execute ‘KILL [CONNECTION | QUERY] ConnectionID’ to send a Kill without truncating ConnectionID.”] [connID=3823]

[2023/12/21 16:46:18.328 +08:00] [ERROR] [terror.go:315] [“encountered error”] [error=“write tcp XXXX:4003->XXXX:47643: write: connection reset by peer”] [stack=“github.com/pingcap/tidb/parser/terror.Log\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/parser/terror/terror.go:315\ngithub.com/pingcap/tidb/server.(*packetIO).writePacket\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/packetio.go:190\ngithub.com/pingcap/tidb/server.(*clientConn).writePacket\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:446\ngithub.com/pingcap/tidb/server.(*clientConn).writeChunks\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2388\ngithub.com/pingcap/tidb/server.(*clientConn).writeResultset\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2285\ngithub.com/pingcap/tidb/server.(*clientConn).handleStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2152\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1972\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1403\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1152\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:648”]

2
| username: 春风十里 | Original post link

The connection was disconnected, “connect reset by peer” means that the peer responded with a TCP RST (reset), terminating a connection. There are many reasons for this issue, such as network instability or the firewall sending several RSTs, which can also lead to similar “connection reset by peer” problems. Generally, specific packet capture and analysis are required.

3
| username: Jellybean | Original post link

This error indicates that the TCP connection to the TiDB server was closed by the other end, causing the TiDB server to encounter an error when returning data or responding to other requests.

Please check if there are any network jitters, if there are any keep-alive requests sent to the TiDB server, if there are any scenarios where the business logic closes the connection prematurely before data is returned, and any other possible issues.

4
| username: 江湖故人 | Original post link
  1. First, confirm whether the above IP is the application server that reported the error and whether the time matches.
  2. Start a long ping to monitor network jitter.
  3. Check if there is any health check tool like HAProxy in between. If there is, you can refer to this [1].
5
| username: dba远航 | Original post link

It feels like the connection was reset.

6
| username: zhanggame1 | Original post link

If the client is not directly accessing TiDB, check if there are proxy timeout settings or firewall timeout settings.

7
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.