Failed to start TiDB node after scaling out TiDB

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb 扩容 tidb节点,无法启动

| username: TiDBer_pJWSVFwH

Looking at the scaling code:

tidb_servers:

  • host: 192.168.110.156

After scaling, the TiDB on 156 cannot start:

log:
[2023/06/27 17:19:39.985 +08:00] [INFO] [region_cache.go:2486] [“[health check] check health error”] [store=192.168.110.41:20161] [error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 192.168.110.41:20161: connect: connection refused"”]
[2023/06/27 17:19:40.984 +08:00] [INFO] [region_cache.go:2486] [“[health check] check health error”] [store=192.168.110.41:20161] [error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 192.168.110.41:20161: connect: connection refused"”]
[2023/06/27 17:19:41.511 +08:00] [INFO] [client_batch.go:581] [“batchRecvLoop re-create streaming fail”] [target=192.168.110.41:20161] [forwardedHost=] [error=“context deadline exceeded”]
[2023/06/27 17:19:41.511 +08:00] [INFO] [client_batch.go:581] [“batchRecvLoop re-create streaming fail”] [target=192.168.110.41:20161] [forwardedHost=] [error=“context deadline exceeded”]
[2023/06/27 17:19:41.524 +08:00] [INFO] [client_batch.go:581] [“batchRecvLoop re-create streaming fail”] [target=192.168.110.41:20161] [forwardedHost=] [error=“context deadline exceeded”]
[2023/06/27 17:19:41.524 +08:00] [INFO] [client_batch.go:581] [“batchRecvLoop re-create streaming fail”] [target=192.168.110.41:20161] [forwardedHost=] [error=“context deadline exceeded”]
[2023/06/27 17:19:41.985 +08:00] [INFO] [region_cache.go:2486] [“[health check] check health error”] [store=192.168.110.41:20161] [error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 192.168.110.41:20161: connect: connection refused"”]
[2023/06/27 17:19:42.986 +08:00] [INFO] [region_cache.go:2486] [“[health check] check health error”] [store=192.168.110.41:20161] [error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 192.168.110.41:20161: connect: connection refused"”]
[2023/06/27 17:19:43.984 +08:00] [INFO] [region_cache.go:2486] [“[health check] check health error”] [store=192.168.110.41:20161] [error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 192.168.110.41:20161: connect: connection refused"”]
[2023/06/27 17:19:44.984 +08:00] [INFO] [region_cache.go:2486] [“[health check] check health error”] [store=192.168.110.41:20161] [error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 192.168.110.41:20161: connect: connection refused"”]
[2023/06/27 17:19:45.983 +08:00] [INFO] [region_cache.go:2486] [“[health check] check health error”] [store=192.168.110.41:20161] [error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 192.168.110.41:20161: connect: connection refused"”]

| username: zhanggame1 | Original post link

Access TiKV at 192.168.110.41:20161 to check if machine 156 can reach port 20161 on machine 41 and whether there is a firewall blocking it.
connect: connection refused

| username: TiDBer_pJWSVFwH | Original post link

The firewall has been turned off, and telnet can connect.

| username: zhanggame1 | Original post link

On 41, use netstat -na | grep 192.168.110.156 to check if there are any incoming connection records.

| username: redgame | Original post link

Network connection issues, storage offline

| username: Anna | Original post link

Check if an SSH key has been set up.

| username: xingzhenxiang | Original post link

Is the firewall enabled?

| username: TiDBer_pJWSVFwH | Original post link

I previously thought the firewall was turned off, but after checking again, I found it wasn’t. Embarrassing…

| username: xingzhenxiang | Original post link

Just disable it directly to avoid future trouble.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.