Using tikv-client-java Version 3.3.5 to Access 4.0.xtidb Cluster Results in Unreachable Error

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 使用tikv-client-java 3.3.5版本访问4.0.xtidb集群，unreachable错误

| username: TiDBer_mEYNZXIU

[TiDB Usage Environment] Production Environment
[TiDB Version] 4.0.x
[Encountered Problem: Symptoms and Impact]
When using tikv-client-java version 3.3.5 to access the 4.0.x TiDB cluster, health check failures and TiKV store unreachable issues occur.

Health check failure:

2023-10-09 16:13:40.279 INFO  org.tikv.common.region.StoreHealthyChecker - fail to check TiKV health, regard as unhealthy. TiKV address: 172.20.216.22:20160
org.tikv.shade.io.grpc.StatusRuntimeException: UNIMPLEMENTED:
	at org.tikv.shade.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:287)
	at org.tikv.shade.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:268)
	at org.tikv.shade.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:175)
	at org.tikv.shade.io.grpc.health.v1.HealthGrpc$HealthBlockingStub.check(HealthGrpc.java:252)
	at org.tikv.common.region.StoreHealthyChecker.checkTiKVHealth(StoreHealthyChecker.java:112)
	at org.tikv.common.region.StoreHealthyChecker.checkStoreHealth(StoreHealthyChecker.java:86)
	at org.tikv.common.region.StoreHealthyChecker.run(StoreHealthyChecker.java:152)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

TiKV store unreachable when reading data:

2023-10-09 16:13:41.487 WARN  org.tikv.common.region.RegionManager - No TiKV store available for region: {Region[28102482] ConfVer[38479] Version[270249] Store[20752437] KeyRange[t\200\000\000\000\000\000\334\360_i\200\000\000\000\000\000\000\001\004\031\252\270\247\037\000\000\000\003\232\000\000\000\001\316\230c]:[t\200\000\000\000\000\000\334\360_i\200\000\000\000\000\000\000\001\004\031\252\270\2478\000\000\000\003\250\000\000\000\001\334\335\346]}
2023-10-09 16:13:41.496 INFO  org.tikv.common.region.RegionManager - Store 20752437 is unreachable, try to get the next replica
2023-10-09 16:13:41.496 WARN  org.tikv.common.region.RegionManager - No TiKV store available for region: {Region[28102482] ConfVer[38479] Version[270249] Store[20752437] KeyRange[t\200\000\000\000\000\000\334\360_i\200\000\000\000\000\000\000\001\004\031\252\270\247\037\000\000\000\003\232\000\000\000\001\316\230c]:[t\200\000\000\000\000\000\334\360_i\200\000\000\000\000\000\000\001\004\031\252\270\2478\000\000\000\003\250\000\000\000\001\334\335\346]}
2023-10-09 16:13:41.504 INFO  org.tikv.common.region.RegionManager - Store 20752437 is unreachable, try to get the next replica
2
| username: TiDBer_od5gmEYo | Original post link

We have encountered a similar issue. How much data do you have? We have over 1 billion records, and the client is unable to retrieve data and reports an “unreachable” error.

3
| username: Billmay表妹 | Original post link

Have you encountered this issue on the latest version of the cluster?

4
| username: Billmay表妹 | Original post link

This issue may be caused by the following reasons:

  1. Version compatibility issue: The tikv-client-java 3.3.5 version may not be fully compatible with the gRPC interface of TiDB 4.0.x version. Since TiDB may introduce some changes between different versions, it is recommended to use the tikv-client-java version that matches the TiDB version to ensure compatibility.
  2. Network connection issue: Health check failures and TiKV store being unreachable may be due to network connection problems. Please ensure that the client can normally access the IP address and port of the TiKV nodes and that the network connection is stable.

To resolve this issue, you can try the following steps:

  1. Confirm version compatibility: Check the documentation of tikv-client-java or the official release notes for version compatibility information to ensure that the version you are using is compatible with TiDB 4.0.x version.
  2. Check network connection: Ensure that the client can normally access the IP address and port of the TiKV nodes. You can try using other tools (such as ping or telnet) to test whether the network connection is normal.