Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.Original topic: 单机版本Tikv， org.tikv.common.exception.TiKVException: TimeOut Exceeded for current operation.
|
username: TiDBer_rr4Rq8SR
【TiDB Environment】 Testing/PoC
【TiDB Version】 5.4.3
【Reproduction Path】 Stress test, putKey
【Encountered Issue: Symptoms and Impact】
The TiKV client occasionally throws an error, but there are no errors on the server side.
org.tikv.common.exception.TiKVException: TimeOut Exceeded for current operation.
at org.tikv.common.util.ClientUtils.getTasks(ClientUtils.java:181)
at org.tikv.raw.RawKVClient.doSendBatchPut(RawKVClient.java:712)
at org.tikv.raw.RawKVClient.batchPut(RawKVClient.java:222)
at org.tikv.raw.RawKVClient.batchPut(RawKVClient.java:206)
2022-12-16 03:59:03,868 WARN SlowLogImpl - A request spent 6001 ms. start=03:58:57.852, end=03:59:03.853, SlowLog:{“trace_id”:14895108676224612315,“error”:“TimeOut Exceeded for current operation.”,“spans”:[{“event”:“batchPut”,“begin”:“03:58:57.852”,“duration_ms”:6001,“properties”:{“keySize”:“2”}},{“event”:“getRegionByKey”,“begin”:“03:58:57.852”,“duration_ms”:0},{“event”:“getRegionByKey”,“begin”:“03:58:57.852”,“duration_ms”:0},{“event”:“callWithRetry”,“begin”:“03:58:57.852”,“duration_ms”:1,“properties”:{“method”:“tikvpb.Tikv/RawBatchPut”}},{“event”:“gRPC”,“begin”:“03:58:57.852”,“duration_ms”:1,“properties”:{“method”:“tikvpb.Tikv/RawBatchPut”}}],“pd_addresses”:[“http://127.0.0.1:2379”],“cluster_id”:“7177588669324898328”}
PD log:
[2022/12/16 03:50:51.372 +00:00] [ERROR] [grpclog.go:75] [“transport: Got too many pings from the client, closing the connection.”]
[2022/12/16 03:50:51.372 +00:00] [ERROR] [grpclog.go:75] [“transport: loopyWriter.run returning. Err: transport: Connection closing”]
[2022/12/16 03:50:51.373 +00:00] [WARN] [grpclog.go:60] [“transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:2379->127.0.0.1:58442: use of closed network connection”]
[2022/12/16 03:50:51.391 +00:00] [ERROR] [grpclog.go:75] [“transport: Got too many pings from the client, closing the connection.”]
[2022/12/16 03:50:51.391 +00:00] [ERROR] [grpclog.go:75] [“transport: loopyWriter.run returning. Err: transport: Connection closing”]
[2022/12/16 03:50:51.391 +00:00] [WARN] [grpclog.go:60] [“transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:2379->127.0.0.1:58448: use of closed network connection”]
[2022/12/16 03:50:51.406 +00:00] [ERROR] [grpclog.go:75] [“transport: Got too many pings from the client, closing the connection.”]
[2022/12/16 03:50:51.406 +00:00] [ERROR] [grpclog.go:75] [“transport: loopyWriter.run returning. Err: transport: Connection closing”]
[2022/12/16 03:50:51.406 +00:00] [WARN] [grpclog.go:60] [“transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:2379->127.0.0.1:58454: use of closed network connection”]
Local disk performance:
[ec2-user@ip-172-31-4-185 tikvdata]$ dd if=/dev/zero of=/mnt/tikvdata/smalletst bs=1024K count=10240
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 14.7687 s, 727 MB/s
【Resource Configuration】
128G memory
【Attachments: Screenshots/Logs/Monitoring】
|
username: 
|
username: 
|
username: 
|
username: