TiKV Error: Unable to Connect Remotely, TiKV Fails to Start

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tikv报错无法远程连接 tikv启动不了

| username: Ann_ann

[Test Environment] Testing environment
[TiDB Version] v5.0.3
[Reproduction Path] Alarm for abnormal TiKV node, then unable to start with the following error:
[Encountered Problem: Phenomenon and Impact]
[FATAL] [server.rs:718] [“failed to start node: Engine(Other("[components/raftstore/src/store/fsm/store.rs:922]: \"[components/raftstore/src/store/peer_storage.rs:462]: [region 16713165] entry at apply index 6176 doesn\\\'t exist, may lose data.\""))”]

Seeking help from experts on how to resolve this issue. Thank you.

2
| username: lemonade010 | Original post link

Please share the architecture diagram. How is it deployed? 3 TiKV nodes?

3
| username: lemonade010 | Original post link

TiKV panic cannot start

  • 4.1.1 sync-log = false, after a power outage, the error unexpected raft log index: last_index X < applied_index Y appears. This is expected and requires recovery of the Region using the tikv-ctl tool.
  • 4.1.2 Deploying TiKV on a virtual machine, kill the virtual machine or a physical machine power outage, the error entries[X, Y] is unavailable from storage appears. This is expected, as the fsync of the virtual machine is unreliable and requires recovery of the Region using the tikv-ctl tool.
  • 4.1.3 Other reasons (unexpected, need to report a bug).
4
| username: Ann_ann | Original post link

It is 3tikv

image
image1380×578 237 KB

5
| username: lemonade010 | Original post link
  • 4.1.2 Deploy TiKV on a virtual machine, kill the virtual machine or power off the physical machine, and the entries[X, Y] is unavailable from storage error appears. This is expected, as the fsync of the virtual machine is unreliable, and the Region needs to be recovered using the tikv-ctl tool.
    Recover the region using the tikv-ctl tool.
6
| username: Ann_ann | Original post link

I’ll give it a try. Thank you.