How TiKV Nodes Perceive Each Other

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TIKV彼此间是怎么感知的

| username: TiDBer_bOR8eMEn

【TiDB Usage Environment】Production Environment
【TiDB Version】5.2.3
There are multiple TiKV nodes, and they can elect a leader. I want to know how they sense each other. Is it through a configuration file or some process? For example, if one node goes down, do the other two know about it, and how do they know?

| username: 普罗米修斯 | Original post link

When a TiKV node goes down, it is detected through the Raft election mechanism. The built-in Raft algorithm is used to achieve data consistency and high availability among multiple replicas.

| username: tidb菜鸟一只 | Original post link

They all interact with PD nodes, and PD manages the state between various TiKV nodes.

| username: 这里介绍不了我 | Original post link

Information collection, scheduling generation, and scheduling execution involving PD:
During information collection, TiKV nodes periodically report two types of heartbeat messages to PD: StoreHeartbeat and RegionHeartbeat:

  • StoreHeartbeat includes basic information about the store, capacity, remaining space, and read/write traffic data.
  • RegionHeartbeat includes the range of the region, replica distribution, replica status, data volume, and read/write traffic data. PD organizes and stores this information for scheduling decisions.
| username: DBAER | Original post link

TiKV and region leader will periodically report to PD.

| username: zhaokede | Original post link

The status of TiKV will be reported to PD; there is a Raft election mechanism between TiKVs.

| username: ti-tiger | Original post link

Through PD, they have heartbeat connections with PD.

| username: TI表弟 | Original post link

Originally, there is Raft. If it crashes, isn’t there still etcd to sense it?

| username: TiDBer_QYr0vohO | Original post link

Through PD’s

| username: 小于同学 | Original post link

Controlled by PD

| username: TIDB-Learner | Original post link

PD and etcd

| username: yytest | Original post link

Through PD, PD is the brain of TiDB.

| username: 数据库真NB | Original post link

There is a communication process for conducting elections, and the election results should also be stored in memory or a file, right?

| username: TiDBer_bOR8eMEn | Original post link

How does PD know there are three TiKV instances, and how can it see them?

| username: zhaokede | Original post link

Through PD, all information must be reported to PD.

| username: TiDBer_H5NdJb5Q | Original post link

The topology written during TiDB deployment will be stored in PD, and PD will schedule it through heartbeat.

| username: TiDBer_小阿飞 | Original post link

TiKV Server:
It is the core component in the TiKV cluster, responsible for data storage and processing. Each TiKV Server instance carries multiple Regions and performs data read and write operations based on the metadata of the Regions.

TiKV Client:
It is the interface between external applications (such as TiDB) and the TiKV Server in the TiKV cluster, responsible for handling various requests and forwarding them to the TiKV Server for execution.

Region:
It is the basic unit of data storage in TiKV. Each Region carries a specific segment of data and is distributed across different nodes in the TiKV cluster. The metadata of the Region, such as ID, range, and node information, is stored in PD. The TiKV Client queries PD to obtain the metadata of the Region to determine the node where the data is located.

Store:
It is the basic unit of storage instances in the TiKV cluster, including storage media such as disks and memory. Each Store is responsible for storing and managing a portion of the Region’s data, and each Store consists of one or more TiKV Server instances.

Raft Server:
It ensures data consistency and high availability by implementing the Raft protocol, enhancing the overall reliability and performance of TiKV.

Peer Instance:
Peer nodes achieve data replication and consistency maintenance through the Raft protocol, ensuring high availability, fault tolerance, and data consistency of the TiKV cluster.

What you mentioned is implemented through Raft.

| username: 友利奈绪 | Original post link

TiKV and region leaders will periodically report to PD, and PD determines their status.

| username: kkpeter | Original post link

The brain that connects the cluster is PD, everything is controlled by PD.

| username: 随缘天空 | Original post link

Through PD scheduling, TiKV will regularly report its heartbeat status to PD. If PD does not receive the heartbeat information from TiKV within a specified period, it will assume that the node is unavailable and will reassign the leader nodes on that node to other nodes for data read and write operations.