Questions about Log Replication

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 日志复制疑问

| username: TiDBer_f4WLyH6T

TiKV write logic (one TiKV has two RocksDB instances, one for logs and one for data)

  1. After the client sends the request, it enters the propose phase.

  2. After proposing, the write request is turned into a RAFT_log (RAFT log includes region number and log) and appended, and the local log is persisted.

  3. The leader sends the log to the followers. After the followers receive the log and successfully write it, they return a message to the leader. When the majority of nodes return the message, the leader considers the write successful and enters the log commit phase.

  4. After the log commit, the data is written to the RocksDB KV store, and then the commit is applied.

Question: If TiKV crashes after the log commit but before the data is written to the RocksDB KV store, will the application return an error or a commit? If the application returns an error, does that mean data inconsistency? (The above write logic is organized by myself, not sure if it’s correct.)

2
| username: Billmay表妹 | Original post link
  1. After the client sends a write request, the request enters the Propose phase of TiKV, where the write request is submitted to the Raft protocol for processing.
  2. After the Propose phase, the write request is converted into a Raft log, which includes the Region ID and the specific operation log. These logs are appended to the Raft log and persisted locally.
  3. The Leader node sends these logs to the Follower nodes. After the Follower nodes receive and successfully write the logs, they send an acknowledgment message to the Leader node. When the majority of nodes return acknowledgment messages, the Leader node considers the write operation successful and enters the log Commit phase.
  4. After the log Commit phase, the data is written into the RocksDB KV database and the Commit operation is applied to ensure data consistency and durability.

Now to answer your questions:

  • If TiKV crashes after the log Commit but before the data is fully written to RocksDB KV, will the application return an error or a Commit?
    • In this case, TiKV will return an error to the application. This is because TiKV ensures data consistency and durability. If the data has not been fully written to RocksDB KV, TiKV considers the write operation incomplete and will return an error to the application instead of a Commit.
  • If the application returns an error, does it mean the data is inconsistent?
    • Yes, if the application receives an error response, it means the write operation was not successfully completed, and there may be data inconsistency. In this case, the application can handle the error accordingly, such as retrying the write operation or taking other recovery measures to ensure data consistency and integrity.
3
| username: TiDBer_f4WLyH6T | Original post link
  • If TiKV crashes after the log commit but before the data is fully written to RocksDB KV, will the application return an error or a commit?
    • In this case, TiKV will return an error to the application. This is because TiKV ensures data consistency and durability. If the data has not been fully written to RocksDB KV, TiKV will consider the write operation incomplete and will return an error to the application instead of a commit.
      Question: At this point, the data is not successfully written from the application’s perspective, but for TiDB, the data is considered fully written, right? (After log persistence, TiKV will perform crash recovery when it comes back up). Does this mean that having a flushback operation at the TiKV layer would be more reasonable?
4
| username: TIDB-Learner | Original post link

The application needs to make a judgment.

5
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.