Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: TiDB的分布式ingest是否考虑过只有部分region导入的情况？是如何解决的？
Recently, I’ve been studying the code related to RocksDB’s ingest and thought that TiDB also uses RocksDB at its core. I would like to ask, in a distributed scenario, is it possible for only some replicas to import data?
For example, on the Raft leader, the SST file is successfully ingested and added to the underlying files. However, on the follower, the ingest fails for some reason, and the data is not imported. At this point, the data on the leader and follower would be inconsistent.
The Raft protocol ensures that after a message is transmitted from the leader to N followers, N / 2 + 1 followers respond to the leader’s message commit (which is the majority), and then the leader will commit the message and proceed to the next processing stage. This ensures data consistency.
Your idea completely bypasses the Raft protocol, so it definitely cannot guarantee consistency. 
I haven’t read the TiDB code yet. Does TiDB ensure data consistency during ingest through Raft? One of my ideas is to first generate SST files and send them to each replica as a backup, then send an ingest trigger message to the Raft leader. Once each replica applies this message, it triggers the ingest operation. The discussion is that during the ingest, if some ingests succeed, it seems that Raft cannot guarantee this, right?
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
|
username: neo5j
|
username: 
|
username: