Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.Original topic: Raft Engine 内存使用
|
username: Hacker_Niit6igX
Based on this issue, if the Raft Engine memory usage exceeds the limit, how will it be handled? For example, if there are a particularly large number of keys.
If Raft Engine’s memory usage exceeds the limit, it will attempt to free some memory through log compaction. If this is not sufficient to reduce memory usage, Raft Engine will stop accepting new data and start rejecting write requests until memory usage drops to an acceptable level.
Stopping the reception of new data might not be very friendly to database products. From the design of Raft Engine, high availability of in-memory indexes is not required. Why not simply use a lightweight KV store, such as LevelDB, to store the indexes? If high availability is not required, there is no need to flush the WAL, which would avoid an additional IO operation. The amount of in-memory indexes can be configured to have fewer levels in LevelDB, such as 2 or 3 levels, which would not impose a significant burden on the system’s CPU and IO resources. Raft Engine can be implemented as the lightest LSM KV separation.
Does Raft Engine have the concept of multiple tables? I couldn’t really find it. 
If Raft Engine’s memory usage exceeds the limit, TiKV will trigger Raft Engine’s GC mechanism to clear some unnecessary data from memory to free up space.
Raft Engine’s GC mechanism is log-based, meaning it frees up memory by deleting some old logs. Specifically, Raft Engine will merge some old logs into a new log and then delete the old logs to reduce memory usage.
If there are a particularly large number of keys, the situation where Raft Engine’s memory usage exceeds the limit may occur more easily. However, through the GC mechanism, TiKV can ensure that Raft Engine’s memory usage is always within a controllable range.
Will this Raft Engine do some caching? For example, after the primary synchronizes the log to the secondary, the secondary will not apply the log immediately (it hasn’t committed yet). So, when the secondary needs to apply the log next time, will it read it from the disk?
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
|
username: 
|
username: 
|
username: 
|
username: