Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: TiKV内存分析(storage.block-cache 只给了5G，但是总内存占用了46G，想知道如何才能将内存降下来)
The TiKV memory usage is explained here:
In production, with only 5G allocated to storage.block-cache, the total memory usage is 46G. I want to know how to reduce the memory usage. Specifically, is there a way to limit the size of the data buffer in memory for the second part of SELECT?
After version 7.5, TiKV memory analysis is supported. I checked and found that the main function entry only used 31%, while 61% was occupied by another clone.
Attached is the original file of the flame graph
profiling_2024-03-14_15-34-38.zip (107.4 KB)
Setting storage.block-cache.capacity to 5GB, but the total memory for TiKV can reach 46GB 
I feel like the memory is not being limited.
I feel the same way, it always seems counterintuitive, so I wanted to ask the experts to analyze it. What data in TiKV memory takes up the most space?
The new version defaults to a shared thread pool, and basically, storage.block-cache.capacity is the main part. You can consider this parameter *2.3 as the memory usage of TiKV.
This should be the memory part of RPC. For example, if the tidb-server receives data relatively slowly, it causes the data transmission between tikv and tidb-server to be slow, resulting in the data in the RPC part being stored in memory and unable to be released. There is a parameter for this.
Normally, this situation does not occur unless there is a problem with your hardware environment.
That shouldn’t be the case. I’ve tested it before, and it definitely can be restricted and won’t exceed the limit.
Generally, it can be restricted. Can I take a look at your deployment architecture?
It seems that I have found the reason. Because I didn’t actively set memory-usage-limit, it defaults to using 75% of the memory. After calculating, 56 * 0.75 = 42, which is not much different from the current 46G memory usage.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
|
username: dba-kit
|
username: 
|
username: 
|
username: