How to Limit Memory Usage When TiKV Memory Consumption is Too High

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TIKV占用内存过高，如何限制内存

| username: lfzihao

[TiDB Usage Environment] Testing
[TiDB Version] 6.5
[Reproduction Path] TIKV memory usage is too high
[Problem Encountered: Phenomenon and Impact]
Our company allocated a virtual server with 32GB of memory. After running for a while, the memory was fully occupied by TIKV, causing the server to freeze and even the monitoring couldn’t detect any content. As shown in the TIKV Memory Usage graph, there are interruptions in the line chart. We then increased the memory to 50GB, but it was still fully occupied. I used tiup cluster edit-config to modify memory-usage-high-water: 0.3 and storage.block-cache.capacity: 2G as shown, and then reloaded the cluster with tiup cluster reload. However, after some time, the memory was again fully occupied. How can I limit the memory usage of TIKV?

image

image
image803×275 14.1 KB

image
image907×571 21.6 KB

image
image877×759 17.2 KB

image

image
image920×435 28.7 KB

2
| username: 裤衩儿飞上天 | Original post link

You have deployed multiple components on one machine, right? For single-node deployment, you also need to consider TiDB server, PD, monitoring, etc., and make reasonable planning and allocation.

3
| username: lfzihao | Original post link

Resources are limited, and this is just for use during the development process. There won’t be high concurrency or anything like that. Is there any configuration related to memory resource limits that we can set for TiDB on a single machine?

4
| username: tidb菜鸟一只 | Original post link

If an OOM situation occurs, try adjusting the grpc-memory-pool-quota parameter.

5
| username: Billmay表妹 | Original post link

Deploy according to the official requirements as much as possible.

6
| username: TiDBer_jYQINSnf | Original post link

Check out this article: TiKV 内存参数性能调优 | PingCAP 文档中心

The underlying layer of TiDB is RocksDB, which has 4 column families (CF). Each CF has a memtable (equal to write-buffer-size * max-write-buffer-number) and a block cache (corresponding to [storage.block-cache]). By adjusting these to smaller values, the memory usage will be reduced.

7
| username: 裤衩儿飞上天 | Original post link

Refer to the configuration file parameter storage.block-cache.capacity for TiKV.
TiKV Configuration File Description | PingCAP Docs

8
| username: xingzhenxiang | Original post link

storage.block-cache.capacity: 15G
The image below shows the configuration location
image