TiKV Memory Leak

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tikv内存泄露

| username: wwb519

[TiDB Usage Environment] Production Environment / Testing / Poc
Production Environment
[TiDB Version]
v6.1.5
[Reproduction Path] What operations were performed to encounter the issue

[Encountered Issue: Problem Phenomenon and Impact]
TiKV by default uses no more than 75% of the server’s memory, but the limit exceeded 90%.

The memory-usage-limit parameter is set to 46G, but the actual usage reached over 54G.

SHOW config WHERE NAME LIKE ‘%rocksdb.defaultcf.block-cache-size%’; 16451MiB

  SHOW config WHERE NAME LIKE '%rocksdb.writecf.block-cache-size%';  9870MiB
   
    SHOW config WHERE NAME LIKE '%rocksdb.lockcf.block-cache-size%'; 1GiB
    
       SHOW config WHERE NAME LIKE '%raftdb.defaultcf.block-cache-size%'; 1GiB
       
       SHOW config WHERE NAME LIKE '%storage.block-cache.capacity%'; 28661MiB
       
        SHOW config WHERE NAME LIKE '%old-value-cache-memory-quota%'; 512MiB
        
         SHOW config WHERE NAME LIKE '%cdc.sink-memory-quota%'; 512MiB

[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]

| username: tidb菜鸟一只 | Original post link

SET config tikv storage.block-cache.capacity=‘40960MiB’;
Setting this parameter to 45% of the total memory generally won’t cause issues.

| username: wwb519 | Original post link

This value is limited and does not exceed 45%.

| username: zhang_2023 | Original post link

Set the parameter storage.block-cache.capacity.

| username: wwb519 | Original post link

This parameter is set to 45% by default. Do I need to explicitly set it?

| username: 这里介绍不了我 | Original post link

Setting config tikv storage.block-cache.capacity like this is more reliable.

| username: h5n1 | Original post link

Check TiKV Detail → Resolved TS → Lock heap size
Take a look at this.

| username: 路在何chu | Original post link

Isn’t this automatically set by the system? I haven’t managed it at all.

| username: wwb519 | Original post link

| username: h5n1 | Original post link

First, refer to 专栏 - TiKV主要内存结构和OOM排查总结 | TiDB 社区

| username: wwb519 | Original post link

I have already checked according to this article and found no abnormalities.