Does TiKV have a parameter similar to TiDB's tidb_server_memory_limit_gc_trigger?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tikv有类似tidb的 tidb_server_memory_limit_gc_trigger 参数吗？

| username: 逍遥_猫

Does TiKV have a parameter similar to TiDB’s tidb_server_memory_limit_gc_trigger? Does TiKV’s memory usage keep growing? Is there a parameter similar to TiDB’s tidb_server_memory_limit_gc_trigger?

2
| username: Billmay表妹 | Original post link
  1. memory-usage-limit: This parameter is used to control the memory limit of the TiKV instance. You can set an appropriate value based on the actual situation to ensure that TiKV does not consume too much memory resources. The value of this parameter should be configured according to the size of the block cache[1].
  2. block-cache-size: This parameter is used to set the block cache size of the TiKV instance. The block cache is a memory area used to cache data blocks, which can improve read performance. You can set an appropriate value based on the actual situation to balance memory usage and read performance[2].

In addition to the above parameters, you can further optimize TiKV’s memory usage through other configuration parameters, such as:

  • rocksdb.max-open-files: This parameter is used to set the maximum number of open files for RocksDB. If TiKV encounters the error “the maximum number of open file descriptors is too small” in some Kubernetes clusters, you can try increasing the value of this parameter[2].
3
| username: Jellybean | Original post link

Question 1: Firstly, in most cases, TiKV’s memory will not grow indefinitely. It will use memory based on the configured usage amount. There is a scenario where, when you read a large amount of data and the network returning to TiDB is slow, it will cause data accumulation. At this time, TiKV’s memory usage will spike, making it prone to OOM (Out of Memory).

Question 2: The commonly used configuration parameters related to TiKV memory are as listed above. For specific content and explanations, please refer to the official documentation.

4
| username: dba远航 | Original post link

Each component has parameters to control memory.

5
| username: 逍遥_猫 | Original post link

Setting the block-cache-size to 40% of physical memory, the value of memory-usage-limit changes with the value of block-cache-size, but the actual memory usage of TiKV keeps growing until OOM, and then continues to grow. From the Grafana monitoring, it can be seen that the usage of block-cache-size is always within the specified range. I am very puzzled, what exactly is causing the TiKV memory usage to keep growing?

6
| username: zhanggame1 | Original post link

To set TiKV memory, you only need to configure block-cache-size. According to my tests, the maximum memory usage of TiKV is 1.25 times the block-cache-size.

7
| username: 江湖故人 | Original post link

It seems that block-cache-size has been deprecated.

8
| username: 有猫万事足 | Original post link

Have these two parameters been modified? Use show config to check the results.

9
| username: zhanggame1 | Original post link

Indeed, it was written incorrectly; it should be block-cache.capacity.

10
| username: tidb菜鸟一只 | Original post link

SHOW config WHERE NAME LIKE ‘%block-cache.capacity%’
It shouldn’t be. After setting the block-cache.capacity size in TiKV, the memory usage of the TiKV process should not exceed 2.25 times the block-cache.capacity parameter. Could you provide the abnormal monitoring graph of TiKV at that time?

11
| username: dba远航 | Original post link

In the new version, block-cache.capacity has been removed.

12
| username: zhanggame1 | Original post link

Which version? Still using 7.5?