TiKV Parameters periodic-compaction-seconds and TTL

translator_bot · June 21, 2024, 7:21am

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tikv参数periodic-compaction-seconds和ttl

| username: chenhanneu

【TiDB Usage Environment】Production Environment / Testing / Poc
【TiDB Version】
【Reproduction Path】What operations were performed when the problem occurred
【Encountered Problem: Problem Phenomenon and Impact】
【Resource Configuration】Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
【Attachments: Screenshots/Logs/Monitoring】

After upgrading from 6.5.3 to 6.5.4, two new parameters were added. Do they need to be adjusted?
Compatibility changes in 6.5.4:

Periodic compaction of RocksDB is disabled by default, making the default behavior of TiKV RocksDB consistent with versions prior to v6.5.0, avoiding a large amount of compaction after the upgrade that could affect system performance. Additionally, TiKV introduces the rocksdb.[defaultcf|writecf|lockcf].periodic-compaction-seconds and rocksdb.[defaultcf|writecf|lockcf].ttl configuration options, supporting manual configuration of RocksDB’s periodic compaction #15355 @LykxSassinator

translator_bot · June 21, 2024, 7:21am

| username: dba远航 | Original post link

Let’s use the default settings for the newly added parameters; we probably don’t need to worry about them.

translator_bot · June 21, 2024, 7:21am

| username: 路在何chu | Original post link

For now, don’t adjust the parameters arbitrarily. Many of TiDB’s parameters are automatically adjusted based on the machine configuration.

translator_bot · June 21, 2024, 7:21am

| username: 路在何chu | Original post link

If adjustments are indeed necessary, we can make them later based on monitoring alerts.

translator_bot · June 21, 2024, 7:21am

| username: 有猫万事足 | Original post link

The parameter periodic-compaction-seconds is not worth researching. If you look under rocksdb now, you can still see many issues.

The ttl parameter seems reliable. In the link above, when the periodic-compaction-seconds parameter doesn’t work, it is recommended to use the ttl parameter instead. However, I can’t figure out why you need to periodically compact the SST files to the bottom layer.

Once SST files are compacted to the bottom layer, the most obvious benefit is the deletion of some expired keys, reducing file size. Is it because storage is running out?

The downside is that when SST files are compacted to the bottom layer, there will be write amplification. If there is no estimation of the data volume and the impact range of SST compression, it can easily lead to periodic slow write queries. I’m also not sure if RocksDB’s TTL operation has a running table/log like TiDB’s TTL that can be checked. If something goes wrong, it will be very troublesome. It is not recommended to modify it.