Conditions for Triggering TiKV Region Merge

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tikv region 合并触发条件

| username: TiDBer_HqvpID2q

[TiDB Usage Environment] Production Environment / Test / Poc
[TiDB Version] 6.5.3
[Reproduction Path] Operations that led to the issue:

  1. Used the TiKV Golang client v2.0.7 to write tens of millions of KV key-value pairs into the TiKV cluster.
  2. Deleted all keys.
  3. Used the transactional Iter interface to scan a specific range of keys, startKey= ff7969676f5c746573745c, endKey = ff7969676f5c746573745cff.

[Encountered Issue: Problem Phenomenon and Impact]

  1. The scanning process took a long time, almost 1 minute in total.
  2. By adding print statements in the TiKV client, it was found that the iter interface sent the getData method to over 1300 regions, but no matching keys were scanned, which is expected since they were all deleted.
  3. The issue is that these regions did not trigger a merge. Under what conditions will they merge? Do some configurations need to be made on PD?

Interestingly, when I used the pd_ctl tool to query the region where the start/end key is located, only one region was returned.

This contradicts the fact that over 1300 regions were accessed. What exactly is the problem?

Please enlighten me.

Update on the progress of this issue: First, I ran the GC example using the Golang client and did a simple test. The problem still exists, i.e., during the scan, it still scans over 1300 regions one by one but finds nothing.
Taking one of the regions as an example, 7979980, as shown below:

Using the PD tool, it can be found that there are keys in region 7979980, as shown below, with a space of 95MB and 204800 keys.

Let’s look at the number of MVCC keys in the region:

At first, I couldn’t understand why there were keys inside, but the scanned quantity was zero. Seeing the explanation on the second floor, it was mentioned that even after GC, there would still be a tombstone version of the key in MVCC. I guess the keys stored in the region now might be tombstone versions of the keys. If so, what operations can completely delete these tombstone keys, thereby triggering region merge and reducing the number of regions?

| username: 有猫万事足 | Original post link

Did you deploy TiKV instances separately without deploying TiDB instances?
If so, in this case, you need to manually trigger GC.

The cleanup of MVCC mainly relies on GC and compaction.
For a detailed explanation, you can refer to the reply below.

| username: zhanggame1 | Original post link

Manually compacting should resolve it.

| username: DBAER | Original post link

I am learning. Is this using TiKV alone?

| username: xfworld | Original post link

Three ways to meet your expectations:

  1. Add a TiDB node to automatically trigger the GC mechanism through the TiDB node.
  2. Use the command line method, manually triggering the GC mechanism via tikv-cli (single node or cluster).
  3. Customize and extend by combining TiKV Golang services to create an automatic GC trigger mechanism to meet this scenario’s requirements…
| username: zhaokede | Original post link

Large-scale deletions will also

| username: TIDB-Learner | Original post link

Automatically merge when it falls below a certain threshold.

| username: TiDBer_rvITcue9 | Original post link

Merge after GC reaches a certain threshold.

| username: Lystorm | Original post link

It means that if TiKV is deployed separately, both GC and region merging have to be done manually.

| username: 呢莫不爱吃鱼 | Original post link

Enable merging after GC reaches the threshold.

| username: zhang_2023 | Original post link

I’ve encountered merges as small as around 20MB, but there’s no exact parameter to control this, right?

| username: zhaokede | Original post link

Automatically activate after reaching the threshold.

| username: 友利奈绪 | Original post link

Useful, thumbs up.

| username: TiDBer_HErMeXDz | Original post link

Automatically merge when it falls below a certain threshold. Split when it grows to a certain extent.

| username: Hacker_xUwtuKxa | Original post link

First mark it, then take a closer look.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.