How to Optimize a TiDB Cluster in a Production Environment to Achieve a Single Table Write Rate of Over 20,000 Rows per Second

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 生产环境tidb集群如何调优实现单表写入速率在2w/s以上

| username: TiDBer_yBunUeUc

[TiDB Usage Environment] Production Environment
[TiDB Version] V7.4.0
Question: How to optimize to achieve a single table insertion rate of over 20,000/s
Configuration: The cluster consists of three machines, configured with 32GB memory, 32-core CPU, 500GB FC storage (HDD and SSD mixed), PD and TiKV are three nodes, TiFlash has 2 nodes

| username: tidb狂热爱好者 | Original post link

Perform hot and cold separation.

| username: wzf0072 | Original post link

TiKV automatically shards the underlying data, dividing all data into several Regions based on key ranges. When the size of a Region exceeds a certain limit, TiKV will split it into multiple Regions.

In scenarios with large amounts of data, there may be too many Regions, leading to increased resource overhead and performance degradation issues. With a fixed amount of data, the larger the Region, the fewer the number of Regions. Starting from v6.1.0, TiDB supports setting custom Region sizes. The default size of a Region is approximately 96 MiB, and increasing it can reduce the number of Regions.

Enabling Hibernate Region or Region Merge can also reduce the performance overhead caused by too many Regions.

Adjusting Region Size Using region-split-size

| username: wzf0072 | Original post link

If it involves high concurrent writes, you can refer to “TiDB High Concurrency Write Best Practices” TiDB 高并发写入场景最佳实践 | PingCAP 文档中心

| username: TiDBer_yBunUeUc | Original post link

Okay, thank you for your valuable opinion.

| username: wzf0072 | Original post link

| username: dba远航 | Original post link

Is the insertion method insert or load data?

| username: TiDBer_yBunUeUc | Original post link

The inserts were done using a synchronization tool.

| username: TiDBer_yBunUeUc | Original post link

Bach has already been limited, 4w is the best.

| username: wangccsy | Original post link

I’m a beginner here to learn. Have you managed to implement it?

| username: 有猫万事足 | Original post link

A small tip, it’s best not to use the DMR version in a production environment; you should use the TLS version.

| username: tidb菜鸟一只 | Original post link

Your cluster has three machines, and you actually deployed 2 TiFlash instances? Can you provide a topology diagram? I feel that resource contention should be quite severe…