Will separating the data disk and WAL log disk improve TiDB write speed?

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 分开配置数据盘和wal日志盘, tidb写入速度会快吗

| username: zhanggame1

【TiDB Usage Environment】Production Environment / Testing / PoC
【TiDB Version】7.5.0
【Reproduction Path】What operations were performed to encounter the issue
【Encountered Issue: Problem Phenomenon and Impact】
The TiKV tuning documentation states:

Directory for RocksDB write-ahead logs. If there are two disks on the machine, you can place the RocksDB data and WAL logs on different disks to improve TiKV performance.

wal-dir = “/tmp/tikv/store”

Has anyone tested this? How much faster can it be?

| username: 江湖故人 | Original post link

Common physical servers typically use RAID1 for the system and RAID10 or RAID5 for data. In this environment, the performance of placing data and logs on two separate disks should be the same.

| username: dba远航 | Original post link

There will be an improvement, but it should not be significant.

| username: zhanggame1 | Original post link

If TiDB uses NVMe SSDs, it does not use RAID.

| username: tidb菜鸟一只 | Original post link

You can choose not to do it, but you can also do it if you want.

| username: zhanggame1 | Original post link

It’s very troublesome to do, the RAID card for NVMe SSD hardware RAID is quite special.

| username: tidb菜鸟一只 | Original post link

Oh, got it.

| username: zhaokede | Original post link

If you use an NVMe card, you probably won’t notice a difference.

| username: changpeng75 | Original post link

Instead of using two disks or two sets of RAID, it’s better to create a single RAID to improve performance and let the system handle scalability. The best practice recommendation is that TiKV should use SSDs. If it’s a SATA SSD, you can use RAID to improve performance. NVMe SSDs generally don’t need it, but if you have extreme performance requirements, you can still do it.

| username: TIDB-Learner | Original post link

It seems there will be an improvement, but the effect is not significant.

| username: yiduoyunQ | Original post link

Writing logs and writing data are different types of I/O operations. Therefore, parameters are provided to separately optimize these two types of I/O operations in depth. How much faster it will be depends on the specific business scenario. You need to test each scenario to see how different disks paired with different directories meet the performance and latency requirements of different businesses. There is no fixed conclusion.

As for whether to separate them, if the disk you are using is already top-notch with no bottleneck issues, then there is no need to separate them. Separating them would just mean adding another top-notch disk to store logs in a different place. More common scenarios are driven by cost-effectiveness considerations, where the disks used are not that top-notch. In such cases, various performance issues may arise with the disks, and the disks needed for writing and storing logs are not large. This provides an optional “local” acceleration.

Finally, traditional physical machine disks are fixed, and logically partitioning them at the upper level is just a superficial change. This wal-dir is more intended for use with cloud disks and centralized storage.

| username: yulei7633 | Original post link

Distributed read and write performance, theoretically, will be faster.

| username: TiDBer_lBAxWjWQ | Original post link

Unless there was a bottleneck before, the impact is probably minimal.

| username: 数据库真NB | Original post link

Not very useful.

| username: Soysauce520 | Original post link

It doesn’t make much sense, it’s better to have one more KV.

| username: zhanggame1 | Original post link

I am also wondering if having multiple TiKVs has any advantages.

| username: 小龙虾爱大龙虾 | Original post link

It will be faster, and putting the raft engine on the SSD will make it much faster.

| username: zhanggame1 | Original post link

Where is the installation path of the raft engine configured?

| username: Jellybean | Original post link

Could you please explain the principle of “fast”?

| username: 小龙虾爱大龙虾 | Original post link

I understand this is similar to Oracle redo. Isn’t redo recommended to use with fast disks?