Why is the Insertion Performance of Virtual Machines So Low?

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 虚拟机的插入性能为什么那么低

| username: vincentLi

[Test Environment for TiDB] Testing
[TiDB Version] 6.5.3
[Reproduction Path]
A self-built TiDB cluster on a virtual machine, simulating a production environment on a single machine.
tiup bench tpch --sf=1 prepare
Insert test data, then check the slow query in the dashboard:

It’s a batch insert statement, click to view details:

Although the machine is a bit trashy, it still uses an M2 interface, so it shouldn’t be this bad when inserting test data. The backoff is quite large. I checked other posts, and they said it’s due to lock conflicts, but at most, the insert should only be a hot block, right? And the key point is that only one program is running, so how can it still be locked?

| username: wfxxh | Original post link

Are there other processes running on the hard drive?

| username: Kongdom | Original post link

First, follow the official documentation to perform an fio test and see the results. The best outcome would be achieving SSD-level performance.

| username: zhaokede | Original post link

The performance is probably still insufficient.

| username: 呢莫不爱吃鱼 | Original post link

The bucket effect depends on your overall configuration.

| username: TiDBer_7S8XqKfl | Original post link

Virtual machines do indeed affect performance; it should be much better on a physical host.

| username: 濱崎悟空 | Original post link

It will be a bit worse.

| username: Jellybean | Original post link

TiDB is designed for HTAP application scenarios in massive data storage environments and has certain hardware requirements. If you are testing and verifying on a virtual machine, you should mainly focus on functionality. Performance will only be unlocked when certain conditions are met.

| username: zhanggame1 | Original post link

Run an fio test on a hard drive and take a look.

| username: FarmerBob | Original post link

What are the specifications of your virtual environment? A single-node TiDB cannot outperform MySQL.

| username: xfworld | Original post link

What is the hardware configuration of the virtual machine? What is the key network configuration?

| username: vincentLi | Original post link

Thank you all, it seems that the virtual disk is full. I created a new 40G disk and reran the bench, and it actually used 20G of space.

| username: vincentLi | Original post link

This raises a question: when the system disk is insufficient, will it cause the TiDB cluster efficiency to decline? From the above issue, it seems that this problem can indeed occur.

| username: TiDBer_TQXaqJ6U-6236 | Original post link

Check if the CPU virtualization feature is enabled, and using an SSD might improve performance.

| username: YuchongXU | Original post link

Check the performance of the relevant disks.

| username: TiDBer_7S8XqKfl-1158 | Original post link

The performance of a virtual machine is closely related to its hardware configuration.

| username: awakening | Original post link

In a virtual machine environment, performance is not a consideration.

| username: 啦啦啦啦啦 | Original post link

The performance of a virtual machine depends on the configuration of its host machine. If the host machine uses NVMe drives, the performance should not be bad.

| username: FutureDB | Original post link

Insufficient disk, memory, and CPU will all affect the performance of the TiDB cluster.

| username: Kongdom | Original post link

:joy: When the system disk space is insufficient, no system can run. When we run out of space, even logging in is slow.