Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.Original topic: Chaos mesh的读负载和写负载如何影响数据库的性能
|
username: TiDB青橙
How to affect the performance of the database, for example: MySQL’s
|
username: TiDB青橙
|
username: The situation is like this: I want to use Chaos Mesh to create a fault that affects the MySQL database. I found that read load and write load are the most likely anomalies to affect the MySQL database. However, after encountering read load and write load anomalies, I found that I couldn’t achieve the desired effect. So I want to ask if there is something wrong with my settings. I didn’t set the path, and the number of threads is set to 100, with the data volume set to 1000GB. Or is there any other way to affect the database’s performance?
Take a look at this valuable content to see if it helps you in any way~
I have an issue with Chaosd. Most of it is related to k8s. I also read the Chaosd documentation and felt that it was a bit too simplistic.
|
username: Describe the specific issue, such as what command you executed, what you expected, and what the actual result was.
I want to use Chaos Mesh to create a fault that affects the MySQL database. I found that read load and write load are the most likely anomalies to affect the MySQL database. However, after creating read load and write load anomalies, I found that I couldn’t achieve the desired effect [effect: being able to feel the page loading slowly or query speed slowing down on the page]. So I want to consult whether there is something wrong with my settings. I didn’t set the path, and the number of threads is set to 100, and the data volume is set to 1000GB. Or is there any other way to affect the database performance?
Your question is too abstract to answer.
Here is the doc for chaosd: Chaosd Introduction | Chaos Mesh. It contains specific scenarios. You can mention which scenario you used, which command you executed, what the result was, and what your expectation was.
Using the command chaosd attack disk add-payload read -s 1000G -n 7 -p /data/zero, I want the effect to be: impacting MySQL’s read performance, so that when operating on the page, the page loading speed is noticeably slower.
|
username: First of all, MySQL reads are generally from memory, and physical reads are relatively rare. Then, if you want to achieve the effect of page stuttering, I think it’s not easy to measure it with read and write loads. How much read load can loading information on a single page generate?
Secondly, you need to know the read and write speed of your disk, the read and write speed of MySQL under current configuration at full load, and the load speed that Chaos Mesh can cause. Suppose the disk write speed is 500MB/s, MySQL’s write speed at full load is 200MB/s, and the load caused by Chaos Mesh is 300MB/s or more, only then can you see a decline in MySQL’s write performance.
No, I was giving this path as an example because I found during testing that this path can only specify the path of a specific file, not a folder. This is one of my questions as well. How should I fill in this path?
Confirmed, there are two points to clarify:
- Currently, the Kubernetes version of Chaos supports disk latency injection, refer to Simulate File I/O Faults | Chaos Mesh, while the physical machine version Chaosd does not support disk latency injection.
- The implementation method of chaosd attack disk add-payload is essentially the dd command, refer to chaosd/pkg/core/disk.go at v1.4.0 · chaos-mesh/chaosd · GitHub, read uses iflag=dsync,fullblock,nocache, refer to chaosd/pkg/core/disk.go at v1.4.0 · chaos-mesh/chaosd · GitHub.
Therefore, chaosd may not meet your needs.
Same as above, the chaosd attack disk add-payload command uses the dd command at the underlying level, so the parameter needs to be a specific file, not a folder.
Well, I am assessing that it is most likely a read-write load. If this fault is not possible, are there any other faults in Chaos Mesh that can affect MySQL performance?
You can either use chaosd to simulate CPU and memory, which uses stress, or you can directly use stress to simulate it, there’s no difference. Alternatively, you can find other tools to simulate disk latency or throughput limits. The functionality of chaosd is limited, and there are more tools available on physical machines.
Well, if that’s the case, then it is not supported in several aspects.
The spike in CPU and memory doesn’t seem to directly affect MySQL’s performance. From what I see on the server, the CPU is maxed out, but page refreshes and other actions are still relatively smooth. May I ask if TiDB has specific tests to evaluate the impact of machine performance and to test the robustness of TiDB?
Both CPU and memory being fully utilized will affect performance. However, in your scenario, you want the page to lag. For the database, the resources needed for page refresh are minimal, so the impact won’t be very noticeable. You can use network anomalies to test the lagging effect on the page, but this is unrelated to the database.
For internal testing using the Kubernetes version, Chaosd might simulate network-related issues. The simplest way to address page lag at the database level could be to simulate business locks, for example?