Is slow concurrent reading related to CPU core binding?

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 并发读慢会和绑核有关吗

| username: TiDBer_uI8QIp1t

[TiDB Usage Environment] Production Environment
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]
Currently, there are 3 servers with 64GB RAM, 16 cores, and 500GB each, each having PD, KV, and DB. CPU binding with numactl is not used. The CPU usage on all three servers is not high, possibly up to 10% at most. The memory usage reaches 50% after running for a while. The speed for a single table is still acceptable, but if there are a dozen requests on the interface, each querying the database, the responses come in pairs, whereas MySQL can handle a dozen simultaneous responses. Are there any configurations that can improve this situation?

| username: caiyfc | Original post link

It’s hard to judge just by the description. You can check out the following article, which includes performance comparisons under different deployment scenarios:
Column - How to Play with TiDB on a Single Machine with 8 NUMA Nodes - Exploring the Optimal Deployment Topology of TiDB Cluster on AMD EPYC Servers | TiDB Community

| username: zhanggame1 | Original post link

For now, don’t worry about binding cores with numctl. Check which SQL queries are slow without running any queries.

| username: Miracle | Original post link

Based on my tests in k8s (with CPU pinning), allocating too few CPUs can affect concurrent performance, even if CPU usage is not high.
Just my experience, for reference only.

| username: 江湖故人 | Original post link

Refer to this 《TiDB Performance Optimization Practice》

| username: Jellybean | Original post link

Based on your description, it seems you are asking why request access to TiDB becomes slow. The issue turns into a slow query analysis scenario. There are many resources in the community and official documentation on how to analyze slow queries, you can search for corresponding optimization methods.

If you want the community to assist with specific analysis, you should post detailed cluster monitoring, slow query SQL, operation logs, etc. The experts in the community will help you brainstorm solutions.

| username: TiDBer_uI8QIp1t | Original post link

It’s not that the queries are getting slower; when queried individually, they are quite fast. It’s just that pages with multiple interfaces get blocked.

| username: Jellybean | Original post link

It is recommended to conduct a full-link latency analysis to clarify the situation when there are many interfaces:

  • Whether the database response time has slowed down (database latency)
  • Observing the latency from the application side (application latency + database latency)
  • Latency of individual interfaces

First, identify the slow areas, and then optimize them accordingly. Otherwise, the discussion here will be too vague, and it won’t be possible to analyze specific issues concretely.

| username: tidb菜鸟一只 | Original post link

If the business is slow, it is recommended to review the business process from top to bottom, rather than looking directly at the database.

| username: dba远航 | Original post link

Is there a parameter that limits concurrency?

| username: TiDBer_uI8QIp1t | Original post link

Now I’m suspicious, but I don’t know which parameters can determine these.

| username: 托马斯滑板鞋 | Original post link

Do you have numactl installed? Please provide the output of numactl --hardware and numastat -m.

| username: oceanzhang | Original post link

The characteristic of TiDB is inherently slow reading.

| username: TiDBer_uI8QIp1t | Original post link

Not installed

| username: TiDBer_uI8QIp1t | Original post link

Is this impact significant?

| username: 托马斯滑板鞋 | Original post link

Is your machine x86 or ARM?