Timeout in Sending Results: Many Interfaces in the Online Environment Experience Network Timeouts

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 发送结果超时,线上环境很多接口网络超时

| username: TiDBer_8mpI7QNz

[TiDB Usage Environment] Production Environment
[TiDB Version]
[Reproduction Path] Network timeout occurs when multiple people call, and checking slow SQL reveals a send timeout. Running it separately is not slow.
[Encountered Problem: Problem Phenomenon and Impact]
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]

After making several consecutive requests to the interface, the monitoring of a node on Alibaba Cloud looks like this. How can it be optimized? Is an internal network speed of 500M/S normal?

Image

| username: tidb菜鸟一只 | Original post link

How many rows are there in total for this SQL?

| username: TiDBer_8mpI7QNz | Original post link

13,000 rows of data

| username: TiDBer_小阿飞 | Original post link

Your diagram indicates that “sending results time” takes the most time. Check the I/O.

| username: zhanggame1 | Original post link

Check if there are any issues with the network, test the speed, it shouldn’t be a 100 Mbps network, right?

| username: 小于同学 | Original post link

Check if there are any issues with the network.

| username: tidb菜鸟一只 | Original post link

It’s still a network issue. Your 13,000 rows of data should normally be very fast. If concurrent returns are slow, it might be a bandwidth problem.

| username: TiDBer_8mpI7QNz | Original post link

After making several consecutive requests to the interface, the monitoring of one of the Alibaba Cloud nodes looks like this. How can it be optimized? Is 500M/S on the intranet normal?

| username: TiDBer_8mpI7QNz | Original post link

Completely unusable, all network timeouts. What should I do?

| username: TiDBer_8mpI7QNz | Original post link

| username: TiDBer_8mpI7QNz | Original post link

Is the large internal network bandwidth usage the reason for this?

| username: tidb菜鸟一只 | Original post link

If you are using Alibaba Cloud, it is best to choose Alibaba Cloud machines under the same switch, and then specify the internal network address of Alibaba Cloud when installing TiDB.

| username: TiDBer_8mpI7QNz | Original post link

It is installed on the intranet.

| username: changpeng75 | Original post link

500M is bits/s, theoretically the highest is just over 60MB.

| username: 源de爸 | Original post link

The amount of data extracted by the SQL itself is relatively large, and then it gets stuck when the frequency is high.

| username: TIDB-Learner | Original post link

Is it read-only or both read and write? Could it be a locking issue? Is the TiDB server using a proxy, such as HAProxy? The configuration might be unreasonable, and the traffic could be limited.

| username: DBAER | Original post link

Makes sense.

| username: YuchongXU | Original post link

It should be a network issue.