Abnormal logs in TiDB Dashboard log search are concentrated on a specific TiDB instance

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TIDBdashboard 日志搜索里的异常日志集中在某台TIDB上

| username: niegang

[TiDB Usage Environment]: Production Environment
[Encountered Problem]: The abnormal logs in the TIDB dashboard log search are concentrated on a specific TIDB instance.
[Resource Configuration]: 3 TiDB, 3 TiPD, 13 TiKV
[Attachment: Screenshot/Log/Monitoring]: Abnormalities can be seen in the screenshot

| username: tidb菜鸟一只 | Original post link

Did you configure the proxy? Could it be that all the load is on this TiDB server?

| username: niegang | Original post link

There are users using HAProxy as a proxy. The connection numbers observed on HAProxy are relatively even:


Moreover, the error screenshot I provided above is within 1 hour. When I query the anomalies over 6 hours, other nodes also show anomalies, but not as much as the one on 172.19.244.69.

What could cause such a phenomenon?

| username: tidb菜鸟一只 | Original post link

Let’s see what the error messages are.

| username: niegang | Original post link

A large number of such errors

| username: 有猫万事足 | Original post link

Drop the statistics of this table and then re-analyze to see.

From the code, it looks like an issue where a decimal number stored in a certain column of a statistics table is too long.

Error location as shown above.

| username: niegang | Original post link

Thanks, I’ll give it a try.

| username: ajin0514 | Original post link

Are you using any proxies?