Are there any optimization measures for TiDB when memory usage exceeds limits?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiDB占用内存超过,有没什么优化措施

| username: TiDBer_AgtNLAAt

[TiDB Usage Environment] Production Environment
[TiDB Version] 7.0
[Reproduction Path] Simple database operations
[Encountered Problem: Problem Phenomenon and Impact]
[Resource Configuration]

As shown in the screenshot, the virtual memory required by pd-server is actually 10.9G.
The physical memory of tikv-server also requires 1.9G.
Is this normal?

| username: zhanggame1 | Original post link

Look at the RES for memory usage, not the VIRT. Both TiKV and TiDB consume a lot of memory, and they are already using more than 1GB without doing any work yet.

| username: TiDBer_AgtNLAAt | Original post link

Are other databases like this too?

| username: cassblanca | Original post link

It’s completely normal. In the past, a big data platform with 40 nodes and a total available memory of 1.5T sometimes wasn’t enough to run large SQL queries on Hive.

| username: tidb菜鸟一只 | Original post link

TiKV is only 1.9G, running anything should easily exceed this…

| username: TiDBer_AgtNLAAt | Original post link

What is the database doing that requires so much memory? I haven’t performed many operations.

| username: zhanggame1 | Original post link

TiDB is like this, it consumes memory even when it’s doing nothing.

| username: Jellybean | Original post link

Judging from the screenshot you provided, deploying multiple nodes such as tipd, tikv, ticdc, tidb, and prometheus on a single machine seems to cause severe resource contention. Any operation might lead to issues.

It is recommended to at least separate tikv and ticdc and deploy them independently on different machine nodes. Otherwise, even in a test environment, it is highly likely to affect your usage, and it would be difficult to analyze and troubleshoot any problems that arise.

| username: TiDBer_AgtNLAAt | Original post link

I am a small business and don’t need a cluster or anything. Can I shut down some processes?

| username: Jellybean | Original post link

Is your usage for local testing or a formal production environment?

| username: Jellybean | Original post link

If you want to verify the functionality and features of TiDB, this installation method is indeed fine. You can also refer to the official local single-machine deployment steps in the “TiDB Database Quick Start Guide” TiDB 数据库快速上手指南 | PingCAP 归档文档站

| username: TiDBer_AgtNLAAt | Original post link

I also do this in the production environment. Do you not provide a standalone version? Is there only a cluster version?

| username: zhanggame1 | Original post link

Why not use MySQL or similar for a single machine? Using TiDB has no advantages.

| username: Jellybean | Original post link

In our production environment, we try to deploy different components on different nodes to achieve mutual isolation and ensure high performance and high availability. TiDB is not deployed on a single machine.

If your business data volume does not exceed around 30 million, the access QPS is not that high, and there is no need for rapid growth in space and QPS in the future, you can actually use a single-machine database (such as MySQL, RDS). Distributed databases like TiDB are more suitable for business scenarios involving massive data storage and high QPS access.

| username: Jellybean | Original post link

If something can be done with a single-node database, there is no need to use a distributed database. This is my personal suggestion.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.