Memory Usage in Transactions

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 事务内存占用

| username: 大鱼海棠

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version] 5.4.1
[Encountered Problem]
[Reproduction Path] What operations were performed that led to the problem
[Problem Phenomenon and Impact]
According to “TiDB in Action,”

    • The memory usage of transactions may be amplified by 3 to 4 times, meaning a 10GB transaction might occupy 30 to 40GB of memory.

How should this be understood? Can any experts explain this?

[Attachments]

Please provide the version information of each component, such as cdc/tikv, which can be obtained by executing cdc version/tikv-server --version.

2
| username: jansu-dev | Original post link

Here are a few points I can think of:

  1. In cases of high memory usage, select * from XXX limit 1, but TiKV returns data to TiDB concurrently, which means TiDB’s receive channel will receive more data, occupying memory.
  2. Due to the implementation method, since TiDB is written in Go, there might be slice copying at different operator levels.
  3. Another scenario is when data comes up from TiKV and needs to be aggregated by operators. The underlying layer might not release memory in advance, meaning the received data occupies space, the data structure used for aggregation occupies space, and the aggregation results also occupy space, etc.

Firstly, TiDB-in-Action is a book written by DB + open-source enthusiasts within 24 hours. To answer this clearly, you need to ask the author which case this statement is based on, what the prerequisites are, what operators are used, and what the execution plan is. Only by understanding these can you truly answer your question. (Additionally, I have reason to suspect that there might be issues with what he wrote:joy:)

BTW: It’s best to provide complete information and your understanding before asking questions. If the respondent hasn’t read TiDB-in-Action, they won’t know where the quote is from or its context.

3
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.