Regarding Limiting Memory Usage During Query Processing

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 关于限制查询过程使用的内存

| username: Peng

【TiDB Usage Environment】Production Environment
【TiDB Version】v6.5

【Encountered Issues: Problem Phenomenon and Impact】

  1. Is the system variable tidb_mem_quota_query used to limit the memory usage of a single connection session?
  2. When some query processes consume a large amount of memory, is it consuming TiDB’s memory or TiKV’s memory? Additionally, if the HashJoin algorithm causes a large memory consumption, is it consuming TiDB’s memory or TiKV’s memory?

Thank you!

| username: zhanggame1 | Original post link

  1. Use the system variable tidb_mem_quota_query to configure the memory usage threshold during the execution of an SQL query, in bytes. This refers to TiDB’s memory usage.

  2. The query process will consume a large amount of memory, occupying both TiDB and TiKV memory.

The join algorithm needs to gather data found by TiKV in TiDB, which consumes a large amount of memory. Retrieving data from TiKV also consumes memory.

| username: Peng | Original post link

Thank you for your reply!

| username: Kongdom | Original post link

  1. It limits the memory used by a single connection session to execute a statement.
  2. Both TiDB and TiKV memory will be occupied. Hash join occupies TiDB memory.
| username: Peng | Original post link

Thank you for the reply!
The Hash Join algorithm performs the Join operation in TiDB, so it occupies TiDB’s memory. Before Hash Join, when fetching data from TiKV, it will occupy TiKV’s memory, is that correct?

| username: zhanggame1 | Original post link

Your understanding is correct. The data filtering after “where” and the aggregate functions are generally executed on TiKV.

| username: Kongdom | Original post link

Yes, you can check the execution plan, which actually has simple annotations indicating where it is being executed.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.