Is TiDB unable to start due to setting the tidb_mem_quota_query value too small?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb_mem_quota_query值设置过小导致tidb无法启动？

| username: Qiuchi

[TiDB Usage Environment]
Testing

[TiDB Version]
6.1

[Encountered Problem]
TiDB fails to start after adjusting tidb_mem_quota_query

[Reproduction Path]
Set set tidb_mem_quota_query = 1024 (better not try), restart TiDB

[Problem Phenomenon and Impact]
TiDB fails to start, logs show [expensivequery.go:110] [“expensive_query during bootstrap phase”], [cache.go:324] [“load mysql.user fail”] [error=“Out Of Memory Quota![conn_id=0]”], [terror.go:292] [“unexpected error”] [error=“[privilege:8049]mysql.user”]

tidb.log (544.5 KB)

2
| username: Kongdom | Original post link

Are you serious about limiting the usable memory to 1024 bytes? The value is too small.

3
| username: Qiuchi | Original post link

At that time, I just wanted to see the behavior during OOM… and then it couldn’t start. I want to ask if this parameter can be adjusted directly when TiDB can’t start.

4
| username: Kongdom | Original post link

Starting from version 6.1, this parameter has become a variable and cannot be modified without starting TiDB.

5
| username: Qiuchi | Original post link

So in this situation, it’s basically game over, right? orz

6
| username: 人如其名 | Original post link

Set oom-action to log, then try restarting.

7
| username: gary | Original post link

Since TiDB is stateless, can scaling up or down and resetting solve the issue?

8
| username: Kongdom | Original post link

In version 6.1, this has been converted into a system variable, and it can only be set after starting.

9
| username: Kongdom | Original post link

This has been persisted, expanding TiDB won’t work either.

10
| username: alfred | Original post link

There must be a configuration where the parameters are located, right?

11
| username: Qiuchi | Original post link

The main question is about how the persisted system variables are stored locally and whether it is possible to modify them directly.

12
| username: Kongdom | Original post link

:rofl: A friend figured out how to start it.

  1. Compile the tidb-server from the source code yourself.
  2. Remove the panic code related to oomAction and replace the original node (any node).
  3. Restart the cluster and modify the system variables.
  4. Shut down the node, replace the original binary package, and then start it.

If it’s not a production environment, you can destroy and rebuild it, but the data will be lost.

13
| username: alfred | Original post link

Yes, that’s right. If there are no configuration files or similar, it would be unreasonable. Such misconfigurations are difficult to avoid.

14
| username: Qiuchi | Original post link

Feasible, successfully started.

15
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.