Under what circumstances are SQL statements not logged when enabling tidb_general_log?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 开启tidb_general_log时，什么情况下不记录SQL语句

| username: TiDBer_6WpezJsp

[TiDB Usage Environment] Production Environment
[TiDB Version] 5.4.1
[Encountered Issue] Seeking advice: When enabling a specific tidb-server, under what circumstances or what kind of SQL statements are not recorded in the general_log?

[Reproduction Path]
The DBA accesses the Zabbix service using a script, and Zabbix connects to a specific tidb-server instance to store and manipulate its metadata. Specifically, the script accesses the Zabbix service API to indirectly delete certain records from a table. However, even though tidb_general_log is enabled on that tidb-server, the delete from statement for the specific table is not recorded, only the select statement is recorded.
Explanation: General logic:
(1) The script first performs a fuzzy query on a certain field (the SELECT someid… FROM t LIKE … statement is recorded in the general_log entry):
select xid, name, ipset from t where ipset like ‘%$ip%’;
(2) Then it deletes based on the primary key value of the queried table (not recorded in the general_log entry).
This step involves the Zabbix server sending and executing a delete operation on the tidb-server.
[Issue Phenomenon and Impact]
The main concern is the confusion regarding tidb-server. It is noted that there is a condition for recording general_log, where the InRestrictedSQL variable must be false. However, this path seems too complex to quickly deduce directly from the tidb code. Therefore, seeking advice on under what circumstances SQL statements are not recorded in the enabled general_log.

[Attachments]

Please provide the version information of each component, such as cdc/tikv, which can be obtained by executing cdc version/tikv-server --version.

2
| username: xfworld | Original post link

tidb_general_log

The enabling method is configured according to the TiDB node. If only one node is configured, only that node will record SQL, and the others that are not configured will not take effect.

Simple setup method:

# Enable general log
curl -X POST -d "tidb_general_log=1" http://{TiDBIP}:10080/settings
# Disable general log
curl -X POST -d "tidb_general_log=0" http://{TiDBIP}:10080/settings

Here is a detailed explanation:

There is no information related to InRestrictedSQL in the description.

3
| username: TiDBer_6WpezJsp | Original post link

Thank you.
To sync up:

  1. The issue was resolved later. It was due to a problem with not updating the file handle after mv/rm the tidb.log file on one of the tidb-server nodes. This was an operational issue that caused the general_log not to be generated. There was no problem with logging itself.
  2. Indeed, this variable in the code is used to mark some SQL/operations that should not be recorded in the general_log, such as internal system SQL operations (on system tables or system metadata). Based on point 1, this also eliminates the logical contradiction we initially misunderstood.
4
| username: forever | Original post link

Please send the introduction and usage of this variable, I couldn’t find the description of this parameter, thanks.

5
| username: Kongdom | Original post link
6
| username: TiDBer_6WpezJsp | Original post link

Are you referring to the variable InRestrictedSQL? If so, this is a variable in the TiDB code, not a user-facing variable in TiDB.
You can search the TiDB code for more details~

7
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.