On-disk behavior of the add index operation on TiDB server

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 关于 add index 操作在 tidb server 上的落盘行为

| username: Qiuchi

[TiDB Environment] Testing
[TiDB Version] 6.5.0
[Reproduction Path] Added a three-column composite index on a table with approximately 4 billion rows
[Encountered Problem: Phenomenon and Impact]

After starting the DDL, the disk usage of one TiDB node rapidly increased to 100%. When querying the DDL job status, the entry disappeared. Upon checking the TiDB node’s disk, a large number of SST files were found in /tmp/tidb/tmp_ddl-4000/35079.

admin show ddl jobs;

In the result of this statement, there are two values for JOB_TYPE: one is add index /* txn-merge /, and the other is add index / ingest */. What is the difference between these two?

Since the status of the DDL cannot be found, can this situation be considered a failure of the operation?

| username: weixiaobing | Original post link

temp-dir Introduced in version v6.3.0

  • The path TiDB uses to store temporary data. If a feature requires using local storage on the TiDB node, TiDB will temporarily store the corresponding data in this directory.
  • During the index creation process, if index creation acceleration is enabled, the data that needs to be backfilled for the newly created index will first be stored in the local temporary storage path of TiDB and then batch imported into TiKV, thereby improving the index creation speed.
  • Default value: “/tmp/tidb”

The default disk configuration is under /tmp/tidb.

add index /* txn-merge */ should use the previous default online index addition method.
Another one is add index /* ingest */, which uses the accelerated index creation method.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.