How is the TiDB index structure stored? Will a new column family be created in RocksDB?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiDB 索引结构是怎么存储的,是会在rocksdb新建一个列簇吗?

| username: TiDBer_C33

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Problem Phenomenon and Impact] According to the official course, data is stored in RocksDB. How is index data stored in RocksDB?
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]

| username: hey-hoho | Original post link

Indexes are ultimately stored in RocksDB in the form of key-value pairs, just like regular data, and no new column families are created.

| username: xiaoqiao | Original post link

(Key, Value) pair

| username: oceanzhang | Original post link

Key values are also the same as row data.

| username: zhang_2023 | Original post link

The same as regular data storage.

| username: tidb菜鸟一只 | Original post link

Both are key+value structures. The data in your table is generally key+all columns, while the index contains key+index columns.

| username: TiDBer_vJGTQABF | Original post link

The same as the data storage principle.

| username: 数据库真NB | Original post link

It is normal data storage, explained in the video course, (Key, Value) pairs, B-trees, etc.

| username: TiDBer_QYr0vohO | Original post link

Stored in KV structure

| username: TiDBer_JUi6UvZm | Original post link

Understand it as the same as the data.

| username: zhanggame1 | Original post link

KV storage where both the index and data are the same

| username: forever | Original post link

KV Mapping of Index Data
For regular indexes, MySQL has the concept of non-clustered indexes, especially in InnoDB, where the child nodes record the primary key information and then retrieve the result data through a table lookup.
In TiDB, index creation is supported, so how is the index information stored? It supports both primary and secondary indexes (including unique and non-unique indexes), and the mapping method is similar to table data.
The design is as follows:
TiDB assigns an IndexID to each index in the table.
For primary and unique indexes, it needs to quickly locate the RowID based on the key value, which will be stored in the value.
Therefore, the generated key-value pairs are:
Key: tablePrefix{TableID}_indexPrefixSep{IndexID}_indexedColumnsValue
Value: RowID
Since the designed key contains indexedColumnsValue, which is the value of the queried field, it can be directly hit or retrieved through fuzzy search. Then, through the RowID in the value, the corresponding row record can be retrieved from the table data mapping.
For regular indexes, a key value may correspond to multiple rows, and the corresponding RowID needs to be queried based on the key value range.
Key: tablePrefix{TableID}_indexPrefixSep{IndexID}indexedColumnsValue{RowID}
Value: null
Based on the field value, a list of relevant keys can be retrieved, and then the row records can be obtained based on the RowID contained in the key.

| username: zhaokede | Original post link

Key values are the same as data tables.

| username: TIDB-Learner | Original post link

You can use SHOW TABLE table_name INDEX index_index_name REGIONS; to check the distribution of regions. The key-value representation differs for primary keys, unique indexes, and secondary indexes.

| username: shigp_TIDBER | Original post link

Indexes in RocksDB are also stored in key-value format.

| username: wangkk2024 | Original post link

nbkls

| username: dba远航 | Original post link

The same as the data storage principle.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.