Is it possible to replace TiDB's storage?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 能否替换tidb的存储?

| username: TiDBer_U58GZgGJ

Is there anyone knowledgeable? In a single-node scenario, the TiDB server has the capability to parse, optimize, and generate execution plans. So, can the storage use a self-developed KV storage instead of TiKV and connect with the TiDB server? Is there any official documentation guiding this integration?

| username: zhaokede | Original post link

It doesn’t seem like that’s the way to do it, that’s a big move.

| username: TiDBer_QYr0vohO | Original post link

Most likely not.

| username: DBAER | Original post link

tidb-server mainly handles SQL parsing and compilation? Do you need to implement this functionality yourself? TiDB has a Go SQL parser.

| username: Swan | Original post link

Then why use TiDB :grin:

| username: Kamner | Original post link

What is the purpose of the replacement?

| username: GoodTiDBer | Original post link

Are you planning to wrap a new database yourself?

| username: dba远航 | Original post link

Are you planning to develop a database independently? TiKV is a core component.

| username: wangkk2024 | Original post link

Won’t it increase costs and risks?

| username: zhanggame1 | Original post link

The official TiDB team also thinks that building a database engine is difficult, so they use RocksDB at the lower level. What do you plan to use at the lower level?

| username: TiDBer_21wZg5fm | Original post link

This involves the underlying functionality, so it is definitely possible. The key is to find the development documentation for this database or to study the source code yourself (difficult).

| username: hey-hoho | Original post link

The source code of the TiDB server itself mocks a storage engine. I guess it should be possible to connect to other storage engines, but it’s super complicated. I haven’t seen any documentation on this, so I can only rely on reading the code.

| username: TIDB-Learner | Original post link

If your skills are strong enough, you can definitely do it. If your skills are lacking, you might as well just give up.

| username: changpeng75 | Original post link

RocksDB, as the core storage engine of TiKV, is deeply integrated with TiDB, and many parameter settings are associated with it. Replacing the TiKV engine is as challenging and labor-intensive as rewriting TiDB.

| username: TiDBer_JUi6UvZm | Original post link

Then why not do it in MySQL? Under the MySQL Server layer, you can connect to storage engines like InnoDB and others.

| username: YuchongXU | Original post link

This is awesome!

| username: wangkk2024 | Original post link

Not recommended to replace.

| username: xiaoqiao | Original post link

How much R&D investment is required?

| username: 呢莫不爱吃鱼 | Original post link

This involves the underlying layer. If you are strong enough, you can give it a try~!

| username: 饭光小团 | Original post link

I haven’t seen any official documentation, unless you look for the corresponding interfaces in the code yourself.