Implementing ANN Retrieval via Plugins?

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 通过插件实现ANN检索?

| username: 魔人逗逗

Recently, I looked into vector databases and followed up with some implementations of ANN. Currently, Postgres supports ANN retrieval through the EXTENSION plugin pgvector. Clickhouse seems to have integrated Annoy and USearch, providing corresponding indexes to offer ANN retrieval (I haven’t looked into the specifics).

I also checked TiDB issues and found that there are no plans in this area at the moment. So, I’m wondering if it’s possible to allow the open-source community to incubate and expand through a similar EXTENSION approach.

The main question is:
Does TiDB have a similar extension or plugin mechanism to support community expansion? :thinking:

| username: 魔人逗逗 | Original post link

This TiDB plugin framework seems to work?!

| username: 有猫万事足 | Original post link

The OnParseEvent interface exists but is not implemented.

So I feel that if you want to do something like pgvector,

INSERT INTO items (embedding) VALUES (‘[1,2,3]’), (‘[4,5,6]’);

providing some unique SQL syntax through a plugin is not feasible. You need to modify the TiDB source code to achieve this.

Additionally, there is already a similar request on the demand board, which was proposed in 2020. It is still under the “Need Triage” category. You can go and give it a thumbs up to speed things up.

| username: 魔人逗逗 | Original post link

Thank you, I’ll take a look at onParseEvent when I have time. It seems like it might be useful for vector storage here, but ANN retrieval requires fetching from TiDB and then filtering.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.