Hi, we are building a multi-tenant app, and we want to enforce some kind of QOS per tenant in the database to avoid noisy neighbour. We are thinking of using TIDB for the same.
As I understand TIDB can easily scale to 1000s of nodes.
The basic architecture what we are thinking is
create 1 TIKV cluster per tenant, using TIKV operator.
make the placement-driver aware of the TIKV cluster (don’t know how we will do)
scale placement-driver as well as TIDB in N:1.
The basic idea is we don’t want to use 1:1 TIDB cluster per tenant as the data would be less for that, but we want to ensure that we have physically segregated the tenant data and a tenant never get throttled for IOPS.
Any guidance on strategies to continue this is highly appreciated.
Thanks
Hi. If you want to control the resourse per tenant, there is no directly multi-tenant support in TiDB. But you can use several traits to get a similar result. You can refer to placement rules:Placement Rules in SQL | PingCAP Docs
TiKV Operator should only be used if you only want to use TiKV, without using TiDB. Otherwise you should use TiDB Operator.
A TiDB cluster consists of TiDB, PD and TiKV together, it’s not possible to have multiple separate TiKV clusters connected to a TiDB cluster, TiKV is part of the cluster.
You can have a single cluster with TiDB and TiKV and then use placement rules to direct what tables are stored on what set of clusters.
Actually, we don’t recommend you to create TiKV cluster manually, it is error-prone. Create many separate TiKVs and use a single PD is not supported by design.
I am able to achieve above using Deploy a Heterogeneous Cluster for an Existing TiDB Cluster | PingCAP Docs. Where I created the master instance with TIDB,TiKV,PD and then created each tenant’s instance having just TiKV with TiDB operator, pointing to master instance. Added tenant label on each tenant cluster and then placement policy to route the data to that dedicated TIKV instance. I got to know that it’s okay to have max labels around 200, so I am thinking of creating 200 Nodes (+HA) of TiKV for 200 tenants placed in same cluster.
If there are any challenges anticipated in continuing this road kindly suggest.
If you only create 1 TIKV cluster per tenant but share the TiDB cluster by multi-tenant, I am afraid that the QPS of DDL requests will be affected, This is because only one DDL owner node in a TiDB cluster processes DDL requests.
I didn’t fully understand this. Does it meant that DDL statements even across schemas can’t be parallelized across TiDB nodes and each DDL execution will acquire a lock from all nodes?
This seems to be a challenge, is there any workaround. For us the usecase is actually very static and we need < 30 tables with same structure in all tenant schemas. The only DDL we anticipate is at the creation of tenant.
How about Insert and Select, will that can be scaled just by adding mode TiDB nodes and replicas for TiKV? By the way we are not anticipating that any tenant data can ever grow more than 50GB. So handling (200 x 50GB) of data volume with 200 nodes (limit 2 core tikv each, but initially packing 16 such nodes on 1 16 core node ) + 200 EBS volumes + 4(TiDB x 16 core)+3 PD nodes we are thinking as production setup.
I didn’t fully understand this. Does it meant that DDL statements even across schemas can’t be parallelized across TiDB nodes and each DDL execution will acquire a lock from all nodes?
No, actually this is because of there is only one TiDB-server can execute the DDL statement in currently TiDB implement. It does’t matter in your sceanario because you only have <30 tables and will not execute DDL frequently.
How about Insert and Select, will that can be scaled just by adding mode TiDB nodes and replicas for TiKV?
Yes, it is nothing to do with Insert and Select, TiDB can support definitely scale out the Write & Read ability by adding new nodes of TiKV and TiDB-server.