Is there a limit to the size of a TiKV cluster, and are there any production deployments with hundreds of PBs?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiKV 集群规模有限制吗，是否有上百PB生产部署的实践？

| username: allex

【TiDB Usage Environment】Production Environment
【TiDB Version】
【Reproduction Path】What operations were performed when the issue occurred
【Encountered Issues: Issue Phenomenon and Impact】
【Resource Configuration】
【Attachments: Screenshots/Logs/Monitoring】

2
| username: tidb菜鸟一只 | Original post link

Handling hundreds of PB with TiDB might be a bit difficult; TiDB generally supports up to 1PB at most, with a maximum of 512 nodes.

3
| username: 我是咖啡哥 | Original post link

Hundreds of petabytes of databases are something that most companies can’t afford. What kind of company is this? Even having petabytes is quite rare.

4
| username: allex | Original post link

The scenario involves metadata storage of hundreds of PB, and I want to see the scalability of TiKV.

According to the official website, it supports PB-level clusters:

TiDB adopts a compute-storage separation architecture, allowing independent scaling of compute and storage. It supports up to 512 compute nodes, with each node supporting up to 1000 concurrent connections, and the cluster capacity can reach PB levels.

5
| username: allex | Original post link

Ordinary company. What is the maximum scale supported by a KV database?

6
| username: 孤君888 | Original post link

100PB? That’s terrifying, isn’t it? I doubt it. We have less than 1TB…

7
| username: xingzhenxiang | Original post link

I currently have 37TB
image

8
| username: allex | Original post link

@Billmay cousin @Kongdom moderators, do you know about this?

9
| username: Kongdom | Original post link
10
| username: Billmay表妹 | Original post link

I remember NetEase’s hot and cold storage is close to 330TB.

11
| username: Billmay表妹 | Original post link

Refer to:
TiDB Cluster Online NVME Disk Replacement Optimization Practice for Hundreds of TB

TiDB Hot and Cold Storage Separation Solution
专栏 - TiDB 冷热存储分离解决方案 | TiDB 社区 (The entire cluster is approximately 330TB)

12
| username: allex | Original post link

Is there a single cluster with PB scale?

13
| username: maokl | Original post link

At present, it feels like having over a hundred terabytes is already affecting backups.

14
| username: allex | Original post link

What limits the maximum support to 512 nodes?

15
| username: allex | Original post link

What does backup refer to?

16
| username: gcworkerishungry | Original post link

Can I ask the experts about this test data and theoretical values: TiDB 6.6.0 Release Notes | PingCAP 文档中心

17
| username: Hacker007 | Original post link

Is this scale of data still suitable for using a database? Even if the cluster supports it, it would be quite difficult to query what you want, right?

18
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.