Some Questions About Deploying TiCDC

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 关于部署 TiCDC 的一些问题

| username: mxd-321

I want to deploy a set of TiCDC. Can any experts give some hardware resource suggestions? The related data volume is around 20GB in stock and around 200MB in increments.

2
| username: 大飞哥online | Original post link

CPU: At least 4 cores
Memory: At least 8GB
Storage: At least SSD
Network: At least gigabit Ethernet card
:rofl:

3
| username: 像风一样的男子 | Original post link

The official recommendation is given. You can appropriately reduce the data volume a bit, and it should be fine.

4
| username: TiDBer_gxUpi9Ct | Original post link

The data is a bit low.

5
| username: Kongdom | Original post link

First, perform a full backup with BR, and then use TiCDC for incremental synchronization.

6
| username: 小龙虾爱大龙虾 | Original post link

It is recommended to follow the official configuration.

7
| username: 随缘天空 | Original post link

Try not to go below the official minimum configuration requirements. TiDB is quite hardware-intensive, and if issues arise later, it will be difficult to determine whether they are due to configuration or machine problems. You can refer to the following link: TiDB 软件和硬件环境建议配置 | PingCAP 文档中心. The requirements for development and production are different.

8
| username: zhanggame1 | Original post link

TiCDC is very resource-intensive, so the configuration must be high.

9
| username: 春风十里 | Original post link

You can enrich the scenario description a bit more. What do you want to do with TiCDC?

From your description, the data volume is not large, but you need to consider the purpose, whether it is for a production environment or a disaster recovery environment.

  1. For a production environment, it is recommended to consider high availability and deploy two nodes to form a cluster. This will double your resource requirements. Currently, your configuration can be lower than the official recommendation because your data is not large.
  2. If the production environment is within a local area network, bandwidth is generally not a concern. However, for remote disaster recovery, you need to consider network latency and bandwidth. Best practices provide guidelines on network latency configuration. For remote disaster recovery, you need to consider the maximum bandwidth and whether it can meet your incremental data needs.
  3. Using TiCDC also requires consideration of limitations.

Best Practices

  • When using TiCDC to synchronize data between two TiDB clusters, if the upstream and downstream latency exceeds 100 ms:
    • For versions before v6.5.2, it is recommended to deploy TiCDC in the region (IDC, region) where the downstream TiDB cluster is located.
    • After optimization, for versions v6.5.2 and later, it is recommended to deploy TiCDC in the region (IDC, region) where the upstream cluster is located.
  • The tables synchronized by TiCDC need to have at least one valid index. The definition of a valid index is as follows:
    • A primary key (PRIMARY KEY) is a valid index.
    • A unique index (UNIQUE INDEX) where each column is explicitly defined as non-null (NOT NULL) in the table structure and does not contain virtual generated columns (VIRTUAL GENERATED COLUMNS).
  • In disaster recovery scenarios, using TiCDC requires configuring redo log to achieve eventual consistency.
10
| username: dba远航 | Original post link

Just follow the official deployment recommendations.

11
| username: TiDBer_小阿飞 | Original post link

At least follow the configuration of the first post, at least configure it a bit :joy:

12
| username: TiDBer_gxUpi9Ct | Original post link

It is recommended to follow the official configuration.

13
| username: Kongdom | Original post link

:joy: OP, remember to come back and close the topic after your practice.

14
| username: andone | Original post link

The configuration is too low, add a bit more.

15
| username: mxd-321 | Original post link

Thank you all for your suggestions. I have already deployed it according to the official configuration.

16
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.