TiCDC Extracts Data to Kafka: How to Assign Each Table to a Separate Topic Instead of All in One Topic

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc 抽取数据到kafka ，想要每个表一个topic, 现在所有的都在一个topic

| username: TiDBer_2Sp8TNiu

【TiDB Usage Environment】Production Environment or Test Environment or POC
【TiDB Version】
【Encountered Issue】
【Reproduction Path】What operations were performed that led to the issue
【Problem Phenomenon and Impact】

【Attachments】

Please provide the version information of each component, such as cdc/tikv, which can be obtained by executing cdc version/tikv-server --version.

2
| username: db_user | Original post link

You can configure filtering rules based on configuration files, with each configuration file corresponding to a table and initiating a synchronization task. This way, each table can have its own topic. However, managing this can be quite cumbersome, and having so many tasks might also create pressure, so it needs to be tested.

3
| username: xfworld | Original post link

You can refer to the topic dispatcher for implementation

https://docs.pingcap.com/zh/tidb/stable/manage-ticdc#topic-分发器

Which version are you using?

4
| username: TiDBer_2Sp8TNiu | Original post link

Version 6.1

5
| username: xfworld | Original post link

Then just follow the method described by @db_user to configure the task file, and use the dispatcher to achieve your requirements.

6
| username: TiDBer_2Sp8TNiu | Original post link

Okay, thank you.

7
| username: system | Original post link

This topic was automatically closed 1 minute after the last reply. No new replies are allowed.