Issues with Using CDC Synchronization to Kafka Dispatcher

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: CDC同步到kafka 分发器使用问题

| username: xmlianfeng

TiDB Version: 6.5.3
According to the official documentation 同步数据到 Kafka | PingCAP 文档中心
When synchronizing to Kafka using TiCDC, you can use topic expressions for automatic distribution. I want to specify the topic name, for example:
{matcher = [‘test1.', 'test5.’], topic = “test”, partition = “table”},

Creating the task prompts Error: [CDC:ErrKafkaTopicExprInvalid]invalid topic expression
Then I saw

In this document, there is an example

image
image1045×184 4.22 KB

May I ask, does version 6.5.3 support the distribution mechanism for specifying topic names?

2
| username: MrSylar | Original post link

The official documentation says it is possible. What kind of error are you encountering with your implementation?

3
| username: xmlianfeng | Original post link

Here is my configuration file. I am also wondering if I wrote it incorrectly.
Configuration file:

image
image707×246 4.08 KB

tiup cdc cli changefeed create --pd=http://192.168.1.181:2379 --sink-uri="kafka://192.168.1.190:9094/fent?kafka-version=2.7.0&protocol=canal-json&partition-num=3&replication-factor=3&compressionType=lz4" --config test-cdc.yaml --start-ts=442982603718393968 --changefeed-id=fent
4
| username: MrSylar | Original post link

Try changing single quotes to double quotes?

5
| username: xmlianfeng | Original post link

I have tested it, but it prompts “invalid topic expression.”
Is it necessary to use an expression?

6
| username: Holland | Original post link

How about adding partition = “xxx” and giving it a try?

7
| username: xmlianfeng | Original post link

Tried it. Same result.
Not sure if it’s because 6.5.3 is not supported or something else.

8
| username: nongfushanquan | Original post link
  • {schema}: Required, used to match the database name.