TiCDC Connection to Kafka Error

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc链接kafka报错

| username: TiDBer_DVnkdzKV

【TiDB Usage Environment】Production
【TiDB Version】5.4
【Encountered Problem】ticdc connection to Kafka reports an error
【Reproduction Path】tiup cdc cli changefeed create --pd=http://127.0.0.1:2379 --changefeed-id=“kafka-canal-json” --sink-uri=“kafka://127.0.0.1:9092/tidb?kafka-version=2.4.0&protocol=canal-json”
【Problem Phenomenon and Impact】
[2022/08/25 17:54:44.103 +08:00] [WARN] [etcd.go:243] [“found etcd key with reserved cluster id”] [key=/tidb/cdc/capture/166ed910-2f90-4db7-b6fd-3e1e151d936a]

[2022/08/25 17:54:44.103 +08:00] [WARN] [etcd.go:243] [“found etcd key with reserved cluster id”] [key=/tidb/cdc/capture/d18ab8bb-e41c-468d-97f2-905eccdb9153]

[2022/08/25 17:54:44.103 +08:00] [WARN] [etcd.go:243] [“found etcd key with reserved cluster id”] [key=/tidb/cdc/owner/a92827bfa4d8cfb]

[2022/08/25 17:54:44.103 +08:00] [WARN] [etcd.go:243] [“found etcd key with reserved cluster id”] [key=/tidb/cdc/owner/a92827bfa4d8cff]

Error: election: no leader
I haven’t used Kafka before, do I need to configure anything for Kafka here? Currently, I have only created a topic named tidb on Kafka.

2
| username: TiDBer_DVnkdzKV | Original post link

Kafka can send and receive messages.

3
| username: TammyLi | Original post link

You used cdc 6.2.0, right? How about changing --pd=http://127.0.0.1:2379 to --server=cdc address and giving it a try?

4
| username: TiDBer_DVnkdzKV | Original post link

It’s still not working. Did I write something wrong?
unknown command “http://172.21.0.14:2379” for “cdc cli changefeed create”

5
| username: TiDBer_DVnkdzKV | Original post link

172.21.0.9 is the IP of my TiCDC and Kafka server. Which service’s IP and port should be used for --server? I didn’t see any explanation in the documentation.

6
| username: TiDBer_DVnkdzKV | Original post link

I found the reason; it should be that my TiDB version is inconsistent with the TiCDC tool version.

7
| username: TiDBer_DVnkdzKV | Original post link

I upgraded TiDB to the latest version, but it still doesn’t work.
./cdc cli changefeed create --sink-uri=‘kafka://172.21.0.9:9092/test?kafka-version=2.6.0&partition-num=1&max-message-bytes=67108864&replication-factor=1&protocol=canal-json’ --changefeed-id=“replication-task-2” --server=http://172.21.0.9:8300
Error: [CDC:ErrKafkaNewSaramaProducer]new sarama producer: dial tcp: lookup VM-0-9-centos on 183.60.83.19:53: no such host
This is the error message. The IP 183.60.83.19:53 is also very strange and is not in my cluster.

8
| username: TiDBer_DVnkdzKV | Original post link

The cause of the problem was found to be the wrong Kafka version being passed.

9
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.