CDC Failed to Create Kafka Changefeed

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: CDC创建kafka changefeed失败

| username: CuteRay

[TiDB Version] TiDB v6.1.1
[Problem Encountered] Failed to create CDC changefeed using tiup ctl, downstream is Kafka
[CDC:ErrKafkaNewSaramaProducer] new sarama producer: dial tcp: lookup kafka03 on [::1]:53: read udp [::1]:35071->[::1]:53: read: connection refused

[Reproduction Path]
tiup ctl:v6.1.1 cdc changefeed create --pd=“http://172.22.21.4:2379” --sink-uri=“kafka://172.21.91.1:9092,172.21.91.2:9092,172.21.91.3:9092/tidb-default-topic?protocol=canal-json&kafka-version=2.2.1&partition-num=3&replication-factor=1&max-message-bytes=104857600&auto-create-topic=true” --changefeed-id=“kafka-changefeed” --config changefeed.toml

[Problem Phenomenon and Impact]
Error:
[CDC:ErrKafkaNewSaramaProducer] new sarama producer: dial tcp: lookup kafka03 on [::1]:53: read udp [::1]:35071->[::1]:53: read: connection refused

kafka03 is the hostname of a machine in the Kafka cluster.

2
| username: weixiaobing | Original post link

It looks like the network is not connected. Both the CDC and the machine executing the tiup command need to access Kafka.

3
| username: CuteRay | Original post link

The network does have limitations, but theoretically, shouldn’t it be enough to just open port 9092?

4
| username: weixiaobing | Original post link

Does the machine with tiup have open ports for telnet to the Kafka machine? The tiup machine also needs to be opened.

5
| username: CuteRay | Original post link

All connected.

6
| username: tidb狂热爱好者 | Original post link

UDP needs to be fully open.

7
| username: db_user | Original post link

It seems that the ctl for CDC was not installed before and needs to be pulled, but pulling requires access to the external network, which is not allowed.

8
| username: tidb狂热爱好者 | Original post link

The error message indicates that the port is not accessible.

9
| username: neilshen | Original post link

This error indicates that TiCDC was denied access when trying to reach the DNS server at [::1]:53. You need to check whether the DNS server at [::1]:53 is functioning properly or verify if the DNS address is incorrectly configured.