TiCDC cannot use KOP to create a changefeed with Pulsar as the downstream

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc无法使用kop创建下游为pulsar的 changefeed

| username: Curiouser

【TiDB Version】
TiDB: 7.2.0
Pulsar: 2.9.3
Pulsar KOP: 2.9.3.4

Using the Kafka client tool offset explorer to directly connect to Pulsar KOP works fine.

【Reproduction Path】Operations that led to the issue
prod-tidb-cdc-pulsar-kop.toml

case-sensitive = true
enable-old-value = true
[filter]
rules = ['test.t1', 'test.t2', 'test.t3', 'test.t4', 'test.t5']
[mounter]
worker-num = 8
[sink]
dispatchers = [
    { matcher = ['test.*'], topic = "{schema}-cdc-{table}" }
]
protocol = "canal-json"

Command to create Changefeed

tiup ctl:v7.2.0 cdc changefeed create  \ 
  --pd=http://192.168.1.1:2379  \
  --sink-uri="kafka://192.168.1.101:9092/ticdc?protocol=canal-json&compression=lz4&kafka-version=2.9.3&insecure-skip-verify=true&replication-factor=1"   \
  --changefeed-id="prod-tidb-cdc-pulsar-kop"  \
  --config /home/tidb/.tiup/ticdc/changefeed/prod-tidb-cdc-pulsar-kop.toml

Error

Error: [CDC:ErrKafkaNewProducer]new kafka producer: kafka: client has run out of available brokers to talk to: EOF
Error: exit status 1
| username: WalterWj | Original post link

This error message indicates that TiCDC cannot connect to the Kafka brokers :thinking:

How about using telnet to check? telnet
Is the configuration correct?:

kafka://<kafka-broker-host>:<kafka-broker-port>/<topic-name>?kafka-version=1.1.1&partition-num=3
| username: Curiouser | Original post link

The image you provided is not visible. Please provide the text you need translated.

| username: yiduoyunQ | Original post link

If it’s a test environment, how about trying a different KOP version? A known KOP issue: [BUG] Get a NONE error when getting broker's config · Issue #1836 · streamnative/kop · GitHub

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.