[CDC:ErrKafkaNewSaramaProducer] new Sarama producer: Kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: [CDC:ErrKafkaNewSaramaProducer]new sarama producer: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

| username: TiDBer_1zWJ77jq

【TiDB Usage Environment】Production\Testing Environment\POC
【TiDB Version】5.4.1
【Encountered Issues】
TICDC connecting to downstream Kafka, where the downstream Kafka is an OEM version (2.3.0-oem-7.0.1) rather than the open-source version, mainly encountering the following two issues:

  1. If the Kafka version is configured to the corresponding open-source version of the OEM (e.g., 2.3.0), it reports an error “[CDC:ErrKafkaNewSaramaProducer]new sarama producer: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)”
  2. If the Kafka version is configured to be exactly the same as the downstream OEM version (e.g., 2.3.0-oem-7.0.1), it reports an error “[CDC:ErrKafkaInvalidVersion]invalid kafka version: invalid version 2.3.0-oem-7.0.1”
    【Reproduction Path】Operations performed that led to the issue
    【Issue Phenomenon and Impact】
    【Attachments】
  • Relevant logs, configuration files, Grafana monitoring (https://metricstool.pingcap.com/)
  • TiUP Cluster Display information
  • TiUP Cluster Edit config information
  • TiDB-Overview monitoring
  • Corresponding module’s Grafana monitoring (if any, such as BR, TiDB-binlog, TiCDC, etc.)
  • Corresponding module logs (including logs from 1 hour before and after the issue)

For questions related to performance optimization or troubleshooting, please download the script and run it. Please select all and copy-paste the terminal output results for upload.

2
| username: db_user | Original post link

The “invalid-version” issue might be due to version incompatibility.

[CDC:ErrKafkaNewSaramaProducer] new sarama producer: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
2) If the Kafka version is configured to be exactly the same as the downstream OEM version (e.g., 2.3.0-oem-7.0.1), the main reasons for this issue are as follows:

  1. Kafka version is not specified.
  2. Kafka has SASL authentication enabled, which CDC does not support.
    When Kafka uses sasl, how do I import data into Kafka using TiCDC · Issue #1106 · pingcap/tiflow · GitHub
  3. The CDC version is not compatible with the current Kafka version.
  4. The listeners in server.properties need to be configured; do not use localhost. Use listeners=PLAINTEXT://kafka_ip:9092.
  5. The CDC machine cannot access Kafka’s 9092 port.
  6. The CDC-CTL machine cannot access Kafka’s 9092 port.

The issue I encountered was because Kafka’s 9092 port was only open to the CDC machine and not to the CDC-CTL machine.

3
| username: system | Original post link

This topic will be automatically closed 60 days after the last reply. No new replies are allowed.