TiCDC sends all data to the same partition in Kafka

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc发送到kafka的数据全部写到同一个分区内

| username: TiDBer_wmWYn7Oo

cdc startup configuration

tiup cdc cli changefeed --pd=http://10.124.7.111:2379,http://10.124.7.112:2379,http://10.124.7.113:2379 create --sink-uri=“kafka://10.82.196.71:9092,10.82.196.72:9092,10.82.196.73:9092,10.82.196.74:9092,10.82.196.75:9092/tidb_cdc_fms_intime-dwd_tms_waybill_incream_cdc?kafka-version=2.1.1&partition-num=5&max-message-bytes=1000012&replication-factor=2” --changefeed-id=“cdctmswaybill” --config cdc_dwd_tms_waybill_incream_cdc.toml



| username: WalterWj | Original post link

Specify the partition-num parameter in the sink-uri, for example: <kafka://localhost:9092/test?partition-num=3> and give it a try.

| username: TiDBer_wmWYn7Oo | Original post link

I see that the official documentation states that this parameter specifies the number of partitions for the downstream Kafka. We have configured it to be consistent with the number of downstream partitions, partition-num=5. Is there any reason for your suggestion to change it to the default configuration, partition-num=3?

| username: redgame | Original post link

It looks like it was written according to your requirements.