TiCDC 6.5.0 Unable to Connect to SASL Kafka Cluster

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiCDC 6.5.0 无法接入 sasl 的kafka集群

| username: GreenGuan

TiCDC cannot connect to Kafka with SASL. Has anyone encountered this issue? Please help take a look.

tiup cdc:v6.5.0 cli changefeed create -c mxxxxx-xxxxx-test --pd=http://xxxxx:3379 --sink-uri="kafka://kf1xxxxxx.com:9092,kf2xxxxxx.com:9092,kf3xxxxxx.com:9092/reader-test.mxxxxx.shbt.ticdc?protocol=canal-json&enable-tidb-extension=true&kafka-version=2.5.1&partition-num=3&kafka-client-id=producer_ticdc-prod_mxxxx_xxxxx_test&max-message-bytes=8388608&replication-factor=2&sasl-user=ticdc-prod&sasl-password=xxxx\$xxxx&sasl-mechanism=SCRAM-SHA-256"  --sort-engine="unified" --config=task_test65.yaml
Starting component `cdc`: /home/tidb/.tiup/components/cdc/v6.5.0/cdc /home/tidb/.tiup/components/cdc/v6.5.0/cdc cli changefeed create -c mxxxxx-shbtcore-test --pd=http://xxxxx:3379 --sink-uri=kafka://kf1xxxxxx.com:9092,kf2xxxxxx.com:9092,kf3xxxxxx.com:9092/reader-test.mxxxxx.shbt.ticdc?protocol=canal-json&enable-tidb-extension=true&kafka-version=2.5.1&partition-num=3&kafka-client-id=producer_ticdc-prod_mxxxx_xxxxx_test&max-message-bytes=8388608&replication-factor=2&sasl-user=ticdc-prod&sasl-password=xxxx$xxxx&sasl-mechanism=SCRAM-SHA-256 --sort-engine=unified --config=task_test65.yaml
[WARN] some tables are not eligible to replicate, []v2.TableName{v2.TableName{Schema:"xxxxx", Table:"xxxxx", TableID:2184, IsPartition:false}, v2.TableName{Schema:"xxxxx", Table:"xxxxx", TableID:2282, IsPartition:false}, v2.TableName{Schema:"xxxxx", Table:"flowstatus", TableID:2470, IsPartition:false}, v2.TableName{Schema:"xxxxx", Table:"location_search", TableID:2610, IsPartition:false}, v2.TableName{Schema:"xxxxx", Table:"xxxx", TableID:2612, IsPartition:true}, v2.TableName{Schema:"xxxxx", Table:"logs", TableID:2803, IsPartition:true}, v2.TableName{Schema:"xxxxx", Table:"size_type", TableID:3094, IsPartition:false}, v2.TableName{Schema:"xxxxx", Table:"xxxxx", TableID:3122, IsPartition:false}}
Could you agree to ignore those tables, and continue to replicate [Y/N]
Y
Error: [CDC:ErrKafkaNewSaramaProducer]new sarama producer: kafka server: The client is not authorized to send this request type
2
| username: db_user | Original post link

Is it possible that the $ symbol in the password is causing the issue?

3
| username: liuis | Original post link

Indeed, I have encountered issues with the “#” symbol before. I suggest the original poster remove the special characters from the password and try again.

4
| username: tidb菜鸟一只 | Original post link

It does seem like there is an issue with the SASL password.

5
| username: xfworld | Original post link

Is the configuration incomplete?

6
| username: GreenGuan | Original post link

Do all the items in the red box need to be configured?

7
| username: liuis | Original post link

It looks like the authorization didn’t go through. It seems like your password has special characters. Try changing your password and see if it works.

8
| username: xfworld | Original post link

You need to fill in all the configuration information provided by the application’s environment.

9
| username: GreenGuan | Original post link

According to the test, the issue lies in the presence of special characters in the password. The logic for handling special characters in the sasl-password parameter of the sink needs improvement. Both versions 5.4 and 6.5 have the same problem. Users inevitably use special characters when setting passwords, so it is hoped that this can be made compatible.

10
| username: liuis | Original post link

I feel it might be a code issue. I encountered the same problem with DM before. Special characters at the underlying level probably need to be handled with escape characters.

11
| username: dba-kit | Original post link

This is actually related to how bash handles special characters. Try using single quotes instead of double quotes.

12
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.