TiCDC Unable to Synchronize TiDB Data to Doris

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc无法同步tidb数据到doris

| username: TiDBer_GAab7x3P

[TiDB Usage Environment] Testing
[TiDB Version] v7.5.1
[Doris Version] v2.0.1
[Reproduction Path]
The operation command is as follows:

tiup cdc cli changefeed create \
    --server=http://xx.xx.xx.xx:8300 \
    --sink-uri="mysql://root:123456@xx.xx.xx.xx:9030/" \
    --changefeed-id="tidb-to-doris-task"

[Problem Encountered: Unable to successfully create CDC synchronization task]
[Resource Configuration] Enter TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]
Error Log 1: Error: [CDC:ErrMySQLConnectionError] fail to open MySQL connection: Error 1228 (HY000): errCode = 2, detailMessage = Variable ‘sql_mode’ can’t be set to the value of ‘ALLOW_INVALID_DATES’

Error Log 2: Error: [CDC:ErrMySQLConnectionError] fail to open MySQL connection: Error 1228 (HY000): errCode = 2, detailMessage = Unknown system variable ‘foreign_key_checks’

Parameter Explanation:

  1. Doris has the sql_mode system variable, but it cannot be set to ‘ALLOW_INVALID_DATES’;
  2. Doris does not have the foreign_key_checks system variable.

Help: Is it possible to skip the system variable settings when creating the task? The methods found online are ineffective.
image

2
| username: zhaokede | Original post link

Remove this SQL mode

3
| username: tidb狂热爱好者 | Original post link

ticdc is not compatible with starrock

4
| username: YuchongXU | Original post link

Not supported

5
| username: 呢莫不爱吃鱼 | Original post link

It seems that CDC does not currently support Doris, right?

6
| username: 数据库真NB | Original post link

Incompatible

7
| username: TiDBer_GAab7x3P | Original post link

I tried to add filter conditions but was unsuccessful. I understand that TiCDC initializes the system variables on the sink side when creating tasks, but Doris does not support these variable settings, or there are no corresponding system variables, such as foreign_key_checks.

8
| username: TiDBer_GAab7x3P | Original post link

Based on verification, it should not be supported. The official documentation states that databases compatible with the MySQL protocol are supported.

9
| username: zhang_2023 | Original post link

This is not supported.

10
| username: TiDBer_GAab7x3P | Original post link

Okay, thank you.

11
| username: TiDBer_GAab7x3P | Original post link

:+1:

12
| username: TiDBer_GAab7x3P | Original post link

:+1: :+1:

13
| username: forever | Original post link

Doris still does not fully support the MySQL protocol, while TiDB is much more compatible than Doris.

14
| username: TiDBer_GAab7x3P | Original post link

It looks like this.

15
| username: TiDBer_QYr0vohO | Original post link

Not supported

16
| username: kelvin | Original post link

This is not supported, right?

17
| username: Kamner | Original post link

TiCDC first to Kafka, then to Doris.

18
| username: TiDBer_JUi6UvZm | Original post link

Got it!