TiDB TiCDC Data Synchronization Failure

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TIDB ticdc 数据同步不过去

| username: Zhou_Ming

[TiDB Usage Environment] Test Environment
[TiDB Version] 6.1
[Encountered Issue] TICDC Data Synchronization Issue
[Reproduction Path] What operations were performed to encounter the issue
[Issue Phenomenon and Impact]
Hello, I tried to verify the data synchronization function. The source is TiDB 6.1 and the target is MySQL 5.5. After creating the changefeed on TiDB, both the source and target have the synchronization test table and primary key set up. After inserting data on the source, it did not synchronize to the target. I checked the changefeed status, and it is normal. Is there anything else that needs to be configured?

Below is the output of the query changefeed details:
Starting component cdc: /root/.tiup/components/cdc/v6.1.1/cdc cli changefeed query --pd=http://192.168.56.103:2379 --changefeed-id=simple-replication-task
{
“info”: {
“upstream-id”: 0,
“sink-uri”: “mysql://root:xxxx@192.168.56.103:3306/”,
“opts”: {},
“create-time”: “2022-10-20T09:05:36.102187123+08:00”,
“start-ts”: 436791655858438149,
“target-ts”: 0,
“admin-job-type”: 0,
“sort-engine”: “unified”,
“sort-dir”: “”,
“config”: {
“case-sensitive”: true,
“enable-old-value”: true,
“force-replicate”: false,
“check-gc-safe-point”: true,
“filter”: {
“rules”: [
“.”
],
“ignore-txn-start-ts”: null
},
“mounter”: {
“worker-num”: 16
},
“sink”: {
“dispatchers”: null,
“protocol”: “”,
“column-selectors”: null,
“schema-registry”: “”,
“transaction-atomicity”: “table”
},
“cyclic-replication”: {
“enable”: false,
“replica-id”: 0,
“filter-replica-ids”: null,
“id-buckets”: 0,
“sync-ddl”: false
},
“consistent”: {
“level”: “none”,
“max-log-size”: 64,
“flush-interval”: 2000,
“storage”: “”
}
},
“state”: “normal”,
“error”: null,
“sync-point-enabled”: false,
“sync-point-interval”: 600000000000,
“creator-version”: “v6.1.1”
},
“status”: {
“resolved-ts”: 436793412800479234,
“checkpoint-ts”: 436793412800479234,
“admin-job-type”: 0
},
“count”: 0,
“task-status”: [
{
“capture_id”: “0b89cd6e-0f76-4eec-88d8-0600df50c7b4”,
“table_ids”: ,
“table_operations”: null
},
{
“capture_id”: “25e6ebd4-8109-4e4c-bfd8-0cff8227ed09”,
“table_ids”: ,
“table_operations”: null
}
]
}

2
| username: neilshen | Original post link

You can check whether the synchronized tables meet the TiCDC requirements, see TiCDC Synchronization Restrictions.

3
| username: Zhou_Ming | Original post link

The table is fine. It’s a simple table. I checked the CDC logs and didn’t see any errors.

CREATE TABLE `test` (
  `id` char(10) NOT NULL,
  `name` char(20) DEFAULT NULL,
  `address` char(30) DEFAULT NULL,
  PRIMARY KEY (`id`) /*T![clustered_index] NONCLUSTERED */
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin
4
| username: Zhou_Ming | Original post link

By the way, my target server is Server version: 5.5.68-MariaDB MariaDB Server. This database should be fine, right?

5
| username: Zhou_Ming | Original post link

I saw this warning in the CDC log.

6
| username: songxuecheng | Original post link

Please send the synchronization configuration file.

7
| username: Zhou_Ming | Original post link

Are you referring to this? I am synchronizing the tables under the ZM database.

8
| username: songxuecheng | Original post link

Configure it here as well and try again.

9
| username: Zhou_Ming | Original post link

Okay, I’ll give it a try. Thank you. I configured it based on this example:
【SOP Series 30】TiCDC Migration - TiDB to MySQL Test - Operations Guide - TiDB Q&A Community (asktug.com)

10
| username: Zhou_Ming | Original post link

Adding the above part still doesn’t work.

11
| username: songxuecheng | Original post link

Please provide the complete CDC logs.

12
| username: Zhou_Ming | Original post link

cdc.log (177.8 KB)

13
| username: Zhou_Ming | Original post link

The attachment is the CDC log.

14
| username: Lucien-卢西恩 | Original post link

Confirm whether the network ports between TiCDC and PD are functioning properly. Does PD have any records related to TiCDC?
[2022/10/19 16:39:02.870 +08:00] [INFO] [base_client.go:275] [“[pd] cannot update member from this address”] [address=http://192.168.56.103:2379]

15
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.