Create a New CDC Task Without Synchronizing Data

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 新建CDC任务 不同步数据

| username: jeff

[TiDB Usage Environment] Testing
Question: Created a CDC task to sync to MySQL. The list shows normal, but there is no data downstream. The data is very small, just for my own testing.

Create command:

tiup ctl:v4.0.13 cdc changefeed create --pd=http://10.106.16.249:2379 --sink-uri="mysql://XX:XX@XX:3306/" --changefeed-id="test-cdc-new" --sort-engine="unified" --config=./testcdc.yaml

Logs:

tidb@tidb-novadax-mon01br-t006:~/ticdc$ tiup ctl:v4.0.13 cdc changefeed query --pd=http://10.106.16.249:2379 -c test-cdc-new
Starting component `ctl`: /home/tidb/.tiup/components/ctl/v4.0.13/ctl cdc changefeed query --pd=http://10.106.16.249:2379 -c test-cdc-new
{
  "info": {
    "sink-uri": "mysql://XX:XXX@XX:3306/",
    "opts": {
      "_changefeed_id": "cli-verify"
    },
    "create-time": "2023-07-18T07:41:05.711431379Z",
    "start-ts": 442935821137870850,
    "target-ts": 0,
    "admin-job-type": 0,
    "sort-engine": "unified",
    "config": {
      "case-sensitive": true,
      "enable-old-value": true,
      "force-replicate": false,
      "check-gc-safe-point": true,
      "filter": {
        "rules": [
          "jeff.*"
        ],
        "ignore-txn-start-ts": null,
        "ddl-allow-list": null
      },
      "mounter": {
        "worker-num": 16
      },
      "sink": {
        "dispatchers": null,
        "protocol": "default"
      },
      "cyclic-replication": {
        "enable": false,
        "replica-id": 0,
        "filter-replica-ids": null,
        "id-buckets": 0,
        "sync-ddl": false
      },
      "scheduler": {
        "type": "table-number",
        "polling-time": -1
      }
    },
    "state": "normal",
    "history": null,
    "error": null,
    "sync-point-enabled": false,
    "sync-point-interval": 600000000000,
    "creator-version": "v4.0.13"
  },
  "status": {
    "resolved-ts": 442937057264795653,
    "checkpoint-ts": 442937057264795653,
    "admin-job-type": 0
  },
  "count": 0,
  "task-status": []
}
| username: TiDB_C罗 | Original post link

TiCDC only supports incremental data. Does your TiDB have data being written?

| username: jeff | Original post link

There are writes, mainly to test CDC.

| username: TiDB_C罗 | Original post link

Is this the function of the “filter”?

| username: cassblanca | Original post link

The command for cdc cli changefeed create --****** is written as follows.

| username: jeff | Original post link

tiup ctl:v4.0.13 cdc changefeed create --pd=http://10.106.16.249:2379 --sink-uri=“mysql://XX:XX@XX:3306/” --changefeed-id=“test-cdc-new” --sort-engine=“unified” --config=./testcdc.yaml

| username: 不输土豆 | Original post link

  1. The cdc changefeed task in your screenshot shows a timestamp of 09:20:00.492, and now it is already 18:07 in the afternoon. The checkpoint indicates the time point to which this task has been synchronized. Data earlier than the checkpoint should have been synchronized; data later than the checkpoint will definitely not be visible downstream.
  2. You should check why the checkpoint is not updating. It is recommended to look at the cdc.log log file.
| username: jeff | Original post link

This server is located abroad.

| username: jeff | Original post link

The issue has been resolved. The table I tested did not have a primary key.

| username: TiDB_C罗 | Original post link

For tables without a primary key, I remember there will be a reminder message when creating a task.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.