Failed to Create CDC Task (Reusing Previous Task ID)

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: cdc创建任务失败（复用以前的任务id）

| username: xxxxxxxx

In TiDB version 4.0.13, reusing a previously used changefeed-id when creating a CDC task will result in an error, even though the task has already been removed. The details are as follows:

The changefeed-id=cdc-122-kafka-maxwell-json was previously used as an id name and later removed. Now, reusing this id name results in the following error, but using a different id does not cause an error.

Image
Image1380×430 87.4 KB

Additionally, there is another issue. Although there are no CDC tasks in the current cluster, there are still alerts related to latency. I would like to inquire about the reason for this phenomenon. When deleting a CDC task, besides executing the remove operation (without specifying --force), is there anything else that needs to be cleaned up?

2
| username: db_user | Original post link

Could you please restart the CDC cluster? If possible, please restart it. I had some residual monitoring issues before, and restarting the CDC resolved them.

3
| username: songxuecheng | Original post link

The cdc remove command will not immediately delete the information. You can try using tiup cdc:xxx cli changefeed remove -c xxxx --pd=xxxxx --force when deleting.

4
| username: xxxxxxxx | Original post link

It’s the same.

5
| username: xxxxxxxx | Original post link

Restarting is possible, but it only addresses the symptoms, not the root cause. It’s not feasible to restart CDC for every task.

6
| username: songxuecheng | Original post link

What is the result of executing --force?

7
| username: db_user | Original post link

There should be some residue in etcd. Deleting the task will have a read-only status. You should be able to see this changefeed with the CLI query (without -s). Let me test if I can delete this. In version 5.4.1, I deleted it directly. Let me look for the bug.

Bug: Changefeed metadata is not cleaned up as expected 24 hours after removing a changefeed by CLI · Issue #1968 · pingcap/tiflow · GitHub, this bug exists.

It seems that it hasn’t been resolved yet. For now, you can use the restart method. Otherwise, you might have to go into etcd and modify it directly.

8
| username: db_user | Original post link

You can refer to the following link for specific usage:

Read-only commands are fine, but other commands are not recommended as they can easily cause issues with the cluster. You can use the above command to check if there is any residual information.

9
| username: xxxxxxxx | Original post link

Okay, thank you.

10
| username: jansu-dev | Original post link

Why is it a match pattern error according to the error message?

11
| username: HACK | Original post link

If there is residual information, is the only way to release it by restarting the cluster?

12
| username: db_user | Original post link

Currently, it is like this. I haven’t seen a solution on the issue yet. For now, resolving the safe point this way is fine. You can also directly modify etcd, but it might bring other issues to the cluster. Therefore, restarting is a relatively safe approach for now.

13
| username: HACK | Original post link

Is this issue sporadic? I’m also worried that if I delete after copying, this problem will occur.

14
| username: db_user | Original post link

It should not be an occasional issue. I have encountered it several times, but I did not encounter it when testing version 5.4. You can follow up on the issue mentioned above. However, it should not have a significant impact on usage.

15
| username: system | Original post link

This topic will be automatically closed 60 days after the last reply. No new replies are allowed.