CDC unable to elect owner, synchronization task interrupted

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: cdc无法选举owner,同步任务中断

| username: wakaka

[TiDB Usage Environment] Production Environment
[TiDB Version] 5.2.2
[Reproduction Path] Pause some changefeeds
[Encountered Problem: Phenomenon and Impact] Monitoring shows no owner, but the command line shows there is one. The checkpoint of the synchronization task is not advancing.
[Resource Configuration]
[Attachment: Screenshot/Log/Monitoring]

image

| username: Billmay表妹 | Original post link

Can you provide more information? For example, monitoring logs, etc. What operations were performed that caused the issue?

| username: Billmay表妹 | Original post link

You can try the following diagnostic steps:

  1. Check TiCDC-related monitoring metrics, such as cdc_owner_election_duration_seconds, cdc_owner_election_count, cdc_processor_puller_event_queue_size, etc., to see if there are any anomalies.
  2. Check the TiCDC logs to see if there are any related error messages, such as no owner for changefeed, failed to elect owner, etc.
  3. Check the TiKV logs to see if there are any related error messages, such as region is not found, raft: rejected transfer leadership, etc.
  4. Check the status of TiCDC synchronization tasks to see if there is a situation where the checkpoint is not advancing. For example, use the command tiup ctl:v5.0.x cdc changefeed query --pd=http://xxx.xxx.xxx.xxx:2379 --changefeed-id=xxx { "state": "normal", "tso": xxx, "checkpoint": "xxxx-xx-xx xx:xx:xx.xxx", "error": null } to query the synchronization task status.
| username: songxuecheng | Original post link

  1. Is the CDC constantly restarting?
  2. Find the owner and try restarting it.
| username: wakaka | Original post link

I couldn’t find the monitoring metrics cdc_owner_election_duration_seconds, cdc_owner_election_count, and cdc_processor_puller_event_queue_size.

| username: wakaka | Original post link

Restarting doesn’t help either; the command shows it exists, but the monitoring shows it doesn’t.

| username: wakaka | Original post link

The TiCDC logs do not contain the keywords “no owner for changefeed” or “failed to elect owner”.

| username: wakaka | Original post link

The CDC tasks are all hanging and not advancing the checkpoint.

| username: huhaifeng | Original post link

Please provide the relevant logs, a complete line, so we can take a look.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.