TiCDC Changefeed Synchronization Task Failed

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc changefeed 同步任务失败

| username: 不输土豆

[TiDB Usage Environment] Production Environment
[TiDB Version] v4.0.16 version data synchronized to v6.5.0 version via ticdc changefeed
[Reproduction Path] Operations performed that led to the issue
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]
The following log is consistently reported in the logs:

[2023/07/07 09:50:20.081 +08:00] [WARN] [client.go:774] ["send request to stream failed"] [addr=10.0.1.39:20160] [storeID=55663883] [regionID=60153811] [requestID=53768]
[error=EOF]

Should we check if there is an issue with the tikv service? I’ll investigate first. Experienced experts, please provide troubleshooting suggestions and ideas, thank you!
After observing, it seems that the reason for the changefeed task’s checkpoint not updating is not because of this. The fundamental reason is that the checkpointTs is not updating, causing the corresponding task to also not update!

2
| username: xfworld | Original post link

What are the details before and after this error log?

3
| username: 不输土豆 | Original post link

Image|690x41

4
| username: xfworld | Original post link

How is the downstream connection configured? For the changfeed…

And what is the status of the changfeed?

5
| username: 不输土豆 | Original post link

The status of the changefeed has always been normal. I found that the resolvedTs of this changefeed task is consistently not updating, so this task is not updating. We should first identify why the resolvedTs is not updating.

6
| username: 不输土豆 | Original post link

There should be no issues with the downstream service; the status of other changefeed tasks is completely normal.

7
| username: 像风一样的男子 | Original post link

Can you take a screenshot of the result of running cdc cli changefeed list --pd=http://ip:2379?

8
| username: redgame | Original post link

Adjust parameters related to Raft, such as raftstore.sync-log, raftstore.apply-pool-size, etc.

9
| username: 不输土豆 | Original post link

Let me answer again. While troubleshooting why resolvedTs was not updating, I found that this changefeed task had recovered. I suspect it is due to the reason shown in the image below:

Image
Image1018×139 33.8 KB

10
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.