Consultation on TICDC Alarm cdc_checkpoint_high_delay Issue

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TICDC报警cdc_checkpoint_high_delay问题请教

| username: jaybing926

[TiDB Usage Environment] Production Environment
[TiDB Version] v5.4.3
[Reproduction Path] What operations were performed when the issue occurred
No operations were performed recently. Yesterday, the checkpoint lag suddenly increased but then resolved itself. At that time, I checked the changefeed status, and it was normal with no errors reported. I also filtered the logs and found no errors. I would like to ask what situations could cause this. I don’t quite understand many of the CDC monitoring indicators; could an expert help take a look?

[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page


| username: dba远航 | Original post link

It should be that there was an issue with the original connection or data retrieval, but it has now been restored.

| username: caiyfc | Original post link

It is possible that a large transaction is blocking the process. You can check the SQL at the start of the delay to see if there is a large transaction.

| username: jaybing926 | Original post link

It’s possible. I am using TiCDC to write data into Kafka. According to Kafka monitoring, the data volume indeed increased significantly yesterday. One reason could be large transactions. Is it possible that the data volume was too large, reaching the maximum processing capacity of TiCDC, causing the delay?

| username: caiyfc | Original post link

It is also possible. This depends on the resource usage of CDC. Generally, if the CPU, memory, and disk I/O are not high, there won’t be a situation where the delay is too high.

| username: jaybing926 | Original post link

I checked the monitoring, and during the problematic period, the load, IO, and memory were all normal. It should be an issue with large transactions. Thanks~

| username: wangccsy | Original post link

In which industries and companies is TiDB currently being used?

| username: jaybing926 | Original post link

TiDB is an HTAP database, so it should be applicable in many industries. Quite a few financial internet companies are using it.

| username: 双开门变频冰箱 | Original post link

It seems to be used more in the financial industry.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.