Error ErrFlowControllerEventLargerThanQuota Occurred During CDC Synchronization

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: cdc同步时报错ErrFlowControllerEventLargerThanQuota

| username: Jason9808

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version] 6.1.2
Error ErrFlowControllerEventLargerThanQuota occurs during CDC synchronization


How should I configure the memory size for this?

| username: 大鱼海棠 | Original post link

The default value of per-table-memory-quota is 10485760. It’s a parameter of cdc-server. Just change it and it will be fine.

| username: Jason9808 | Original post link

It seems that version 6.1.2 doesn’t work. Do we need version 6.5 for this?

| username: Billmay表妹 | Original post link

The ErrFlowControllerEventLargerThanQuota error indicates that the flow control event exceeded the quota limit during TiCDC synchronization. This error typically occurs when the synchronization speed of TiCDC exceeds the processing capacity of the target TiDB cluster, causing TiCDC to be unable to write data to the target TiDB cluster.

To resolve this issue, you can try the following methods:

  1. Adjust the flow control quota of TiCDC by modifying the TiCDC configuration file. The specific configuration items are sink.max-producers and sink.producer-queue-size. You can appropriately increase the values of these two configuration items based on the actual situation to improve the synchronization speed of TiCDC.
  2. Enhance the processing capacity of the target TiDB cluster by increasing the number of nodes in the TiDB cluster or upgrading the hardware configuration of the TiDB cluster.
  3. Adjust the synchronization speed of TiCDC by modifying the cdc.rate-limit configuration item to limit the synchronization speed of TiCDC, thereby avoiding exceeding the processing capacity of the target TiDB cluster.
| username: Jason9808 | Original post link

The large amount of single-row data seems to have nothing to do with the rate. It should be necessary to increase the threshold of per-table-memory-quota, but I couldn’t find how to set it in version 6.1.2, although it is available in version 6.5.

| username: 大鱼海棠 | Original post link

Please share your configuration, so I can see how you wrote it.

| username: Jason9808 | Original post link

Should it be written here?

| username: 大鱼海棠 | Original post link

Remove the two square brackets after cdc.

| username: 大鱼海棠 | Original post link

The image is not visible, please provide the text you need translated.

| username: tidb狂热爱好者 | Original post link

Are you synchronizing data between two TiDB instances?

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.