TiCDC Node Memory Alarm

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc节点内存报警

| username: jiangh

The memory of two TiCDC nodes is slowly increasing, and they need to be restarted every few days to release memory; this is particularly noticeable when the upstream is deleting tables in batches (due to business needs, thousands of tables are created and deleted periodically). TiDB version: 5.2.3

| username: jiangh | Original post link

Is there a parameter to limit the memory usage of CDC nodes?

| username: xfworld | Original post link

Check which synchronization task is occupying the memory…
Are there any anomalies…

| username: jiangh | Original post link

How to view and handle it?

| username: xfworld | Original post link

Refer to the metrics

| username: jiangh | Original post link

Sorry, I can’t assist with that.

| username: Kongdom | Original post link

Specify the upper limit of the memory quota for this Changefeed in the Capture Server. For the excess usage, it will be prioritized for reclamation by the Go runtime during operation. The default value is 1073741824, which is 1 GB.
memory-quota = 1073741824

| username: jiangh | Original post link

It seems that version 5.2.3 does not have this parameter.

| username: 随缘天空 | Original post link

You can try adjusting the --sorter-max-memory-consumption parameter to limit the memory usage of TiCDC to avoid excessive memory consumption. Also, try to avoid deleting too many tables at once. Consider performing the table deletion operation in batches to reduce the memory pressure on TiCDC.

| username: jiangh | Original post link

I couldn’t find this parameter. Is there a wiki?

| username: Kongdom | Original post link

Sorry, I didn’t notice the version. Can you refer to this for handling?

| username: Kongdom | Original post link

Is the resource_control here okay?

resource_control:
memory_limit: “2G”

resource_control: Runtime resource control. All configurations under this field will be written into the systemd service file, with no restrictions by default. The resources that can be controlled are as follows:

  1. memory_limit: Limits the maximum runtime memory, for example, “2G” means using up to 2GB of memory.
  2. cpu_quota: Limits the maximum runtime CPU usage, for example, “200%”.
  3. io_read_bandwidth_max: Maximum bandwidth for disk I/O reads, for example: “/dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0 100M”.
  4. io_write_bandwidth_max: Maximum bandwidth for disk I/O writes, for example: “/dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0 100M”.
  5. limit_core: Controls the size of core dumps.
| username: 随缘天空 | Original post link

I can’t remember if it’s this parameter either. You can check the parameters related to TiCDC memory in the system environment variables, and then try 系统变量 | PingCAP 文档中心

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.