DM load synchronization gradually slows down. Restarting TiDB recovers it, but it slows down again after a few minutes

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dm load时期同步逐渐变慢。 重启tidb恢复,等待几分钟继续变慢

| username: TiDBer_TAjrKikG

[TiDB Usage Environment] Production Environment
[TiDB Version] v6.5.1
[Reproduction Path] DM full synchronization from MySQL to TiDB. Data size: 99GB
[Encountered Problem: Phenomenon and Impact] Initially very fast, but synchronization slows down over time. Must restart the TiDB cluster to speed up the load.
[Resource Configuration]
DM master and DM worker on the same machine: 16 cores, 32GB
TiKV: 3 nodes, each with 16 cores, 32GB SSD
TiDB: 3 nodes, each with 8 cores, 16GB SSD
PD: 3 nodes, each with 8 cores, 16GB SSD
[Attachments: Screenshots/Logs/Monitoring]

| username: Jellybean | Original post link

Have you checked the logs of the DM master and worker? See if there is any obvious abnormal information in them.

| username: tidb菜鸟一只 | Original post link

How is the memory usage of TiDB?

| username: TiDBer_TAjrKikG | Original post link

After parameter testing, it is related to compression. When I disable compression, I can continuously load data.

| username: TiDBer_TAjrKikG | Original post link

Around 70% has nothing to do with memory.

| username: TiDBer_TAjrKikG | Original post link

Why does compressing data transmission cause an inability to continuously write data?

| username: TiDBer_pkQ5q1l0 | Original post link

The parameter rocksdb.defaultcf.disable-auto-compactions: true is very problematic. After setting it, as the data volume increases, it may affect a series of issues related to PD scheduling.

| username: TiDBer_pkQ5q1l0 | Original post link

Use with caution, use with caution! :grinning:

| username: TiDBer_TAjrKikG | Original post link

It seems like this, while writing, I couldn’t get in.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.