Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.Original topic: dm同步速度很慢，同步很长时间才追了几个版本，是tidb问题还是dm问题
|
username: Jjjjayson_zeng
【TiDB Usage Environment】Production Environment / Testing / PoC
【TiDB Version】
【Reproduction Path】What operations were performed when the issue occurred
【Encountered Issue: Issue Symptoms and Impact】
【Resource Configuration】
【Attachments: Screenshots / Logs / Monitoring】
You can check the write latency of the downstream database to see if the bottleneck is in TiDB. Or look at the following parameters:
|
username: syncers: # Configuration parameters for the sync processing unit
global: # Configuration name
worker-count: 16 # Number of concurrent threads for applying binlog that has been transferred to the local, default is 16. Adjusting this parameter will not affect the concurrency of upstream log fetching but will put significant pressure on the downstream.
batch: 100 # Number of SQL statements in a transaction batch migrated to the downstream database, default is 100, generally recommended not to exceed 500.
Do these parameters need adjustment?
The image you provided is not visible. Please provide the text content that you need translated.
If DM has sufficient resources, you can increase the worker-count and batch size.
Isn’t 6 and 100 enough? Do you still need a lot more?
Post the DM monitoring metrics and take a look at “Remaining time to sync” and “Replication lag” first.
I am looking at this, I don’t know what the two items you mentioned are.
The DM-Monitor-Professional and DM-Monitor-Standard dashboards are independent and not part of the TiDB cluster.
Do I need to deploy it? I only have the monitoring that comes with the deployment cluster.
The DM deployment document is as follows: 使用 TiUP 部署 DM 集群 | PingCAP 文档中心
It includes the Grafana configuration.
I don’t know if this is what you’re talking about, but I can’t see what you’re referring to anyway.
Open them all to see if there’s the kind I mentioned; you didn’t specify the TiDB version.
If it’s slow, analyze it: what is currently being executed? Check which data is being synchronized.
If I turn it off, I still have the issue of duplicate primary keys. I enabled it to use the REPLACE INTO solution when inserting.
Could you please send the DM version? If the data in MySQL and TiDB are the same, there generally won’t be a primary key duplication issue. However, some versions of DM do not synchronize the truncate table statement, which may result in duplicate key errors.
This project is deployed in the customer’s on-premises environment, and I currently don’t have access. I will ask them for permission to check it. Is there a recommended version for the DM project that is currently deployed?
The DM version needs to be consistent with the TiDB version. Which version are you currently using? Can you determine if it’s a bandwidth issue and analyze the resource usage of the TiDB cluster?
If you are deploying a new DM project, it is best to directly deploy the latest LTS version, which is version 6.5.
PS: The DM version can be completely different from the TiDB cluster version, as there is no dependency between them. The role of DM is to parse binlogs and generate standard SQL statements, without being restricted to a specific TiDB version.
|
username: 
|
username: 
|
username: 
|
username: 
|
username: