Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: datax写入tidb速度慢！
[TiDB Usage Environment] Production Environment / Testing
[TiDB Version] 5.4.0
[Encountered Problem: Problem Phenomenon and Impact]
Reading data from MySQL and writing to TiDB is very slow, only about 200 rows per second; conversely, from TiDB to MySQL is fine, with several thousand rows per second.
[Attachment: Screenshot/Log/Monitoring]
Sincerely seeking advice from everyone!
You can check the resource usage of each TiDB component to see if there are any bottlenecks.
Check the resource usage in the monitoring and see if there are any write hotspots.
May I ask, which testing tool is this using?
Take a look at the usage of the TiDB cluster.
A data synchronization tool from Alibaba
I am using version 5.4.3. I did not encounter this issue when importing MySQL into TiDB through DataX, with an execution speed of 4000 rows per second. Please check the TiDB Dashboard SQL statement analysis to see which step is taking the most time.
Is the configuration of DataX throttled?
Thank you all for providing ideas and help. The company’s environment is on an intranet, and currently, due to the pandemic, I can’t go there. In a few days, I will follow the ideas given by everyone to troubleshoot. Thanks again~
You should check the configuration file of DataX. If the current command does not meet your needs, you can try writing a bit of Python script to call DataX for data extraction.
DataX has a sleep configuration for traversing tables, you can try adjusting this.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
|
username: TiDBer_wfWT9nN5
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: