Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.Original topic: 高并发下的实时写入
|
username: TiDBer_U71iNHJh
The project involves real-time writing and reading under high concurrency. I would like to ask, regarding real-time writing, is it better to directly insert into TiDB under 2000 concurrent requests, or to buffer through Kafka and Flink for batch writing? How is TiDB’s performance if we directly insert point inserts?
You can conduct a stress test yourself to understand the performance of your database cluster. 2000 concurrent connections are not considered high, and building a cluster according to the server configuration on the official website will definitely support it.
Time is a bit tight, and I want to avoid detours as much as possible. If TiDB’s high-concurrency point-insert performance is good, then there’s no need to set up Kafka and Flink. The architecture is heavy, and maintenance is also difficult.
There should be no problem with the standard configuration. Performance largely depends on the hardware environment.
No problem, I asked my colleague who works in the production environment, and 2000 concurrent writes to TiDB is not an issue.
If the hardware is not bad, then there is no problem.
Regardless of the type of database, accumulating small batches and writing them all at once will definitely improve performance by reducing the number of transaction waits and interactions.
2k concurrency is no pressure for TiDB. For convenience, you can directly use insert into …
It is recommended to optimize~
Directly write. You can use the built database resources to perform stress testing and fatigue testing.
I’ve heard that 3 TiDB servers can support 2000 concurrent connections. If that’s not enough, just add more TiDB servers.
There is no problem with 2000 concurrent connections; the advantage of TiDB is its support for high concurrency.
Don’t accumulate batches; having too many rows can turn into a large transaction, which is more likely to cause issues. Many optimizations for small transactions have already been implemented. It’s best if you test it yourself: one insert per row versus one insert per 500 rows.
Submitting 500-1000 lines at a time should be more appropriate.
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: 
|
username: