Three Data Centers, Three Locations: Some Questions About Setting Up Local Data Center for Follower Read

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 三地三中心，设置本机房就近读取 Follower Read的一些疑惑

| username: wuweizhan

TiDB 6.1.1, Three Data Centers in Three Locations
[TiDB Version]
Refer to the official documentation, Follower Strong Consistency Read
When a TiKV follower node processes a read request, it first uses the Raft ReadIndex protocol to interact with the current leader of the Region to obtain the latest commit index of the current Raft group. After locally applying the latest commit index obtained from the leader, it can start the normal read request processing flow.

Question: Does this mean that even if Follower Read + closest-replicas is configured, the client in the local data center requesting the local TiKV region-follower still needs to go back to the region-leader in another data center to get the commit index? This way, the traffic still needs to cross data centers. Assuming the dedicated network latency between locations is 30ms, will the response time for requests within the local data center still exceed 30ms?

Question 2: Current main issues with TSO allocation:
TSO allocation is provided by the PD Leader node, which can lead to increased CPU utilization on the Leader node under heavy requests, affecting transaction latency.
PD Follower nodes are mostly idle, resulting in low system resource utilization.
When TiDB accesses the PD Leader across data centers, the latency between data centers increases transaction latency.

Obtaining the latest commit index of the current Raft group + obtaining TSO, it feels like the latency would be completely unacceptable.

2
| username: WalterWj | Original post link

Will :flushed:

3
| username: wuweizhan | Original post link

Is there a way to avoid this 30ms latency for requests within the same data center?

4
| username: TiDBer_jYQINSnf | Original post link

You won’t know if the region you’re reading is the latest without taking it. You definitely need to take it.

5
| username: magic | Original post link

I have stress tested it before, using follower read, the QPS and latency compared to non-follower read are much worse.

6
| username: wuweizhan | Original post link

Follower read QPS/latency still has an advantage, right?

7
| username: magic | Original post link

Follower read QPS/latency has no advantage =-= Performance is much worse.

8
| username: wuweizhan | Original post link

Enabling follower read actually made the performance worse??

9
| username: WalterWj | Original post link

Disadvantages of enabling follower read: 1. One more network RPC interaction, 2. Possible data delay requiring waiting.
The advantage is that in a cross-data center scenario, it can reduce cross-network traffic by reading from the local data center. Additionally, it is effective when read hotspots are concentrated on a single leader.

10
| username: magic | Original post link

In principle, it is like this.