PD TSO Timestamp Issue

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: PD TSO时间戳问题

| username: lemonade010

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path]
[Encountered Problem: Problem Phenomenon and Impact] I have some questions about PD’s TSO. If the NTP times of the three PD machines are not aligned, with the primary node PD having the correct time and the follower nodes having delayed times, will the TSO decrease if the primary node fails and a follower becomes the primary? Will this cause data anomalies?
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]

| username: TiDBer_jYQINSnf | Original post link

It won’t get smaller. When allocating, it will be compared with the previous one and can only get larger, not smaller.

| username: TiDBer_CQ | Original post link

No, it won’t. Because the global version number of TSO is composed of physical time and logical time. Even if the physical time of the follower node is older, the logical time will still maintain a monotonically increasing order, so no abnormal data will appear.

| username: forever | Original post link

TSO Clock Synchronization

After a PD is successfully elected, it cannot start serving immediately. It needs to ensure that the allocated ts is not less than the ts allocated by the previous leader. First, the PD reads the maximum ts that the previous leader might have allocated from etcd. Then, it checks the local clock to ensure it is greater than the previous ts (if the clocks between different PDs are not synchronized, it may need to sleep and wait). Finally, it persists the current time +3s (adjustable via tso-save-interval) as the new “maximum allocatable ts” to etcd.

The time for this step mainly depends on the degree of clock synchronization. If ntp is normally enabled, it can be completed quickly.

| username: lemonade010 | Original post link

@TiDBer_jYQINSnf @forever @TiDBer_CQ Thank you all for the explanations.

| username: DBAER | Original post link

TSO, as a logical clock, maintains monotonicity and will not experience rollback. However, if deployed across multiple centers, clock issues may arise. Generally, it is placed in one center, or most PD nodes are in one center.

| username: dba远航 | Original post link

No, this issue would definitely be considered during the design phase.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.