Issues Related to the --start-ts Parameter of TICDC Changefeed

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 关于TICDC changefeed 的参数–start-ts参数相关问题

| username: timyao

[TiDB Usage Environment] Production Environment
[TiDB Version] V7.1.1
[Reproduction Path] Operations performed that led to the issue
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

Questions:

  1. Can the TICDC changefeed parameter --start-ts value only be TSO (a long string of numbers)? Can it be a date, such as “2023-07-31 13:55:00”?

  2. The official documentation states that the --start-ts value can be taken from the Pos value in the metadata file generated by dumpling (a long string of numbers TSO), but my metadata Pos is a date. Why is this?
    My tool version is: dumpling-v7.2.0-linux-amd64.tar.gz
    The metadata content is:

Started dump at: 2023-08-01 00:02:54
SHOW MASTER STATUS:
Log: tidb-binlog
Pos: 2023-07-31 13:55:00
GTID:

Finished dump at: 2023-08-01 02:15:13

In the export command, --snapshot “2023-07-31 13:55:00” was specified (deliberately set 1 minute after gc_safe_point)
The current gc_safe_point=20230731-21:54:00.184 +0800

  1. By default, should --snapshot “2023-07-31 13:55:00” be in the timezone of system_tz in mysql.tidb? (My cluster is set to UTC)
| username: timyao | Original post link

Question 2: I tested it myself, and if you don’t specify --snapshot “2023-07-31 13:55:00” or the --snapshot value is a TSO value, the exported metadata Pos value is no longer a date but a TSO value.

Regarding questions 1 and 3, I haven’t verified them yet. If anyone has experience, could you please help answer them?

| username: ShawnYan | Original post link

I’m not very familiar with Dumpling, but I found some related documentation:

Converting TSO to time is also very convenient

Check the system time zone, mine is Shanghai

[root@shawnyan ~]# ll /etc/localtime 
lrwxrwxrwx. 1 root root 35 Feb 10 22:19 /etc/localtime -> ../usr/share/zoneinfo/Asia/Shanghai
| username: timyao | Original post link

Can the TICDC changefeed parameter --start-ts value be specified as a date?

| username: ShawnYan | Original post link

The type of start-ts is uint64, not a date type.

| username: redgame | Original post link

The time for the --snapshot parameter is relative to the system time zone set on the TiDB server.

| username: timyao | Original post link

Teacher, in my situation, the Pos value of metadata is already a date. Since the ICDC changefeed parameter --start-ts must be a TSO value, how can I find the TSO value corresponding to the full Pos value?

| username: timyao | Original post link

SELECT @@global.time_zone, @@session.time_zone, @@global.system_time_zone;
All three of my results are UTC. Does this mean that the value of --snapshot corresponds to UTC?

| username: timyao | Original post link

Teacher, how to convert time to TSO?

| username: 像风一样的男子 | Original post link

Convert TSO
SELECT TIDB_PARSE_TSO(431675774157717575);
SELECT conv(concat(bin(unix_timestamp(‘2022-01-06 12:30:59’) * 1000),‘000000000000000001’),2,10);

| username: ShawnYan | Original post link

This post is worth bookmarking