Is the key-value (KV) change log the same as the raft log?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: key-value (KV) change log是raft log吗

| username: TiDBer_SHQw04Jz

【TiDB Usage Environment】Production\Test Environment\POC
Test

【TiDB Version】
6.1

【Encountered Problem】
The CDC official documentation is as follows

I would like to ask, where is the key-value (KV) change log obtained from? Is it obtained from the raft log?
If so, if the GC time is 24 hours, does that mean the raft log also needs to be retained for 24 hours?
Thank you

【Reproduction Path】What operations were performed to encounter the problem
【Problem Phenomenon and Impact】
【Attachments】

  • Relevant logs, configuration files, Grafana monitoring (https://metricstool.pingcap.com/)
  • TiUP Cluster Display information
  • TiUP Cluster Edit config information
  • TiDB-Overview monitoring
  • Corresponding module Grafana monitoring (such as BR, TiDB-binlog, TiCDC, etc.)
  • Corresponding module logs (including logs one hour before and after the problem)

If the question is related to performance optimization or troubleshooting, please download the script and run it. Please select all and copy-paste the terminal output results for upload.

2
| username: xfworld | Original post link

You can refer to an official description
Equivalent to TiDB’s GC capabilities and configuration, TiCDC also has corresponding keep-alive capabilities.

TiCDC sets the survival validity period for the service GC safepoint to 24 hours,
which means that if the TiCDC service is restored within 24 hours of interruption, data loss due to GC can be avoided.

Raft logs will also be appended to RocksDB. TiKV uses a dual RocksDB approach, separating KV and Raft logs.
The retention of Raft logs is managed by a different mechanism.

3
| username: HACK | Original post link

The kv change log involved in ticdc is not a raft log. The kv change log here is a row changed event provided by TiKV that hides most of the internal implementation details.

This is done on a per-region basis.

4
| username: TiDBer_SHQw04Jz | Original post link

May I ask where these KV change logs are stored? Are they also in RocksDB?

5
| username: HACK | Original post link

See if this post is helpful to you:

6
| username: system | Original post link

This topic was automatically closed 1 minute after the last reply. No new replies are allowed.