During long running import GC continue to work and create a safe point.
CDC not able to start a stream - but what is the reason for that. if new MYSQL binlog event should be treated like an update or insert as a new data.
Application environment:
PoC environment
TiDB version:
8.5.0
Reproduction method:
dumping huge database
lightening for a week.
try to start CDC for replication to catch-up with updates
Problem:
Error: [CDC:ErrStartTsBeforeGC]fail to create or maintain changefeed because start-ts 456073056878592000 is earlier than or equal to GC safepoint at 456324467184631808
The error message “[CDC:ErrStartTsBeforeGC] Failed to create or maintain changefeed because start-ts 456073056878592000 is earlier than or equal to GC safepoint at 456324467184631808” indicates that the specified start timestamp (start-ts) for a TiCDC changefeed is earlier than or equal to the current garbage collection (GC) safepoint in the TiDB cluster. This situation prevents the creation or maintenance of the changefeed because the data at the specified start-ts has already been garbage collected and is no longer available.
Explanation
Garbage Collection (GC) in TiDB: TiDB uses a mechanism called Multi-Version Concurrency Control (MVCC) to manage data versions. Over time, older versions of data that are no longer needed are cleaned up by the GC process to reclaim storage space and maintain system performance. The GC safepoint is a timestamp that marks the boundary before which all data versions can be safely deleted.
TiCDC and Start-ts: TiCDC (TiDB Change Data Capture) is a tool used to capture and replicate incremental data changes from a TiDB cluster to downstream systems. When creating a changefeed, a start-ts is specified, which indicates the point in time from which data changes should be captured. This start-ts must be later than the current GC safepoint to ensure that the data is still available for replication.
Error Cause: The error occurs because the specified start-ts (456073056878592000) is earlier than the GC safepoint (456324467184631808). This means that the data at the start-ts has already been garbage collected, and TiCDC cannot access it to start the replication process.
Solution
To resolve this error, you need to ensure that the start-ts for the changefeed is set to a value later than the current GC safepoint. Here are the steps you can take:
Check the Current GC Safepoint: Before creating a changefeed, check the current GC safepoint in the TiDB cluster. You can do this by querying the tikv_gc_safe_point from the system tables or using the TiDB monitoring tools.
Set an Appropriate Start-ts: When creating a changefeed, ensure that the start-ts is greater than the current GC safepoint. If you are unsure of the exact timestamp to use, you can set the start-ts to 0 or omit it, allowing TiCDC to automatically use the current timestamp as the start point.
Adjust GC Settings if Necessary: If you need to capture data from a specific past point in time, consider adjusting the GC settings to retain data for a longer period. This can be done by increasing the tidb_gc_life_time variable, but be cautious as this can impact system performance by retaining more data versions.
Monitor and Adjust Regularly: Regularly monitor the GC settings and adjust them as needed based on your data retention and replication requirements. This will help prevent similar issues in the future.
By following these steps, you can successfully create and maintain changefeeds in TiCDC without encountering the start-ts before GC safepoint error.
Hi @Petro_Maliyenko , The error means the GC safe point of TiKV is larger then the start ts of changefeed. So the updates are already cleaned up. You have to stop the GC during the preparation of the downstream.