Can TiCDC save change logs locally?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc可以将变更日志保存到本地吗

| username: Holland

【TiDB Usage Environment】Testing
【TiDB Version】v7.1.2
【Reproduction Path】What operations were performed to encounter the issue
【Encountered Issue: Issue Phenomenon and Impact】

I hope to achieve functionality similar to MySQL binlog through TiCDC’s change logs, allowing us to know what SQL was executed at what time. Can it be directly saved locally instead of being saved through intermediaries like Kafka?

【Resource Configuration】Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
【Attachments: Screenshots/Logs/Monitoring】

2
| username: 像风一样的男子 | Original post link

Supports direct saving to storage services Amazon S3, GCS, Azure Blob Storage, and NFS.

image
image1113×703 40.9 KB

3
| username: Holland | Original post link

Okay, I’ll give it a try.

4
| username: heiwandou | Original post link

Sure, NFS S3 object storage.

5
| username: Holland | Original post link

I tried it, each SQL is a separate file. Can it be output to a single file in chronological order?

6
| username: 像风一样的男子 | Original post link

You can check the documentation to split by database, table, and date.

  • date-separator: Separates the file path by the transaction commit date. The default value is day. The optional values are as follows:
    • none: Does not separate the file path by date-separator. For example, all files with version number 9999 for test.table1 are stored under the path s3://bucket/bbb/ccc/test/table1/9999.
    • year: Separates the file path by the transaction commit year. For example: s3://bucket/bbb/ccc/test/table1/9999/2022.
    • month: Separates the file path by the transaction commit year and month. For example: s3://bucket/bbb/ccc/test/table1/9999/2022-01.
    • day: Separates the file path by the transaction commit year, month, and day. For example: s3://bucket/bbb/ccc/test/table1/9999/2022-01-02.
7
| username: dba远航 | Original post link

Sure.

8
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.