How to View Real-Time Data Changes in TiDB

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb如何查看实时数据变化

| username: yulei7633

How can TiDB view real-time data changes? For example, in MySQL, you can use “show master logs;” to see if there are insert, delete, or update operations, indicating data changes. What commands does TiDB use to view this?

| username: tidb菜鸟一只 | Original post link

Enable CDC to synchronize modification records to Kafka or similar locations such as files.

| username: yulei7633 | Original post link

Is it necessary to enable CDC or binlog?

| username: TiDBer_嘎嘣脆 | Original post link

Logs can be analyzed.

| username: 小龙虾爱大龙虾 | Original post link

In the Affected Rows By Type panel in Grafana TiDB, it can display data changes.

| username: TiDBer_HErMeXDz | Original post link

Try tidb-binlog.

| username: tidb菜鸟一只 | Original post link

MySQL’s master log is only available when binlog is enabled. If it’s not enabled, there won’t be any…

| username: yulei7633 | Original post link

TiCDC can be used, and TiDB-Binlog can also be enabled, but if TiCDC needs to synchronize to downstream, it becomes somewhat redundant.

| username: 健康的腰间盘 | Original post link

Binlog

| username: tidb菜鸟一只 | Original post link

It can be directly stored on storage like S3.

| username: xiaoqiao | Original post link

You can synchronize to MySQL and view it through MySQL.

| username: TIDB-Learner | Original post link

:face_with_peeking_eye: :face_with_peeking_eye:

| username: TIDB-Learner | Original post link

In real-time, it would be difficult to achieve without tools or secondary development. If the timeliness requirement is not high, you can use the system’s built-in table data, such as select table_name, table_rows from INFORMATION_SCHEMA.TABLES;.

| username: changpeng75 | Original post link

Therefore, triggers are still very necessary, as they allow you to define and capture the changes in data that you are concerned about.

| username: 呢莫不爱吃鱼 | Original post link

I am looking at it in MinIO.

| username: Soysauce520 | Original post link

TiDB Binlog, BR log, and TiCDC can all record data changes. If you want to record SQL, you can use Binlog or enable the general log to record all SQL (including SELECT).

| username: yulei7633 | Original post link

Enabling binlog or ticdc, can you only view real-time data changes by connecting to the downstream database? This doesn’t seem very scientific.

| username: yulei7633 | Original post link

TiDB Binlog, BRLog, and TiCDC indeed record data changes. How can we view these data changes? Do we have to synchronize them to the downstream and view them in the downstream database?

| username: Soysauce520 | Original post link

Binlog can be parsed through repo without needing to go downstream. TiCDC has support for Kafka, but BR log does not yet have an official parsing tool.

| username: zhang_2023 | Original post link

There is on Grafana