Deleting and rebuilding a table in BDR mode will cause CDC desynchronization

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: BDR模式删除表重建后会导致CDC不同步

| username: h5n1

Steps to Reproduce for v7.1.2

Scenario 1:

  1. Create a changefeed with BDR mode enabled (bdr-mode = true).
  2. Create tables on both the source and target ends, insert data on the source end, and the data is normally synchronized to the target end.
  3. After dropping and recreating the table on the source end, data insertion cannot be synchronized. Other tables also cannot be synchronized, although the CDC task status is normal.
    In non-BDR mode, synchronization in the above scenario has no issues.

cdc.txt (8.3 KB)

image
image1122×370 20.4 KB

Additionally, there is an issue where finding the TSO of DDL operations is inconvenient. The admin show ddl jobs command does not display it, and you need to check the logs, which is very inconvenient.

Scenario 2:
After changing the tidb_source_id on the target end to be inconsistent with the source end, CDC cannot synchronize.

There are no error messages in the cdc.log.

bdr-mode = true
[filter]
rules = [‘sbtest.*’]
[mounter]
worker-num = 24

2
| username: fubinz | Original post link

Thank you very much for your feedback. We are investigating this issue and will get back to you shortly.

3
| username: h5n1 | Original post link

Scenario Three:

  1. Pre-create the initialization table for sysbench in the target database.
  2. Use sysbench to initialize several tables in the source database (create tables + insert data).
  3. If the changefeed is created before executing the sysbench prepare command in the source database, the inserted data cannot be synchronized to the target database.
  4. If the changefeed is created after the sysbench prepare completes the create table and starts inserting data, the data can be synchronized. However, since the default synchronization start timestamp is the current time, the synchronized data will be incomplete.
4
| username: asddongmen | Original post link

For Scenario 1, I have tested and obtained the following results:

1. Create tables t1, t2, and t3 upstream and downstream.
2. Create a changefeed and enable BDR mode to synchronize from the source to the target.
3. Insert data into t1, normal synchronization.
4. Drop table t2.
5. Continue to insert data into t1 and t3, normal synchronization (dropping t2 does not affect t1).
6. Drop table t1.
7. Create table t1.
8. Insert data into t1, unable to synchronize normally.
9. Insert data into t3, normal synchronization (dropping t1 does not affect t3).
9. Pause + resume changefeed, normal synchronization resumes.

The issue in Scenario 1 has been identified and is being fixed.

5
| username: asddongmen | Original post link

Scenario 2 could not be reproduced. Could you please provide detailed execution steps?
Here’s what I did:

  1. Set up upstream and downstream
  2. Create changefeed and enable BDR mode
  3. Change upstream sourceID to 6, downstream to 7
  4. Insert data into upstream
  5. Data can be seen normally inserted in downstream
6
| username: h5n1 | Original post link

First, change the source_id, then create the changefeed.

7
| username: asddongmen | Original post link

Okay, thank you for the reply. I will test it again today.

8
| username: h5n1 | Original post link

Tested the previously described scenarios with version 7.1.3 today, and no issues have been found so far.