TiCDC Unable to Capture Data After Renaming Table

translator_bot · June 23, 2024, 2:32am

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ticdc 重命名表采集不到数据

| username: 大飞飞呀

【TiDB Usage Environment】Production\Test Environment\POC
【TiDB Version】
v5.0.4
【Encountered Problem】
After starting ticdc and applying the filter rule test_yafei.test4*, renaming the table test3 to test4 and writing data into test4, ticdc does not collect the data.

【Reproduction Path】What operations were performed that led to the problem
【Problem Phenomenon and Impact】

translator_bot · June 23, 2024, 2:32am

| username: HACK | Original post link

Did he reconfigure the table mapping, and then the new table after renaming doesn’t replicate data?

translator_bot · June 23, 2024, 2:32am

| username: h5n1 | Original post link

CDC captures changelog based on table_id, and filters according to the filtering rules when synchronizing downstream. Since test3 was not originally synchronized, its changelog will not be captured. In future versions, changing a non-filtered table name to a filtered table name will result in an error.

github.com/pingcap/tiflow

[TiCDC Bug Report] lost a table when rename the table in the upstream

opened 09:20AM - 24 Nov 21 UTC

closed 06:08AM - 02 Sep 22 UTC

zier-one

type/bug component/sink severity/major area/ticdc affects-5.1 affects-5.0 affects-4.0 affects-6.0 affects-6.1 affects-6.2

### What did you do? 1. create a `changefeed.toml`: ``` [filter] rules = […'test.tt'] ``` 2. create a changefeed with the `changefeed.toml`: `$ cdc cli changefeed create --sink-uri="blackhole://" --config changefeed.toml` 3. create table `tt` and insert a row in the upstream: ``` MySQL [test]> create table tt ( id int primary key ); Query OK, 0 rows affected (0.11 sec) MySQL [test]> insert into tt value (1); Query OK, 1 row affected (0.01 sec) ``` 4. create table `tt_2` and insert some rows in the upstream: ``` MySQL [test]> create table tt_2 ( id int primary key ); Query OK, 0 rows affected (0.10 sec) MySQL [test]> insert into tt_2 value (1); Query OK, 1 row affected (0.02 sec) MySQL [test]> insert into tt_2 value (2); Query OK, 1 row affected (0.00 sec) ``` 5. add a column for table `tt_2` : ``` MySQL [test]> alter table tt_2 add column a int ; Query OK, 0 rows affected (0.26 sec) ``` 6. drop table `tt`: ``` MySQL [test]> drop table tt; Query OK, 0 rows affected (0.24 sec) ``` 7. rename `tt_2` to `tt`: ``` MySQL [test]> rename table tt_2 to tt; Query OK, 0 rows affected (0.09 sec) ``` 8. insert some rows to the new `tt`: ``` MySQL [test]> insert into tt value (3,1); Query OK, 1 row affected (0.01 sec) MySQL [test]> insert into tt value (4,1); Query OK, 1 row affected (0.01 sec) ``` ### What did you expect to see? TiCDC will capture all the changelogs from table `tt`, whether this table has been renamed or not. ### What did you see instead? The program only captures table `tt` which have not been renamed. the BlockHole log: ``` # grep BlockHoleSink cdc-*/ticdc.log | grep -v FlushRowChangedEvents | grep -v Checkpoint cdc-2/ticdc.log:[2021/11/24 17:06:47.590 +08:00] [DEBUG] [black_hole.go:69] ["BlockHoleSink: DDL Event"] [ddl="{\"StartTs\":429324974450016261,\"CommitTs\":429324974450016264,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"tt\",\"TableID\":45,\"ColumnInfo\":[{\"Name\":\"id\",\"Type\":3}]},\"PreTableInfo\":null,\"Query\":\"create table tt ( id int primary key )\",\"Type\":3}"] cdc-2/ticdc.log:[2021/11/24 17:06:58.527 +08:00] [DEBUG] [black_hole.go:41] ["BlockHoleSink: EmitRowChangedEvents"] [row="{\"start-ts\":429324977529946113,\"commit-ts\":429324977529946114,\"row-id\":1,\"table\":{\"db-name\":\"test\",\"tbl-name\":\"tt\",\"tbl-id\":45,\"is-partition\":false},\"table-info-version\":429324974450016264,\"replica-id\":0,\"columns\":[{\"name\":\"id\",\"type\":3,\"flag\":11,\"value\":1}],\"pre-columns\":null}"] cdc-2/ticdc.log:[2021/11/24 17:09:08.735 +08:00] [DEBUG] [black_hole.go:69] ["BlockHoleSink: DDL Event"] [ddl="{\"StartTs\":429325011241664520,\"CommitTs\":429325011241664522,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"tt_2\",\"TableID\":47,\"ColumnInfo\":[{\"Name\":\"id\",\"Type\":3}]},\"PreTableInfo\":null,\"Query\":\"create table tt_2 ( id int primary key )\",\"Type\":3}"] cdc-2/ticdc.log:[2021/11/24 17:10:03.820 +08:00] [DEBUG] [black_hole.go:69] ["BlockHoleSink: DDL Event"] [ddl="{\"StartTs\":429325025856192513,\"CommitTs\":429325025856192515,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"tt_2\",\"TableID\":47,\"ColumnInfo\":[{\"Name\":\"id\",\"Type\":3},{\"Name\":\"a\",\"Type\":3}]},\"PreTableInfo\":{\"Schema\":\"test\",\"Table\":\"tt_2\",\"TableID\":47,\"ColumnInfo\":[{\"Name\":\"id\",\"Type\":3}]},\"Query\":\"alter table tt_2 add column a int\",\"Type\":5}"] cdc-2/ticdc.log:[2021/11/24 17:10:14.836 +08:00] [DEBUG] [black_hole.go:69] ["BlockHoleSink: DDL Event"] [ddl="{\"StartTs\":429325028687609857,\"CommitTs\":429325028687609859,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"tt\",\"TableID\":45,\"ColumnInfo\":[{\"Name\":\"id\",\"Type\":3}]},\"PreTableInfo\":{\"Schema\":\"test\",\"Table\":\"tt\",\"TableID\":45,\"ColumnInfo\":[{\"Name\":\"id\",\"Type\":3}]},\"Query\":\"drop table tt\",\"Type\":4}"] cdc-2/ticdc.log:[2021/11/24 17:10:24.592 +08:00] [DEBUG] [black_hole.go:69] ["BlockHoleSink: DDL Event"] [ddl="{\"StartTs\":429325031479181318,\"CommitTs\":429325031479181319,\"TableInfo\":{\"Schema\":\"test\",\"Table\":\"tt\",\"TableID\":47,\"ColumnInfo\":[{\"Name\":\"id\",\"Type\":3},{\"Name\":\"a\",\"Type\":3}]},\"PreTableInfo\":{\"Schema\":\"test\",\"Table\":\"tt_2\",\"TableID\":47,\"ColumnInfo\":[{\"Name\":\"id\",\"Type\":3},{\"Name\":\"a\",\"Type\":3}]},\"Query\":\"rename table tt_2 to tt\",\"Type\":14}"] ``` and there is another question: why the DDLs from `tt_2` are inputted to the BlackHole sink? they should be filtered. ### Versions of the cluster Upstream TiDB cluster version (execute `SELECT tidb_version();` in a MySQL client): ```console v4.0.14 ``` TiCDC version (execute `cdc version`): ```console v4.0.14 ```

github.com/pingcap/tiflow

ddl_puller(ticdc): discard DDLs that irrelevant with changefeed

pingcap:master ← asddongmen:filter_ddl

opened 02:55AM - 24 Jul 22 UTC

asddongmen

+947 -163

### What problem does this PR solve?  Issue Number: ref #6447 #3607 ### What is changed and how it works? After this PR, TiCDC filters out DDLs that are not related to changefeed in `ddl_puller`, which can reduces the latency and memory consumption of replicating DDLs. It is worth noting that before this PR, for `rename table` DDL, cdc will filter it with the new table name. After this PR, cdc will filter `rename table` by its old table name, which is more in line with the semantics of the filter rule. ``` For example: ---------------------------------- filter: test.t1(white list) DDL1: rename test.t1 to test.t11 DDL2: rename test.t11 to test.t1 ---------------------------------- Before this pr: CDC will ignore DDL1, sink DDL2. It will cause upstream and downstream inconsistent. Result: 1. Upstream has table 't11', but downstream still holds table 't1'. 2. When sink DDL2, if downstream don't have the table 't11' before, it may raise error in CDC. (This changefeed don't care about the 't11' actually) ---------------------------------- After this pr: CDC will sink DDL1, report error DDL2. Result: 1. Both upstream and downstream will have table 't11'. 2. CDC reports error for DDL2. More details in following explaination. (This changefeed don't care about the table 't11' before. User need confirm the error before sinking DDL2) ------------------------------------ ``` And, if the user wants to rename a table with an old table name that does not match filter.rule to a new table name that matches filter.rule, then an error will be reported directly. For example, we have `filter.rule = ['test.t1']`, and there are tables name `test.t11` in the upstream TiDB: 1. excute `rename table 'test.t11' to 'test.t1'` will report an error. In addition, if you rename multiple tables in a single one `rename table` DDL, TiCDC will apply the rule mention above to every single subordinate `rename table` DDL jobs. For example, we have `filter.rule = ['test.t1', `test.t3`]`, and there are tables name `test.t1`, `test.t2` in the upstream TiDB: 1. excute `rename table 'test.t1' to 'test.t11', 'test.t2' to 'test.t22'`, the `rename table 'test.t1'` will be replicated and 'test.t2' to 'test.t22'` will be filtered out since `test.t1` match `filter.rule` but `test.t2` does not. 3. excuet `rename table 'test.t2' to 'test.t3', 'test.t1' to 'test.t11'` will report an error, since `test.t2` does not match `filter.rule` but `test.t3` does, TiCDC do not allow such behavior. ### Check List #### Tests - [x] Unit test - [x] Integration test - [x] Manual test (add detailed scripts or steps below) 1. start cdc server 4. create a chengfeed with `filter.rule = ['test1.t1', 'test1.t2', 'test1.t3', 'test1.t4'] ` 5. create t1, t2, t3, t4, t5, t6 in upstream tidb. 6. exec : `rename table t1 to t11` (replicated) 7. exec : `rename table t11 to t1` (error) ```shell ➜ bin git:(filter_ddl) ✗ ./cdc cli changefeed list [ { "id": "test-1", "namespace": "default", "summary": { "state": "error", "tso": 435389327982985217, "checkpoint": "2022-08-19 11:07:58.939", "error": { "addr": "127.0.0.1:8300", "code": "CDC:ErrSyncRenameTablesFailed", "message": "[CDC:ErrSyncRenameTablesFailed]table's old name is not in filter rule, and its new name in filter rule table id '108', ddl query: [rename table t11 to t1], it's an unexpected behavior, if you want to replicate this table, please add its old name to filter rule." } } } ] ``` 8. exec: `rename table t2 to t22, t3 to t33` (replicated) 9. exec: `rename table t22 to t222, t33 to t333` (discard) 10. exec: `rename table t222 to t2, t333 to t3` (error) ```shell ➜ bin git:(filter_ddl) ✗ ./cdc cli changefeed list [ { "id": "test-1", "namespace": "default", "summary": { "state": "error", "tso": 435389397673967617, "checkpoint": "2022-08-19 11:12:24.789", "error": { "addr": "127.0.0.1:8300", "code": "CDC:ErrSyncRenameTablesFailed", "message": "[CDC:ErrSyncRenameTablesFailed]table's old name is not in filter rule, and its new name in filter rule table id '110', ddl query: [rename table t222 to t2, t333 to t3], it's an unexpected behavior, if you want to replicate this table, please add its old name to filter rule." } } } ] ``` 11. exec: `rename table t4 to t44, t5 to t55` It will be splited into two parts, the `rename table t4 to t44` part is replicated and `t5 to t55` is ignored. You can learn more discussion detail in this internal doc: https://pingcap.feishu.cn/wiki/wikcnJ6SaB5cGLeG5OrM4nf6MPg #### Questions ##### Will it cause performance regression or break compatibility? ##### Do you need to update user documentation, design documentation or monitoring documentation? ### Release note ```release-note Improve replicating performance by discard DDLs that irrelevant with a changefeed. ```

translator_bot · June 23, 2024, 2:32am

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.