Can DM synchronization filter fields?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: DM同步能否过滤字段

| username: TiDBer_mGBoAnW9

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration]
[Attachments: Screenshots / Logs / Monitoring]
Can DM synchronization filter certain fields when syncing to the downstream database?

| username: 啦啦啦啦啦 | Original post link

It should not be possible. DM supports two methods of data filtering:
One is the database table blacklist/whitelist, with the smallest granularity being the table.
The other is binlog event filtering, such as not synchronizing truncate operations.

| username: TiDBer_mGBoAnW9 | Original post link

I read the documentation but couldn’t find anything related to filtering fields.

| username: liuis | Original post link

Cannot filter fields.

| username: 有猫万事足 | Original post link

It is based on binlog row mode replication.
Row mode will record the values of all fields.

Of course, in principle, reducing some fields is not impossible, but it should not be supported now.
Moreover, if you want to do desensitization through DM, there will still be sensitive data in the binlog, which is unavoidable.

| username: Hacker007 | Original post link

The smallest granularity is the table; field-level granularity is not supported yet.

| username: TiDB_C罗 | Original post link

Create a view to achieve it.

| username: coderv | Original post link

It won’t work.

| username: cassblanca | Original post link

Refer to this, it might help you: Manage Table Schemas for TiDB Data Migration | PingCAP Documentation Center

| username: redgame | Original post link

Create a transitional table.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.