There is a small amount of duplicate data when migrating from upstream sharded databases to downstream TiDB using DM

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: DM 从上游分库迁移至下游tidb中存在少量重复数据

| username: 麻烦是朋友

【TiDB Version】
DM Version:
Cluster type: dm
Cluster name: dm-calc
Cluster version: v6.1.0
Deploy user: tidb
SSH type: builtin

tiup dm -v
Starting component dm: /home/tidb/.tiup/components/dm/v1.10.2/tiup-dm -v
tiup version 1.10.2 tiup
Go Version: go1.18.3
Git Ref: v1.10.2
GitHash: 2de5b500c9fae6d418fa200ca150b8d5264d6b19
【Environment Description】
Upstream: Sharded environment.
Downstream: TiDB environment, downstream table structure has one more column ID than upstream, id bigint(20) unsigned NOT NULL /*T![auto_rand] AUTO_RANDOM(5), to prevent data conflicts.
DM synchronization status is normal:
“taskName”: “sharding_busi_detail”,
“taskStatus”: “Running”,
“sources”: [
“slave-mysql-x-xx-11851”,
“slave-mysql-x-xx-11852”,
“slave-mysql-x-xx-11853”,
“slave-mysql-x-xx-11854”,
“slave-mysql-x-xx-11855”,
“slave-mysql-x-xx-11856”,
“slave-mysql-x-xx-11857”
]
【Problem Encountered】
When querying the downstream duplicate data in the upstream, only one record is found, but there are duplicate data in the TiDB environment.
Downstream TiDB duplicate data: (grouping condition is the primary key of the upstream table) 183 records

1668046725382
16680467253821192×646 11.9 KB

Duplicate data 366 records
1668046921641
16680469216411183×642 32.2 KB

Upstream MySQL duplicate data query:
Only one record is found in each shard.
1668047148107
16680471481071198×646 46.8 KB

Could you provide some troubleshooting ideas? I have already read this document: 《专栏 - 加载中 | TiDB 社区

2
| username: lance6716 | Original post link

Could you please provide the upstream and downstream table structures?

3
| username: 麻烦是朋友 | Original post link

Upstream MySQL table structure:

CREATE TABLE `t_busi_detail` (
  `did` varchar(100) NOT NULL COMMENT 'Transaction detail ID',
  `ewb_no` varchar(100) NOT NULL COMMENT 'Waybill number',
  `serial_no` varchar(100) DEFAULT NULL COMMENT 'Serial number',
  `account_id` varchar(100) NOT NULL COMMENT 'Account ID',
  `deposit_site_id` bigint(20) NOT NULL COMMENT 'Bank ID',
  `deposit_site_name` varchar(100) NOT NULL COMMENT 'Bank name',
  `site_id` bigint(20) NOT NULL COMMENT 'Branch ID',
  `site_name` varchar(100) NOT NULL COMMENT 'Branch name',
  `distribution_site_id` bigint(20) NOT NULL COMMENT 'Distribution ID',
  `distribution_site_name` varchar(100) NOT NULL COMMENT 'Distribution name',
  `account_type` int(11) NOT NULL COMMENT 'Account type',
  `data_source` tinyint(4) NOT NULL COMMENT 'Data source (User: 1, System: 0)',
  `busi_id` varchar(100) NOT NULL COMMENT 'Business ID (stores the ID of the business data, billing business stores the bill ID)',
  `charge_calculate_id` varchar(100) NOT NULL COMMENT 'Billing business ID, non-billing data stores 0',
  `amount` decimal(10,2) NOT NULL,
  `fines_amount` decimal(10,2) DEFAULT NULL COMMENT 'Compensation amount',
  `transaction_type` tinyint(4) NOT NULL COMMENT 'Transaction type (0: income, 1: expenditure)',
  `balance` decimal(20,2) NOT NULL COMMENT 'Balance after transaction',
  `charge_itme_id` bigint(8) NOT NULL COMMENT 'Fee item ID',
  `parent_charge_item_id` bigint(8) DEFAULT NULL COMMENT 'Parent fee item ID',
  `send_time` datetime DEFAULT NULL COMMENT 'Sending time',
  `send_site_id` bigint(8) DEFAULT NULL COMMENT 'Sending site ID',
  `send_site_name` varchar(300) DEFAULT NULL COMMENT 'Sending site name',
  `send_site_id_1` bigint(8) DEFAULT NULL COMMENT 'Primary sending site ID',
  `send_site_name_1` varchar(300) DEFAULT NULL COMMENT 'Primary sending site name',
  `dispatch_site_id` bigint(8) DEFAULT NULL COMMENT 'Dispatch site ID',
  `dispatch_site_name` varchar(300) DEFAULT NULL COMMENT 'Dispatch site name',
  `sign_site_id` bigint(8) DEFAULT NULL COMMENT 'Sign site ID',
  `sign_site_name` varchar(300) DEFAULT NULL COMMENT 'Sign site name',
  `sign_site_id_1` bigint(8) DEFAULT NULL COMMENT 'Primary sign site ID',
  `sign_site_id_name` varchar(300) DEFAULT NULL COMMENT 'Primary sign site name',
  `sign_time` datetime DEFAULT NULL COMMENT 'Sign time',
  `fill_storehouse` tinyint(1) DEFAULT NULL COMMENT 'Fill warehouse',
  `weight` decimal(8,2) DEFAULT NULL COMMENT 'Actual weight of the waybill',
  `calc_weight` decimal(8,2) DEFAULT NULL COMMENT 'Settlement weight of the waybill',
  `piece` bigint(5) DEFAULT NULL COMMENT 'Number of pieces',
  `goods_type_id` bigint(8) DEFAULT NULL COMMENT 'Product type (dictionary table services_type)',
  `send_customer_id` varchar(100) DEFAULT NULL COMMENT 'Sending customer ID',
  `send_customer_name` varchar(300) DEFAULT NULL COMMENT 'Sending customer name',
  `hedge_flag` tinyint(4) DEFAULT NULL COMMENT 'Hedge (1: normal, 0: hedge)',
  `modify_time` datetime DEFAULT NULL COMMENT 'Modification time',
  `created_by` bigint(8) NOT NULL COMMENT 'Creator ID',
  `created_name` varchar(300) DEFAULT NULL COMMENT 'Creator name',
  `created_time` datetime NOT NULL COMMENT 'Creation time',
  `valid_flag` tinyint(4) NOT NULL COMMENT 'Related fee (1: related to account name, 0: just through the bank)',
  `remark` varchar(1000) DEFAULT NULL COMMENT 'Remark',
  `sys_remark` varchar(1000) DEFAULT NULL COMMENT 'System remark',
  `ts` datetime DEFAULT NULL COMMENT 'Timestamp',
  `var_standby_1` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_2` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_3` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_4` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_5` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_6` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_7` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_8` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_9` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_10` varchar(50) DEFAULT NULL COMMENT 'Standby field',
  `goods_category` int(10) DEFAULT NULL COMMENT 'Fragile goods identifier, 1: fragile, 0: no',
  `var_standby_12` int(10) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_13` int(10) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_14` int(10) DEFAULT NULL COMMENT 'Standby field',
  `var_standby_15` int(10) DEFAULT NULL COMMENT 'Standby field',
  `wd_balance` decimal(20,2) NOT NULL DEFAULT '0.00' COMMENT 'Current withdrawal balance',
  PRIMARY KEY (`did`,`created_time`),
  KEY `idx_t_busi_detail_1` (`busi_id`,`charge_itme_id`,`site_id`) USING BTREE,
  KEY `idx_t_busi_detail_2` (`site_id`,`created_time`,`charge_itme_id`) USING BTREE,
  KEY `idx_t_busi_detail_3` (`ewb_no`) USING BTREE,
  KEY `I_t_busi_detail_04` (`deposit_site_id`,`created_time`),
  KEY `idx_t_busi_detail_4` (`serial_no`) USING BTREE,
  KEY `idx_t_busi_detail_5` (`deposit_site_id`,`account_type`,`charge_itme_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='Transaction detail table'
/*!50500 PARTITION BY RANGE  COLUMNS(created_time)
(PARTITION P20220401 VALUES LESS THAN ('2022-04-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20220402 VALUES LESS THAN ('2022-04-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20220403 VALUES LESS THAN ('2022-05-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20220501 VALUES LESS THAN ('2022-05-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20220502 VALUES LESS THAN ('2022-05-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20220503 VALUES LESS THAN ('2022-06-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20220601 VALUES LESS THAN ('2022-06-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20220602 VALUES LESS THAN ('2022-06-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20220603 VALUES LESS THAN ('2022-07-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20220701 VALUES LESS THAN ('2022-07-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20220702 VALUES LESS THAN ('2022-07-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20220703 VALUES LESS THAN ('2022-08-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20220801 VALUES LESS THAN ('2022-08-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20220802 VALUES LESS THAN ('2022-08-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20220803 VALUES LESS THAN ('2022-09-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20220901 VALUES LESS THAN ('2022-09-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20220902 VALUES LESS THAN ('2022-09-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20220903 VALUES LESS THAN ('2022-10-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20221001 VALUES LESS THAN ('2022-10-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20221002 VALUES LESS THAN ('2022-10-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20221003 VALUES LESS THAN ('2022-11-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20221101 VALUES LESS THAN ('2022-11-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20221102 VALUES LESS THAN ('2022-11-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20221103 VALUES LESS THAN ('2022-12-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20221201 VALUES LESS THAN ('2022-12-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20221202 VALUES LESS THAN ('2022-12-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20221203 VALUES LESS THAN ('2023-01-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20230101 VALUES LESS THAN ('2023-01-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20230102 VALUES LESS THAN ('2023-01-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20230103 VALUES LESS THAN ('2023-02-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20230201 VALUES LESS THAN ('2023-02-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20230202 VALUES LESS THAN ('2023-02-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20230203 VALUES LESS THAN ('2023-03-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20230301 VALUES LESS THAN ('2023-03-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20230302 VALUES LESS THAN ('2023-03-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20230303 VALUES LESS THAN ('2023-04-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20230401 VALUES LESS THAN ('2023-04-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20230402 VALUES LESS THAN ('2023-04-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20230403 VALUES LESS THAN ('2023-05-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20230501 VALUES LESS THAN ('2023-05-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20230502 VALUES LESS THAN ('2023-05-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20230503 VALUES LESS THAN ('2023-06-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20230601 VALUES LESS THAN ('2023-06-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20230602 VALUES LESS THAN ('2023-06-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20230603 VALUES LESS THAN ('2023-07-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20230701 VALUES LESS THAN ('2023-07-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20230702 VALUES LESS THAN ('2023-07-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20230703 VALUES LESS THAN ('2023-08-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20230801 VALUES LESS THAN ('2023-08-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20230802 VALUES LESS THAN ('2023-08-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20230803 VALUES LESS THAN ('2023-09-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20230901 VALUES LESS THAN ('2023-09-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20230902 VALUES LESS THAN ('2023-09-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20230903 VALUES LESS THAN ('2023-10-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20231001 VALUES LESS THAN ('2023-10-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20231002 VALUES LESS THAN ('2023-10-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20231003 VALUES LESS THAN ('2023-11-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20231101 VALUES LESS THAN ('2023-11-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20231102 VALUES LESS THAN ('2023-11-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20231103 VALUES LESS THAN ('2023-12-01 00:00:00') ENGINE = InnoDB,
 PARTITION P20231201 VALUES LESS THAN ('2023-12-10 00:00:00') ENGINE = InnoDB,
 PARTITION P20231202 VALUES LESS THAN ('2023-12-20 00:00:00') ENGINE = InnoDB,
 PARTITION P20231203 VALUES LESS THAN ('2024-01-01 00:00:00') ENGINE = InnoDB,
 PARTITION P_MAX VALUES LESS THAN (MAXVALUE) ENGINE = InnoDB) */

Downstream TiDB table structure:

CREATE TABLE `t_busi_detail` (
  `id` bigint(20) unsigned NOT NULL /*T![auto_rand] AUTO_RANDOM(5) */,
  `did` varchar(100) NOT NULL COMMENT 'Transaction detail ID',
  `ewb_no` varchar(100) NOT NULL COMMENT 'Waybill number',
  `serial_no` varchar(100) DEFAULT NULL COMMENT 'Serial number',
  `account_id` varchar(100) NOT NULL COMMENT 'Account ID',
  `deposit_site_id` bigint(20) NOT NULL COMMENT 'Bank ID',
  `deposit_site_name` varchar(100) NOT NULL COMMENT 'Bank name',
  `site_id` bigint(20) NOT NULL COMMENT 'Branch ID',
  `site_name` varchar(100) NOT NULL COMMENT 'Branch name',
  `distribution_site_id` bigint(20) NOT NULL COMMENT 'Distribution ID',
  `distribution_site_name` varchar(100) NOT NULL COMMENT 'Distribution name',
  `account_type` int(11) NOT NULL COMMENT 'Account type',
  `data_source` tinyint(4) NOT NULL COMMENT 'Data source (User: 1, System: 0)',
  `busi_id` varchar(100) NOT NULL COMMENT 'Business ID (stores the ID of the business data, billing business stores the bill ID)',
  `charge_calculate_id` varchar(100) NOT NULL COMMENT 'Billing business ID, non-billing data stores 0',
  `amount` decimal(10,2) NOT NULL,
  `fines_amount` decimal(10,2) DEFAULT NULL COMMENT 'Compensation amount',
  `transaction_type` tinyint(4) NOT NULL COMMENT 'Transaction type (0: income, 1: expenditure)',
  `balance` decimal(20,2) NOT NULL COMMENT 'Balance after transaction',
  `charge_itme_id` bigint(8) NOT NULL COMMENT 'Fee item ID',
  `parent_charge_item_id` bigint(8) DEFAULT NULL COMMENT 'Parent fee item ID',
  `send_time` datetime DEFAULT NULL COMMENT 'Sending time',
  `send_site_id` bigint(8) DEFAULT NULL COMMENT 'Sending site ID',
  `send_site_name` varchar(300) DEFAULT NULL COMMENT 'Sending site name',
  `send_site_id_1` bigint(8) DEFAULT NULL COMMENT 'Primary sending site ID',
  `send_site_name_1` varchar(300) DEFAULT NULL COMMENT 'Primary sending site name',
  `dispatch_site_id` bigint(8) DEFAULT NULL COMMENT 'Dispatch site ID',
  `dispatch_site_name` varchar(300) DEFAULT NULL COMMENT 'Dispatch site name',
  `sign_site_id` bigint(8) DEFAULT NULL COMMENT 'Sign site ID',
  `sign_site_name` varchar(300) DEFAULT NULL COMMENT 'Sign site name',
  `sign_site_id_1` bigint(8) DEFAULT NULL COMMENT 'Primary sign site ID',
  `sign_site_id_name` varchar(300) DEFAULT NULL COMMENT 'Primary sign site name',
  `sign_time` datetime DEFAULT NULL COMMENT 'Sign time',
  `fill_storehouse` tinyint(1) DEFAULT NULL COMMENT 'Fill warehouse',
  `weight` decimal(8,2) DEFAULT NULL COMMENT 'Actual weight of the waybill',
  `calc_weight` decimal(8,2) DEFAULT NULL COMMENT 'Settlement weight of the waybill',
  `piece` bigint(5) DEFAULT NULL COMMENT 'Number of pieces',
  `goods_type_id` bigint(8) DEFAULT NULL COMMENT 'Product type (dictionary table services_type)',
  `send_customer_id` varchar(100) DEFAULT NULL COMMENT 'Sending customer ID',
  `send_customer_name` varchar(300) DEFAULT NULL COMMENT 'Sending customer name',
  `hedge_flag` tinyint(4) DEFAULT NULL COMMENT 'Hedge (1: normal, 0: hedge
4
| username: Min_Chen | Original post link

Hello, please provide the task configuration YAML. Additionally, did you perform any operations during the synchronization period? For example, DDL or any exceptions occurred. If so, please provide the corresponding timestamps and DM worker logs.

5
| username: 麻烦是朋友 | Original post link

Configuration file:
cat sharding_busi_detail.yaml
name: “sharding_busi_detail”
task-mode: “all”

target-database:
host: “172.17.x.xx”
port: xxxx
user: “root”
password: “n7NLD6TpEuGe4kEnVhgxy0Dr67IMIXA=”

routes:
route-rule-0:
schema-pattern: “calcdb0*”
target-schema: “calcdb”

route-rule-1:
schema-pattern: “calcdb0*”
table-pattern: “t_busi_detail”
target-schema: “calcdb”
target-table: “t_busi_detail”

filters:
filter-rule-0:
schema-pattern: “calcdb0*”
events: [“all dml”]
action: Do

filter-rule-1:
schema-pattern: “calcdb0*”
table-pattern: “t_busi_detail”
events: [“truncate table”, “drop table”]
sql-pattern: [“ALTER\s+TABLE[\s\S]*ADD\s+PARTITION”, “ALTER\s+TABLE[\s\S]*DROP\s+PARTITION”]
action: Ignore

mysql-instances:

source-id: “slave-mysql-x-xx-11851”
block-allow-list: “bw-rule-1”
route-rules: [“route-rule-0”, “route-rule-1”]
filter-rules: [“filter-rule-0”, “filter-rule-1”]

source-id: “slave-mysql-x-xx-11852”
block-allow-list: “bw-rule-1”
route-rules: [“route-rule-0”, “route-rule-1”]
filter-rules: [“filter-rule-0”, “filter-rule-1”]

source-id: “slave-mysql-x-xx-11853”
block-allow-list: “bw-rule-1”
route-rules: [“route-rule-0”, “route-rule-1”]
filter-rules: [“filter-rule-0”, “filter-rule-1”]

source-id: “slave-mysql-x-xx-11854”
block-allow-list: “bw-rule-1”
route-rules: [“route-rule-0”, “route-rule-1”]
filter-rules: [“filter-rule-0”, “filter-rule-1”]

source-id: “slave-mysql-x-xx-11855”
block-allow-list: “bw-rule-1”
route-rules: [“route-rule-0”, “route-rule-1”]
filter-rules: [“filter-rule-0”, “filter-rule-1”]

source-id: “slave-mysql-x-xx-11856”
block-allow-list: “bw-rule-1”
route-rules: [“route-rule-0”, “route-rule-1”]
filter-rules: [“filter-rule-0”, “filter-rule-1”]

source-id: “slave-mysql-x-xx-11857”
block-allow-list: “bw-rule-1”
route-rules: [“route-rule-0”, “route-rule-1”]
filter-rules: [“filter-rule-0”, “filter-rule-1”]

block-allow-list:
bw-rule-1:
do-dbs: [“calcdb0*”]
do-tables:
- db-name: “calcdb0*”
tbl-name: “t_busi_detail”

After initializing the data, it will stop because there is an extra column downstream. Follow the upstream table structure and perform the following operations:
tiup dmctl --master-addr 172.17.x.xxx:8261 binlog-schema update -s slave-mysql-x-xx-11851 sharding_busi_detail calcdb01 t_sub_account_busi_detail --from-source
tiup dmctl --master-addr 172.17.x.xxx:8261 binlog-schema update -s slave-mysql-x-xx-11852 sharding_busi_detail calcdb02 t_sub_account_busi_detail --from-source
tiup dmctl --master-addr 172.17.x.xxx:8261 binlog-schema update -s slave-mysql-x-xx-11853 sharding_busi_detail calcdb03 t_sub_account_busi_detail --from-source
tiup dmctl --master-addr 172.17.x.xxx:8261 binlog-schema update -s slave-mysql-x-xx-11854 sharding_busi_detail calcdb04 t_sub_account_busi_detail --from-source
tiup dmctl --master-addr 172.17.x.xxx:8261 binlog-schema update -s slave-mysql-x-xx-11855 sharding_busi_detail calcdb05 t_sub_account_busi_detail --from-source
tiup dmctl --master-addr 172.17.x.xxx:8261 binlog-schema update -s slave-mysql-x-xx-11856 sharding_busi_detail calcdb06 t_sub_account_busi_detail --from-source
tiup dmctl --master-addr 172.17.x.xxx:8261 binlog-schema update -s slave-mysql-x-xx-11857 sharding_busi_detail calcdb07 t_sub_account_busi_detail --from-source

tiup dmctl --master-addr 172.17.x.xxx:8261 resume-task sharding_busi_detail

tiup dmctl --master-addr 172.17.x.xxx:8261 check-task sharding_busi_detail.yaml

6
| username: 麻烦是朋友 | Original post link

Ultimately, create a composite primary key downstream and handle the small number of duplicates at the source separately.

7
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.