What is a DM Cluster?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 啥是DM集群

| username: TiDBer_fiJaQjDR

Can you describe the DM cluster?

| username: xfworld | Original post link

TiDB Data Migration (DM) is a convenient data migration tool that supports full data migration and incremental data synchronization from databases compatible with the MySQL protocol (MySQL, MariaDB, Aurora MySQL) to TiDB. Using the DM tool helps simplify the data migration process and reduce data migration operation and maintenance costs.

Product Features

  • Maintains MySQL compatibility just like TiDB. Highly compatible with the MySQL protocol, and commonly used features and syntax of MySQL 5.7 and MySQL 8.0.
  • Supports DML & DDL event synchronization. Supports parsing and synchronizing DML and DDL events in binlog.
  • Supports database and table merging synchronization mode. Easily merge and synchronize data from various sharded MySQL instances upstream into a single table in TiDB downstream. Supports custom synchronization rules to accommodate various synchronization needs and automatically identifies and handles DDL changes in upstream sharded MySQL, significantly reducing operation and maintenance costs.
  • Built-in filters to flexibly adapt to different scenarios. Supports filtering MySQL binlog events during data synchronization using predefined event types, regular expressions, SQL expressions, and other methods.
  • Centralized management. DM supports cluster scales of thousands of nodes, allowing simultaneous operation and centralized management of numerous data migration and synchronization tasks.
  • Optimization for third-party Online Schema Change tool changes. In the MySQL ecosystem, tools like gh-ost and pt-osc are widely used. DM has special optimizations for their change processes to avoid migrating unnecessary intermediate data. For more details, refer to online-ddl.
  • High availability. Supports free scheduling of migration tasks across different nodes, and a small number of worker node failures will not affect ongoing tasks.

It’s very easy to search for it… :rofl: :rofl: :rofl: :rofl: :rofl: :rofl: :rofl: :rofl: :rofl:

| username: zhang_2023 | Original post link

Data Migration High Availability Mechanism | PingCAP Documentation Center

| username: 友利奈绪 | Original post link

TiDB Data Migration (DM) is an open-source data migration tool used to migrate data from traditional databases to the distributed database TiDB or to synchronize data between TiDB clusters. It is efficient, stable, and easy to use, making it suitable for various data migration and synchronization scenarios.

| username: TiDBer_fiJaQjDR | Original post link

It’s not the DM tool, it’s the DM cluster.

| username: Jack-li | Original post link

DM cluster is a highly available and scalable database solution that achieves data synchronization, load balancing, and failover through various cluster modes (such as data guard, read-write separation, shared storage cluster, etc.) to ensure data consistency and service continuity.

| username: xfworld | Original post link

In order to meet the convenience and adaptability of DM tool operation and maintenance, an architecture is needed to support it. The DM cluster is an instance designed to meet this architecture…

| username: zhaokede | Original post link

:+1: Learning

| username: TiDBer_QYr0vohO | Original post link

DM is a convenient data migration tool that supports full data migration and incremental data synchronization from databases compatible with the MySQL protocol (MySQL, MariaDB, Aurora MySQL) to TiDB. Using the DM tool helps simplify the data migration process and reduce the operational costs of data migration.

| username: onlyacat | Original post link

DM can be single-node.
However, setting up three instances distributed in different locations can achieve high availability.
DM-master and DM-worker can be roughly compared to PD and TiDB.

| username: QH琉璃 | Original post link

Learn a bit.

| username: TiDBer_JUi6UvZm | Original post link

Got it.

| username: terry0219 | Original post link

It’s a migration tool that can synchronize MySQL data to TiDB.

| username: yulei7633 | Original post link

Simply put, it is a migration tool. The upstream can be TiDB or MySQL, and the downstream can also be TiDB, MySQL, Kafka, etc.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.