What is the difference between using local relay-log and not using it in DM synchronization?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: DM同步使用本地relay-log和不使用有什么区别

| username: 舞动梦灵

I am preparing to deploy a new MySQL to TiDB synchronization environment. Previously, there were two old setups: one used local relay-log, and the other did not use relay-log during installation. It didn’t seem to make much difference. I personally think that using local relay-log might cause a slight delay. If relay-log is not used, does it directly analyze the binlog on the connected MySQL master and then convert it into SQL to transfer to the target TiDB? What impact does this have on MySQL?

| username: TiDBer_ivan0927 | Original post link

Using local relay-log can alleviate the pressure on the upstream database to a certain extent, improve synchronization stability and data persistence capabilities, but it may bring some performance overhead.

| username: Kongdom | Original post link

Advantages:

After enabling the relay log, multiple synchronization tasks from the same upstream can reuse the relay log that has already been pulled locally, reducing the pressure on the upstream.
By enabling the relay log in advance, DM will automatically retain sufficient logs locally, ensuring the normal progress of incremental tasks.

Disadvantages:

Since the relay log needs to be written to disk, this process will generate external IO and some CPU consumption, which may lengthen the entire synchronization chain, thereby increasing data synchronization latency. For synchronization tasks that are extremely sensitive to latency, it is temporarily not recommended to use the relay log.

| username: dba远航 | Original post link

In terms of stability, it is still recommended to enable the relay log.

| username: xmlianfeng | Original post link

Using relay will download the binlog locally, which can make synchronization more stable in case of unstable upstream and downstream network conditions.

| username: onlyacat | Original post link

After enabling the relay log, you need to pay more attention to the local disk space.

| username: 舞动梦灵 | Original post link

I want to know how much pressure it puts on the upstream MySQL if the relay log is not enabled. In the current environment, it doesn’t seem to feel any pressure or it’s not noticeable.

| username: Kongdom | Original post link

:thinking: This hasn’t been tested in practice, so I can’t provide any useful advice. If the current environment allows, you can try testing it.

| username: 舞动梦灵 | Original post link

I don’t want to enable it, mainly because I don’t want to add another parameter. Lazy :joy:
The old environment had local relay logs set up by the previous person, with three sets of databases and three data sources synchronized to one TiDB.
Later, two of the data sources failed to synchronize, so I redeployed these two on a new server with DM, without setting up local logs.
But it’s been a few months now, and I haven’t noticed any issues. Moreover, the database without local logs synchronizes seven to eight million records daily, and I haven’t seen any impact. The data sources are connected to the replica databases.
Now, in the new environment, I plan to connect it to the primary database source. Because previously, whenever the primary database added an index, there was master-slave delay, and TiDB inevitably had delays.
There’s no difference between connecting DM data sources to the primary or replica databases, right?

| username: Kongdom | Original post link

:yum: Haven’t tried it, you can start a new thread to discuss it.

| username: lemonade010 | Original post link

The relay log will be written to the local disk. Although it will write more data, the synchronization effect will be much better.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.