When using the tidbbinlog component for data synchronization, the pump generates logs, but the drainer cannot parse them

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 在使用tidbbinlog组件做数据同步时，pump会产生日志，但是drainer无法解析

| username: TiDBer_菜鸟

When using the tidbbinlog component for data synchronization, the pump generates logs, but the drainer cannot parse them. Checking the drainer logs shows an error for a non-existent drainer node, and it seems like the pump is not pushing to the correct drainer. The error log is as follows: [2024/01/24 09:57:38.813 +08:00] [ERROR] [pump.go:140] [“pump create pull binlogs client failed”] [id=esc-10-tidb4-pump-drainer:8250] [error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 10.xx.xx.xx:8250: i/o timeout"”] [errorVerbose=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 10.xx.xx.xx:8250: i/o timeout"\ngithub.com/pingcap/errors.AddStack\n\t/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201029093017-5a7df2af2ac7/errors.go:174\ngithub.com/pingcap/errors.Trace\n\t/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201029093017-5a7df2af2ac7/juju_adaptor.go:15\ngithub.com/pingcap/tidb-binlog/drainer.(*Pump).createPullBinlogsClient\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb-binlog/drainer/pump.go:238\ngithub.com/pingcap/tidb-binlog/drainer.(*Pump).PullBinlog.func1\n\t/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb-binlog/drainer/pump.go:139\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357”]

Has anyone encountered this issue before? The non-existent drainer address points to the TiFlash service node.

2
| username: 像风一样的男子 | Original post link

Are the drainer and pump components network-connected? You can test port 8250 on the server using telnet.

3
| username: 小龙虾爱大龙虾 | Original post link

Have you done any special operations on the pump before, such as forced scaling? You can check by using show pump status.

PS: Leave the last two digits of the IP in the logs to distinguish between different machines.

4
| username: TiDBer_菜鸟 | Original post link

By using the commands show pump status and show drainer status, I found that there are two non-existent nodes. It is possible that the previous administrator migrated the database without properly removing them.

5
| username: TiDBer_菜鸟 | Original post link

The network is connected.

6
| username: 小龙虾爱大龙虾 | Original post link

Use this binlogctl to handle it, the previous person must have operated abnormally.

7
| username: TiDBer_菜鸟 | Original post link

Successfully modified the status using binlogctl, then successfully took it offline.

8
| username: wangccsy | Original post link

Newbie learning. We are still using relational databases.

9
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.