TiUP's support for single-machine multi-IP deployment is not very good

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tiup 对单机多IP部署的支持不是很好

| username: TiDBer_Jbnrb1WG

The issue with deploying multiple IPs on a single machine using tiup is that it will repeatedly deploy node_exporter. For example, with 3 servers and 6 IPs, tiup will deploy 6 node_exporters. If services (such as tikv) listen on different IPs with the same port, the generated systemd service files will overwrite each other.

| username: Fly-bird | Original post link

Specify the IP and node in the configuration file.

| username: TiDBer_Jbnrb1WG | Original post link

Is there any documentation? Please provide some guidance.

| username: Kongdom | Original post link

Three servers? Six IPs?

| username: TiDBer_Jbnrb1WG | Original post link

Each server has two logical network cards, each configured with an IP address.

| username: 像风一样的男子 | Original post link

Shouldn’t you deploy the service on a single subnet? Are you planning to deploy with 2 network cards on 2 subnets?

| username: Kongdom | Original post link

So why not use 3 IPs? What is the reason for using 6 IPs?

| username: TiDBer_Jbnrb1WG | Original post link

In theory, this can achieve two instances on a single server each using a different network card, with different IP addresses and the same port.

| username: Kongdom | Original post link

So these are two clusters, right? I can’t figure it out~ :joy:

| username: forever | Original post link

What is the point of doing this? Both one and two are running on the same server.

| username: 像风一样的男子 | Original post link

Are the IPs of your two network cards in the same subnet?

| username: TiDBer_Jbnrb1WG | Original post link

In fact, the official team has considered this issue, and there is a configuration item listen_host that specifies the IP to listen on. However, it may not have been thoroughly tested, and using tiup for deployment can lead to the aforementioned problem.

For example, with the following configuration, a single server is configured with two IPs, and the expected result is to start two TiDB instances listening on the following addresses and ports respectively:

  • 10.0.0.1:4000
  • 10.0.0.2:4000

This way, although there are multiple instances on the same server, the port can be uniformly set to 4000.

In actual deployment, the generated configuration can take effect as the above listening addresses. However, when deploying two instances, the generated systemd configuration file names are both tidb-4000.service, causing the second instance’s file to overwrite the first one.

The consequence is that the first instance will not start.

tidb_servers:
  - host: 10.0.0.1
    listen_host: 10.0.0.1
    port: 4000
  - host: 10.0.0.2
    listen_host: 10.0.0.2
    port: 4000
| username: Kongdom | Original post link

This should be a simple hybrid deployment. You can refer to this deployment configuration and take a look at the detailed one.
image

| username: TiDBer_Jbnrb1WG | Original post link

Yes, I referred to the document you provided, but there will be issues in achieving the effect I mentioned.

| username: tidb菜鸟一只 | Original post link

You mean you have two network cards on your server, one with IP 10.0.0.1 and the other with IP 10.0.0.2, and you want to start two TiDB instances on this machine, both using port 4000, and bind them to the two network cards. However, because TiDB generates systemd services as tidb-4000.service, one of the instances will be overwritten, right?

It is generally recommended not to use the same port for two instances on the same host. Moreover, the status port of the tidb-server is actually bound to 0.0.0.0, so the two services cannot share the same status port…

| username: TiDBer_Jbnrb1WG | Original post link

You can currently use listen_host to specify the listening IP.

| username: tidb菜鸟一只 | Original post link

You can make a product suggestion. Theoretically, the official documentation does not prohibit the same host from doing this. There might be some issues with the service generation logic.

| username: Billmay表妹 | Original post link

I feel like there will be a conflict!