Incorporating TiDB Clusters Deployed with Ansible into TiUP Management

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 关于ansible部署的tidb集群纳入tiup的管理

| username: TiDBer_nXXIyPM5

[TiDB Usage Environment] Testing
[TiDB Version] 4.0.11
[Background]
The online TiDB 4.0.11 is deployed using a company-customized Ansible script, which is basically modified according to the tidb-ansible-4.0.9 method. This time, we want to upgrade TiDB to version 6.1.2 using tiup. An error occurred when bundling to tiup.
[Reproduction Path] Using the command tiup cluster import -d /home/tidb/tidb-ansible will definitely cause an error.
[Encountered Problem: Problem Phenomenon and Impact]
The operations are as follows:

  1. Since the deployment was done using the standard tidb-ansible-4.0.9 script, I first roughly matched the current environment to the 4.0.9 ansible script deployment (the data directories for pd and tikv were not changed, not sure why renaming the directories would cause the service to fail to start, but it doesn’t seem important at the moment).
  2. Using tiup cluster import -d /home/tidb/tidb-ansible results in an error, please see screenshot 1 for details.
    Encountered Problems:
  3. Tiup import failed, unable to proceed with subsequent steps, the error message is unclear.
    [Resource Configuration]
    [Attachments: Screenshots/Logs/Monitoring]
    tikv2
    tikv21360×749 32.2 KB

    tikv1
    tikv11380×422 722 KB
2
| username: xfworld | Original post link

Renaming directories can cause inconsistencies in the information recorded in tidb-ansible and may result in errors.

In fact, it is best to follow the official practices for tool migration. If there are various modifications, it may lead to some issues, and these issues can be difficult to resolve.

Currently, it is recommended to use tiup to create a new TiDB cluster and migrate the data over, which will be simpler.

3
| username: TiDBer_nXXIyPM5 | Original post link

Thank you. Do you have any good solutions for data migration? Since the version is relatively old (4.0.11), we haven’t tried TiCDC yet. It seems that TiCDC is only available from version 5.0 onwards.

4
| username: xfworld | Original post link

There are many solutions for data migration:

  1. Dumping: Exporting SQL or CSV files by table logic.

  2. Binlog: The old CDC solution, which has single point of failure issues, but many companies have used it (it was the only option at the time).

  3. BR: Physical backup.

You can also combine these methods depending on the specific environment requirements. As long as the data can be moved over, it works. :smiley_cat:

5
| username: TiDBer_nXXIyPM5 | Original post link

May I ask, what is Binlong’s solution? I haven’t encountered it before. Since the online environment doesn’t have TiDB installed and is directly using TiKV, both dumping and BR cannot be used, which I have tried before. Also, I want to ask, is it certain that TiCDC cannot be used on relatively older versions?

6
| username: xfworld | Original post link

The binlog solution requires TiDB, and the capability has been available since version 3.X, so version 4.X is more stable. However, due to the single point of failure issue, TiCDC was used as a replacement in version 5.X.

For a pure TiKV cluster, you would need to write your own functionality to store data… It seems there are no other good solutions.

7
| username: TiDBer_nXXIyPM5 | Original post link

May I ask if it is possible to deploy a TiDB in a pure TiKV cluster and then use the binlog solution you mentioned? The data in the pure TiKV cluster is written by clients using go-client, and the data volume is very large.

8
| username: xfworld | Original post link

No, TiDB + TiKV uses the standard SQL protocol. The way you are using it is in KV API mode, which is not compatible…

9
| username: TiDBer_nXXIyPM5 | Original post link

Thank you. To confirm with you again, as I mentioned, using the pure TiKV method, BR, TiCDC, Binlog, and Dumping cannot perform similar data migration or backup tasks. Even if TiDB is deployed, if it does not use the standard SQL protocol, it cannot be used, right?

10
| username: xfworld | Original post link

You are using the pure client mode

image
image982×787 66.1 KB

In this mode, it does not interact with TiDB, so the mentioned ecosystem tools are not supported.

On GitHub, the only discussion is about the TXN API. If it’s the rawAPI, you probably have to figure it out on your own.

11
| username: TiDBer_nXXIyPM5 | Original post link

It’s a pure client mode, got it, thank you.

12
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.