[TiDB Community Wisdom Collection] 2022 Community User Upgrade Guide & Comprehensive Column Upgrade Articles

translator_bot · June 23, 2024, 8:37am

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 【TiDB 社区智慧合集】2022 社区用户升级指南&专栏升级文章大全

| username: Billmay表妹

Good news for those preparing to upgrade! My cousin has prepared a super comprehensive upgrade guide for you to refer to:

Practical Experience - Recording a Major Version Upgrade (4.0.13→5.4.0)

I. Background

Currently, our company’s TiDB cluster is running databases for 5 business systems. These 5 businesses are relatively important systems for the company, characterized by high concurrent writes, high concurrent queries, or large batch data queries.
TiDB iterates quickly, and although TiDB v5.4.0 GA version is about a year later than TiDB v4.0.13, it has significant functional and performance improvements (tested) that can enhance write performance, query performance, especially multi-table joint queries (MPP).

II. Upgrade Purpose

Improve TiDB cluster write and query performance, with 5.4.0 being 10-15% better than 4.0.13.
The TiFlash component adds MPP (query pushdown) functionality, significantly improving TiDB cluster multi-table join query performance.
Enhance more user-friendly maintenance and management methods.
Replace old cluster servers with new ones, doubling the maximum capacity of the TiDB cluster.

“TiDB Cross-Version Upgrade” – Process Overview (3.0.10→5.3.0)

Upgrade Background

The original cluster version is too low, making maintenance difficult, so a version upgrade is decided.
Testing shows that v5.3.0 has significant performance improvements over v3.0.10.
Decided to upgrade TiDB v3.0.10 to TiDB v5.3.0.

Upgrade Method

This plan uses Dumpling+Lightning+TiDB Binlog.

【Upgrade Method Division】
Generally divided into downtime upgrade and non-downtime upgrade.
Based on the literal meaning, we can choose according to business requirements. If the business allows downtime upgrade, it is relatively safer and faster. If the business does not allow downtime, we mainly choose non-downtime upgrade.

Non-downtime upgrade according to the official documentation requires a specific method for rolling upgrade.
Rolling upgrade might be a good choice for us, but the issues are:

Business needs rollback, and our rollback plan usually requires full backup + incremental method, making rollback progress slow.

If the version gap is too large, continuous rolling upgrade increases uncontrollable factors.

The old version usually uses Ansible installation, while the new version wants to use tiup for management, making operations more complex.
# Due to various reasons, we finally decided to use Dumpling+Lightning+TiDB Binlog to effectively avoid a series of cumbersome issues.

Obtain relevant information
Create the target cluster for TiDB v5.3.0
Use Dumpling to export data from the original cluster
Use Lightning to import data into the target cluster
Start Drainer for incremental synchronization
Use sync-diff-inspector for data verification
Set up rollback path
Switch business

More details: 专栏 - 《TiDB跨版本升级》 --流程概述 | TiDB 社区

TiDB Upgrade and Case Sharing (TiDB v4.0.1 → v5.4.1)

Why Upgrade?

Low version TiDB peripheral components are not well supported, such as low version CDC has memory leak issues, etc.
TiDB panic bug triggered in actual work.
Under the background of cost reduction and efficiency improvement, enhance TiDB performance or throughput.
Contribute actual business scenarios to the community.

How to Upgrade?

TiDB upgrades are divided into downtime upgrade and non-downtime upgrade, with non-downtime upgrade further divided into smooth upgrade and forced upgrade (–force). Below is a brief explanation of the process, without too much detail, only introducing the core.

High-Concurrency Request TiDB Cluster Business Lossless Upgrade

TiDB Cross-Version Upgrade - First Attempt by Newcomers (3.0.8→5.4.0)

Upgrade Background

The original cluster version is too low, making maintenance difficult, so a version upgrade is decided.
Testing shows that v5.4.0 has significant performance improvements over v3.0.10.
Decided to upgrade TiDB v3.0.10 to TiDB v5.4.0.

Upgrade Method

This upgrade uses Dumpling+Lighting+TiDB binlog.

【Upgrade Method Division】
Generally divided into downtime upgrade and non-downtime upgrade.
Based on the literal meaning, we can choose according to business requirements. If the business allows downtime upgrade, it is relatively safer and faster. If the business does not allow downtime, we mainly choose non-downtime upgrade.

Non-downtime upgrade according to the official documentation requires a specific method for rolling upgrade.
Rolling upgrade might be a good choice for us, but the issues are:

Business needs rollback, and our rollback plan usually requires full backup + incremental method, making rollback progress slow.

If the version gap is too large, continuous rolling upgrade increases uncontrollable factors.

The old version usually uses Ansible installation, while the new version wants to use tiup for management, making operations more complex.
# Due to various reasons, we finally decided to use Dumpling+Lightning+TiDB Binlog to effectively avoid a series of cumbersome issues.