Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.Original topic: 我和TiDB的故事 | 遇上你是我的缘
|
username: h5n1
[First Encounter]
As an Oracle DBA (with no future) who has always been in the traditional industry, I have had more exposure to open-source databases, mainly MySQL. My attitude towards domestic databases has been that they are often just modified versions of open-source ones, and the chances of using them in core systems are slim.
I don’t remember which year or how many snowfalls ago, but I occasionally heard about TiDB, a relatively popular open-source database, probably around version 2.0. Some friends in a QQ group tested TiDB and reported that it was too slow and not as good as expected. At that time, I thought I might never use TiDB in my life.
[Encounter]
One day, TiDB personnel came to our company to discuss TiDB, giving me a preliminary understanding of it. Gradually, I learned about TiDB’s developments through my social circle, such as the release of TiFlash in version 4.0. Until one day in April 2021, I received a request to sync some tables from an RDS production database to a MySQL instance using Kafka for partial queries. We used a machine with only a few SAS disks, and the data sync maxed out the IO. We needed another solution.
When they approached me, I thought of TiDB. The next task was for them to find machines while I researched TiDB. After some maneuvering, we found five machines, but they only had 12 SAS disks in total. Running them might not even match the performance of an SSD laptop, but we had to make do. To distribute the IO as much as possible, we ran TiDB and PD on two machines and TiKV on three machines, each with four SAS disks in RAID 0.
With the help of leader balancing and RAID caching, we easily handled the data sync pressure. The data volume was around 8TB, and disk reads through primary and unique keys were within tens of milliseconds. However, large data queries with massive disk reads were too slow.
[Luck]
To learn TiDB, asktug was a necessary path. I asked many questions on asktug and clearly remember trying to solve the issue of setting up local images in a mixed X86 and ARM environment (the system couldn’t connect to the internet, so we had to download packages and upload them to the server). Mr. Ao @ hey-hoho and his team stayed up late to research the solution and wrote a dedicated article (专栏 - TiDB在X86和ARM混合平台下的离线部署和升级 | TiDB 社区). Thanks again, Mr. Ao.
Maybe because I was active on tug during that time, one day, I received a WeChat message from my cousin (finally, a girl took the initiative to message me) asking if I would like to try being a moderator. I was both happy and scared—happy to be recognized to some extent, but scared of misleading others and embarrassing myself. After some internal struggle, I agreed, and thus began my journey with TiDB.
After some time on tug, not only did I improve my knowledge and skills, but I also met some experts and enjoyed helping others solve problems. I also felt the charm of distributed systems and open-source. All of this stemmed from my cousin’s invitation, which was a turning point for me. I’m very lucky to have met her, and I thank her.
[Evangelism]
One reason I recommend TiDB is the influence of its official documentation. Compared to the documentation of some big companies, TiDB’s documentation is very rich and includes many blog posts explaining principles. Asktug also has many rich cases, which is rare among domestic databases. I was impressed by this open attitude. As I learned more about TiDB, I started writing articles about it on tug and my company’s public account, both as notes for myself and to spread knowledge. (Column address: https://tidb.net/u/h5n1/post/alz, company public account: https://mp.weixin.qq.com/s/56rHLPIAyWAWcEYXeaynTQ).
To help more people in the company understand TiDB, I organized an internal training session, explaining TiDB’s basic architecture, key features, and industry cases. To attract more attendees, I applied for some giveaways from my cousin, who not only generously helped but also acted as a model. Unfortunately, only 78 people attended the training, which was below expectations.
[Blessings]
Over the past year, I have witnessed the improvements and changes brought by each TiDB version and had the honor of participating in the 6.0 book rush event, feeling the community’s vitality once again. I wish TiDB, a truly open-source database, all the best. Additionally, I have some suggestions for TiDB:
- Strengthen traditional industries
TiDB’s main active users are currently in the internet industry, with less focus on traditional industries. Traditional industries have more security requirements, such as for online image file downloads, PD component security, and TiUP permissions, which may pose challenges.
- Enhance official documentation
Most of the official documentation is detailed, but some content is still lacking. For example, in monitoring, there are no detailed explanations of the various metrics on the panels, which would help in problem diagnosis.
- Provide more granular metrics
For example, in SQL execution plans, after running explain analyze, TiDB can show specific information for each operator, such as actRowS, filter predicates, cop task time, memory consumption, etc. In most cases, it’s easy to identify issues, but sometimes SQL execution takes a long time, and the execution plan or dashboard doesn’t show any problems. Monitoring with a 30-minute granularity also doesn’t reveal issues, making it hard to quickly and clearly locate problems. If the execution process could be further refined by stages, similar to Oracle’s wait events, it would help quickly analyze issues and lower the entry barrier for many people.
[Main Attraction]
It’s her, it’s her!
|
username: 
|
username: 
|
username: 
|
username: 
Does the Grafana+Prometheus official website tell you what the TiDB metrics on the dashboard mean?
|
username: 
|
username: 
|
username: 
|
username: