Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.Original topic: 配置冷热隔离分区策略,新增lable disk=hhd 出现数据写入慢,cpu拉的很高
[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version]
6.5.2
Some unexpected issues occurred last night while configuring the hot and cold strategy:
- First step: set the SSD and HDD strategies, second step: set all historical data to the specified SSD strategy, for example, DATABASE estadver “CONSTRAINTS=”“[+disk=ssd]”“” SCHEDULED.
- Added 2 new KV instances with labels set to disk=hhd.
Phenomenon: After an hour, the CPU usage of the cluster increased significantly, real-time offline data writing became very slow, and some data started to migrate to the KV nodes labeled as hhd. After scaling down the newly added hhd, it returned to normal.
Process:
First step:
Configured strategy: CREATE PLACEMENT POLICY onssd CONSTRAINTS=“[+disk=ssd]”;
CREATE PLACEMENT POLICY onhdd CONSTRAINTS=“[+disk=hhd]”;
Second step: Set all historical databases to the SSD strategy to avoid data being written to hhd, DATABASE estadver “CONSTRAINTS=”“[+disk=ssd]”“” SCHEDULED.
Third step: Added and expanded KV nodes with the label set to hhd.
Impact: After observing for an hour, data was being written to the hhd disk (which should not have happened), the CPU usage of the machines labeled as hhd was very high, and real-time offline writing was very slow.
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachment: Screenshot/Logs/Monitoring]