No new data is written after expanding TiKV

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 扩容 tikv 后没有新数据写入

| username: rebelsre

[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version]
[Reproduction Path] Refer to the official documentation to use tiup to expand the cluster
[Encountered Problem: Problem Phenomenon and Impact]
The new tikv node has no data written, and the leader blocks have not migrated
[Resource Configuration]
[Attachments: Screenshots / Logs / Monitoring]



| username: 像风一样的男子 | Original post link

Are there schedule tasks in the operator section of PD monitoring?

| username: caiyfc | Original post link

Did the original cluster’s TiKV have labels set? Were labels set during the expansion?

| username: rebelsre | Original post link

This is available when operating offline nodes, but this also synchronizes data to the old node, and the new node still doesn’t receive any data.

| username: rebelsre | Original post link

I don’t recall this part, or how can I check it?

| username: caiyfc | Original post link

Use tiup cluster edit-config <cluster-name> to check the config of pd in servers_config and the config of each tikv node.

| username: h5n1 | Original post link

You can view the label in information_schema.tikv_store_status. Additionally, check the weight of TiKV and see if there is an evict scheduler: pd-ctl scheduler show.

| username: 像风一样的男子 | Original post link

Check the store limit under pd-ctl to see how large it is.

| username: rebelsre | Original post link

I noticed there are some label-related configurations. Should I remove these configurations?

| username: caiyfc | Original post link

For mixed deployment of TiKV, you need to set labels to avoid having the same replica on the same machine. I see that you have set the label to “host” here, so you need to set labels for other TiKV instances that don’t have them yet. If you can restart TiKV, you can set the labels in the edit-config and then restart the newly added TiKV nodes. If you cannot restart, after adding the labels in edit-config, use pd-ctl to set the labels for the corresponding store.

| username: tidb菜鸟一只 | Original post link

The label you applied to the old nodes is directly specified by the host, which should be for a mixed environment. So, apply the label to the new nodes as well, still according to the host, because I see the new nodes should also be in a mixed environment.

| username: rebelsre | Original post link

Okay, I’ll try adding a label.

| username: rebelsre | Original post link

I am taking a TiKV node offline. Will there be any issues if I restart, or can I just restart the TiKV of the new nodes (those that were not configured with labels before)?

| username: caiyfc | Original post link

Just restart the new node.

| username: zhanggame1 | Original post link

TiDB has label and placement rule restrictions on data distribution. Check if they were configured originally and whether the configuration is reasonable.

| username: rebelsre | Original post link

After adding the label and restarting TiKV, I saw that data was being written to the new node. Thank you, everyone.

| username: caiyfc | Original post link

After solving it, you can select the best answer to help other students solve the problem more effectively.

| username: redgame | Original post link

Adding a label or restarting TiKV, which one is effective?

| username: caiyfc | Original post link

Restarting is just to make the added label take effect.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.