The number of replicas configured for dm-worker in the DMCluster deployed on Kubernetes is different from the actual number generated

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: kubernets 部署dm集群DMCluster中dm-worker配置副本数和实际生成不一样

| username: love-cat

[TiDB Usage Environment] Production Environment
[TiDB Version] 5.2.2
[Reproduction Path]
[Encountered Problem: Problem Phenomenon and Impact]
Deploying the DM cluster through Kubernetes, the DMCluster is configured with 29 replicas, but actually produces 32 replicas, as shown below:

image

[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

| username: Billmay表妹 | Original post link

The number of dm-worker replicas in the DM cluster is controlled by StatefulSet, not by DMCluster. Therefore, if you configured 29 dm-worker replicas in DMCluster but actually generated 32 replicas, it might be due to incorrect StatefulSet configuration.

You can check and resolve this issue by following these steps:

  1. Check if the StatefulSet configuration is correct. You can get the detailed information of the StatefulSet using the following command:
kubectl describe statefulset <statefulset-name> -n <namespace>

You need to check if the following fields are set correctly:

  • spec.replicas: It should be the same as the number of dm-worker replicas in DMCluster.
  • spec.volumeClaimTemplates: It should be the same as the dm-worker PVC template in DMCluster.
  1. Check if there are other dm-worker replicas. You can get detailed information of all dm-worker replicas using the following command:
kubectl get pods -l app.kubernetes.io/component=dm-worker -n <namespace>

If there are other dm-worker replicas, you can delete them using the following command:

kubectl delete pod <pod-name> -n <namespace>

Please note that this will stop the running dm-worker replicas and restart a new replica.

| username: love-cat | Original post link

Okay, thanks, I’ll give it a try.

| username: love-cat | Original post link

I have 32 statefulsets, but after I modified it to 29 through kubectl edit statefulset dm1-dm-worker -n <namespace>, it automatically changes back to 32.

| username: love-cat | Original post link

This is a DMCluster configuration: