Can TiKV be labeled when starting TiKV using k8s, and how to configure it specifically?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 请问用k8s的方式启动tikv时可以给tikv打标签吗，具体如何配置？

| username: GreenGuan

Background: There are multiple fast disks on a host. To avoid the majority of replicas failing (default 3 replicas) due to k8s scaling up/down on the same physical machine, the anti-affinity in k8s is not very suitable for scenarios with multiple namespaces in the same k8s cluster. Therefore, the idea of labeling tikv was considered. However, when deploying with tiup, the host (which is the pod name for k8s) must be specified. So, there are two questions:

  1. Can the pod name be customized?
  2. How to assign labels to the pod?
2
| username: 数据小黑 | Original post link

Just to confirm, did you deploy using tiup on k8s? Have you tried TiDB Operator? Refer to: TiDB on Kubernetes 用户文档 | PingCAP 文档中心, starting from 1.1, it supports configuring nodeSelector, affinity, and tolerations for the tidb-operator chart #855

3
| username: GreenGuan | Original post link

The cluster deployed using TiDB Operator can use the nodeSelector parameter to solve the issue of scaling within the same namespace to different hosts. However, whether it can solve the issue of reusing the same set of k8s clusters across different namespaces is uncertain. For example:

There are 12 hosts, numbered h1-12, one k8s cluster, and two namespaces corresponding to business A and business B. If affinity is configured, business A corresponds to h1-6, and when it scales, it will expand to h7.

Due to the configured affinity, it is uncertain whether business B will continue to use the host resources of h1-7.

4
| username: 数据小黑 | Original post link

My understanding is that the labels set for affinity have nothing to do with the namespace. You can try testing your idea.

5
| username: GreenGuan | Original post link

Yes, there is one more point that I have been unable to figure out.
Method one: Tagging TiKV is based on the storage metadata of PD and KV nodes to schedule regions.
Method two: The k8s method is to linearly scale through affinity.

The main difference between method one and method two is whether two stores can be placed on the same host simultaneously.
Method one: Yes.
Method two: No, it can only scale linearly.

6
| username: wuxiangdong | Original post link
  1. Once a pod name is created, it cannot be modified.
  2. Pod labels can be modified using kubectl labels -n tidb-admin pod xxx xx=xx.
7
| username: yiduoyunQ | Original post link
  1. Pod names are usually in the format {cluster-name}-{type}-{ordinal}, such as basic-pd-0; if deployed across k8s, it will additionally include clusterDomain. You can refer to the relevant code here.
  2. “How to specify labels for pods” requires clarifying the requirements first.
    2.1 TiDB Service High Availability for k8s scheduling, nodeSelector will be deprecated in future k8s versions, and can be directly replaced with affinity.
    2.2 Data High Availability is the scheduling of data at the TiDB layer, which involves labeling the TiKV store. For additional information not covered in the documentation, you can use storeLabels.