Issue with k8s Deployment Operator Lacking ClusterRole

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: k8s 部署operator 没有clusterrole问题

| username: h5n1

Due to management requirements, only the highest permissions for the namespace can be granted, not clusterrole permissions. When installing with Helm, the settings are as follows:

clusterScoped: false
rbac:
  create: false
timezone: Asia/Shanghai
operatorImage: 10.17/zongbu-sre/pingcap/tidb-operator-arm64:v1.4.0
imagePullPolicy: IfNotPresent
tidbBackupManagerImage: 10.17/zongbu-sre/pingcap/tidb-backup-manager-arm64:v1.4.0
features: []
appendReleaseSuffix: false
controllerManager:
  create: true
  serviceAccount: tidb-controller-manager
  clusterPermissions:
    nodes: true
    persistentvolumes: true
    storageclasses: true
  logLevel: 2
  replicas: 1
  resources:
    requests:
      cpu: 500m
      memory: 500Mi
  autoFailover: true
  pdFailoverPeriod: 5m
  tikvFailoverPeriod: 5m
  tidbFailoverPeriod: 5m
  tiflashFailoverPeriod: 5m
  dmMasterFailoverPeriod: 5m
  dmWorkerFailoverPeriod: 5m
  affinity: {}
  nodeSelector: {}
  tolerations: []
  selector: []
  env: []
  securityContext: {}
  podAnnotations: {}
scheduler:
  create: true
  serviceAccount: tidb-scheduler
  logLevel: 2
  replicas: 1
  schedulerName: tidb-scheduler
  resources:
    limits:
      cpu: 500m
      memory: 500Mi
    requests:
      cpu: 500m
      memory: 500Mi
  kubeSchedulerImageName: 10.17/zongbu-sre/dyrnq/kube-scheduler-arm64-1:v1.24.9 
  affinity: {}
  nodeSelector: {}
  tolerations: []
  securityContext: {}
  podAnnotations: {}
  configmapAnnotations: {}

Set clusterScoped: false and rbac.create: false. There were no errors during deployment, but there are no Pods.

$ helm install test-tidb-operator ./tidb-operator --namespace=tidb-xktkj
NAME: test-tidb-operator
LAST DEPLOYED: Tue Feb 21 14:24:10 2023
NAMESPACE: tidb-xktkj
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Make sure tidb-operator components are running:

    kubectl get pods --namespace tidb-xktkj -l app.kubernetes.io/instance=test-tidb-operator

$ kubectl get pods --namespace tidb-xktkj 
No resources found in tidb-xktkj namespace.

$ kubectl describe rs  tidb-controller-manager-54dfd9d6d5 -n tidb-xktkj
 Error creating: pods "tidb-controller-manager-54dfd9d6d5-" is forbidden: error looking up service account tidb-xktkj/tidb-controller-manager: serviceaccount "tidb-controller-manager" not found
| username: yiduoyunQ | Original post link

Refer to the documentation at TiDB Operator RBAC 规则 | PingCAP 文档中心

| username: h5n1 | Original post link

$ kubectl get clusterrole | grep tidb
Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:kube-amp:xx-test-common-work-cluster-kube-amp-386935687479365" cannot list resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope
| username: yiduoyunQ | Original post link

The image you provided is not visible. Please provide the text you need translated.

| username: h5n1 | Original post link

Can you explain a bit more? Currently, my users only have role permissions and namespace permissions, without clusterrole. Below this line, there is also a section.

| username: yiduoyunQ | Original post link

The document lists the permissions required for non-namespaced k8s resources by the operator (tidb-controller-manager), corresponding to the clusterrole.

| username: h5n1 | Original post link

In other words, in my current situation, it won’t run without clusterrole? Can read-only permissions meet the requirements?

| username: yiduoyunQ | Original post link

The “explanation” in the image above shows the operations that will be executed in the operator sync logic. It’s not very clear how the program will behave when there is no permission.

| username: h5n1 | Original post link

Then it’s unsolvable. TiDB requires quite high permissions, and so does tiup.