Can the rules file of Prometheus in an existing TiDB cluster be modified?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 已存在的tidb集群是否可以修改集群的prometheus的rules文件规则

| username: Csong

Is it possible to modify the rules file of Prometheus in an existing TiDB cluster?
I checked the documentation: rule_dir: This field specifies a local directory that should contain complete *.rules.yml files. These files will be transferred to the target machine during the cluster configuration initialization phase as Prometheus rules. Testing also confirms that it cannot be modified.
Does this mean that only new clusters can modify the rules, and existing clusters cannot?

2
| username: Csong | Original post link

Or how to discard TiDB’s own default rules.

3
| username: Csong | Original post link

Found the answer:
cmds := string{
“mkdir -p %[1]s/conf”,
find %[1]s/conf -type f -name "*.rules.yml" -delete,
find %[1]s/bin/prometheus -maxdepth 1 -type f -name "*.rules.yml" -exec cp {} %[1]s/conf/ \;,
find %[1]s/conf -maxdepth 1 -type f -name "*.rules.yml" -exec sed -i -e 's/ENV_LABELS_ENV/%[2]s/g' {} \;,
}

After looking at the code, as long as the rules files in the cluster’s bin/prometheus directory are copied, and the rule_dir in the monitor-servers section is added in edit-config, you can use your own rules.

Additionally, you should do it again after upgrading the version.

4
| username: Csong | Original post link

tiup/pkg/cluster/spec/monitoring.go

5
| username: alfred | Original post link

It should support custom rules.

6
| username: Csong | Original post link

Yes, I mainly want to discard the built-in rules and use my own. I happened to see the copy logic of his rules in the code, so I can bypass it.

7
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.