How to Modify the Default PLACEMENT POLICY in TiDB?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiDB如何修改默认的PLACEMENT POLICY?

| username: dba-kit

TiDB Version: 6.1.1
Expectation:
Added three SATA disk TiKV nodes, hoping not to trigger normal table region scheduling, and only place a few historical partitions on them. By default, TiDB writes data as evenly as possible, and new regions will also be written to the newly added SATA disk machines (since they are new nodes, normal rebalance will also be triggered). However, I actually do not want normal data to be written to these machines because the read and write performance of SATA disks is very poor.

Problem:
The current idea is to create two PLACEMENT POLICY, let the normal tables use the on_ssd policy, and cold data use the on_sata policy. The policies are as follows:

CREATE PLACEMENT POLICY on_ssd CONSTRAINTS="[-disk=sata]";
CREATE PLACEMENT POLICY on_sata CONSTRAINTS="[+disk=sata]";

However, according to the official documentation on placement-rules-in-sql, you can only create PLACEMENT POLICY and then set it for specific databases/tables/partitions. There is also a note:

Note that the inheritance between partitions and tables is different from the inheritance here. Changing the placement policy of a table will also apply the new policy to its partitions. However, only when no placement policy is specified during table creation will the table inherit the placement policy from the database, and subsequent changes to the database will not affect the already inherited tables.

This means that I need to manually execute ALTER TABLE t2 PLACEMENT POLICY=on_ssd; for each existing database and table; if a new database is added later, it also needs to explicitly specify the on_ssd PLACEMENT POLICY.

Requirement:
Is there any way to modify the default PLACEMENT POLICY (i.e., the policy for tables when no PLACEMENT POLICY is explicitly specified)? Or set TiKV to not participate in normal data write scheduling by default, but allow tables with explicitly specified PLACEMENT POLICY to write data to these nodes?

2
| username: 数据小黑 | Original post link

Here is my understanding:

  1. It is possible to set a database-level default placement policy. Newly created tables will inherit the database’s placement policy by default unless specified otherwise.
  2. For existing tables, the placement policy needs to be manually specified.

The above default placement policy is specified at the database level, and there is no global policy. The workload of configuring default placement rules at the database level on a TiDB instance is acceptable.

3
| username: dba-kit | Original post link

This is also a pitfall. I have already written a simple script to set all tables to on_ssd. However, if you forget to add the on_ssd rule when creating a new database, it is still possible to place the data on SATA disk machines.

4
| username: 数据小黑 | Original post link

This is a management and technical issue. If there are difficulties in management, it is indeed necessary to make some technical supplements, such as having a default value or something.

5
| username: dba-kit | Original post link

The temporary solution can only be like this. However, it is still recommended that the official add a parameter to support adjustments.

6
| username: dba-kit | Original post link

Although it is not possible to change the default strategy through Placement Rules in SQL, it can be changed through pd-ctl. Here, by adding the following rule to the default rule, it is tested that the default data (without any PLACEMENT POLICY) will not be written to the TiKV nodes with the label disk=sata.

      "label_constraints": [
      {
        "key": "disk",
        "op": "notIn",
        "values": [
          "sata"
        ]
      }
      ]

The complete configuration and operation steps are:

[tidb@prod-tiup-f01 ~]$ cat default-rule.json 
{
  "group_id": "pd",
  "group_index": 0,
  "group_override": false,
  "rules": [
    {
      "group_id": "pd",
      "id": "default",
      "start_key": "",
      "end_key": "",
      "role": "voter",
      "count": 3,
      "label_constraints": [
      {
        "key": "disk",
        "op": "notIn",
        "values": [
          "sata"
        ]
      }
      ],
      "location_labels": [
        "zone",
        "dc",
        "host"
      ]
    }
  ]
}
[tidb@prod-tiup-f01 ~]$ tiup ctl:v6.1.1 pd -u ****:2379 -i
Starting component `ctl`: /home/tidb/.tiup/components/ctl/v6.1.1/ctl pd -u ****:2379 -i
» config placement-rules rule-bundle set pd --in="default-rule.json"
7
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.