Adding Learner Replica via PLACEMENT POLICY Not Effective

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 通过PLACEMENT POLICY增加Learner副本不生效

| username: dba-kit

According to the placement-rules-in-sql#advanced-placement-options documentation, version 7.5 supports setting the number of Learner replicas. However, when performing the following operations, it was found that the scheduling remains in the INPROGRESS state.

CREATE PLACEMENT POLICY add_oltp_leaner CONSTRAINTS="[-disk=sata]" LEARNERS=1 ;
ALTER TABLE test.t_partition PARTITION p202304  PLACEMENT POLICY add_oltp_leaner;
SHOW PLACEMENT where Target like '%test.t_partition%';

| username: zhaokede | Original post link

Is there a log?

| username: dba-kit | Original post link

Strangely, there is only one log modifying the Placement Rule in PD, but executing operator show reveals nothing. However, it remains in the INPROGRESS state.

| username: shigp_TIDBER | Original post link

Is it because of network or system pressure that it hasn’t succeeded yet? Let’s observe for a while.

| username: dba-kit | Original post link

The modifications were made yesterday afternoon, and it’s been over 12 hours now. Additionally, this table is a test table with a total data volume of only a few hundred thousand. The main issue is that the Operator hasn’t been generated, and I’m not sure which step it’s stuck at.

| username: TiDBer_JUi6UvZm | Original post link

Marking it. The place rules and label confused me, still learning.

| username: 数据库真NB | Original post link

It didn’t take effect, it was just set.

| username: 有猫万事足 | Original post link

I feel like this is a bug.

| username: 小龙虾爱大龙虾 | Original post link

What you mean is that no roles should be placed on SATA, and there should be one learner. Also, remember to specify the survival preference.

| username: dba-kit | Original post link

Yes, the actual requirement is like this, but during testing, it was found that something might be set incorrectly, and it has not taken effect.

| username: 小龙虾爱大龙虾 | Original post link

What is your cluster topology like? Can you share the diagram from the dashboard?

| username: dba-kit | Original post link

There are a total of three DCs. Here is a screenshot of the topology of one DC. The other two DCs are similar. Each DC is divided into three types of Labels, but it doesn’t seem to have much to do with this issue, since the -disk option is used to exclude a certain type of Label.

| username: nolouch-PingCAP | Original post link

Please provide the following information:

  1. The placement rules corresponding to the Learner in PD.
  2. The region data corresponding to show table test.t_partition regions, preferably with the corresponding partition.
  3. The monitoring data of the PD rule checker.
| username: oceanzhang | Original post link

It is probably an issue with the table’s load, which is why it hasn’t succeeded?

| username: dba-kit | Original post link

  1. The corresponding Rule in PD is:
{
  "group_id": "TiDB_DDL_45199",
  "group_index": 80,
  "group_override": true,
  "rules": [
    {
      "group_id": "TiDB_DDL_45199",
      "id": "partition_rule_45199_0",
      "index": 40,
      "start_key": "74800000000000b0ff8f00000000000000f8",
      "end_key": "74800000000000b0ff9000000000000000f8",
      "role": "voter",
      "is_witness": false,
      "count": 3,
      "label_constraints": [
        {
          "key": "disk",
          "op": "notIn",
          "values": [
            "sata"
          ]
        },
        {
          "key": "engine",
          "op": "notIn",
          "values": [
            "tiflash"
          ]
        }
      ],
      "create_timestamp": 1713431877
    },
    {
      "group_id": "TiDB_DDL_45199",
      "id": "partition_rule_45199_1",
      "index": 40,
      "start_key": "74800000000000b0ff8f00000000000000f8",
      "end_key": "74800000000000b0ff9000000000000000f8",
      "role": "learner",
      "is_witness": false,
      "count": 1,
      "label_constraints": [
        {
          "key": "disk",
          "op": "notIn",
          "values": [
            "sata"
          ]
        },
        {
          "key": "engine",
          "op": "notIn",
          "values": [
            "tiflash"
          ]
        }
      ],
      "create_timestamp": 1713431877
    }
  ]
}
  1. Running show table test.t_partion regions shows that some regions are indeed in the PENDING state:
*************************** 63. row ***************************
             REGION_ID: 15631951
             START_KEY: t_45197_
               END_KEY: t_45198_
             LEADER_ID: 15631954
       LEADER_STORE_ID: 1
                 PEERS: 15631952, 15631953, 15631954
            SCATTERING: 0
         WRITTEN_BYTES: 39
            READ_BYTES: 0
  APPROXIMATE_SIZE(MB): 1
      APPROXIMATE_KEYS: 0
SCHEDULING_CONSTRAINTS: CONSTRAINTS="[-disk=sata]"
      SCHEDULING_STATE: SCHEDULED
*************************** 64. row ***************************
             REGION_ID: 15631955
             START_KEY: t_45198_
               END_KEY: t_45199_
             LEADER_ID: 15631956
       LEADER_STORE_ID: 4
                 PEERS: 15631956, 15631957, 15631958
            SCATTERING: 0
         WRITTEN_BYTES: 0
            READ_BYTES: 0
  APPROXIMATE_SIZE(MB): 1
      APPROXIMATE_KEYS: 0
SCHEDULING_CONSTRAINTS: CONSTRAINTS="[-disk=sata]"
      SCHEDULING_STATE: SCHEDULED
*************************** 65. row ***************************
             REGION_ID: 15635914
             START_KEY: t_45199_i_1_0419afd60000000000013132343100000000fb013100000000000000f80380000000002771cf013034383132363030ff0000000000000000f7014341425534493449ff4c324d4e34430000fd01434c303030303531ff3538000000000000f9013031323034310000fd
               END_KEY: t_45199_r_72057594042656696
             LEADER_ID: 15635916
       LEADER_STORE_ID: 2
                 PEERS: 15635915, 15635916, 15635917
            SCATTERING: 0
         WRITTEN_BYTES: 0
            READ_BYTES: 0
  APPROXIMATE_SIZE(MB): 44
      APPROXIMATE_KEYS: 273502
SCHEDULING_CONSTRAINTS: CONSTRAINTS="[-disk=sata]" LEARNERS=1
      SCHEDULING_STATE: PENDING
*************************** 66. row ***************************
             REGION_ID: 15635894
             START_KEY: t_45199_r_72057594042656696
               END_KEY: t_45199_r_648518346345467670
             LEADER_ID: 15635895
       LEADER_STORE_ID: 4
                 PEERS: 15635895, 15635896, 15635897
            SCATTERING: 0
         WRITTEN_BYTES: 0
            READ_BYTES: 0
  APPROXIMATE_SIZE(MB): 101
      APPROXIMATE_KEYS: 669709
SCHEDULING_CONSTRAINTS: CONSTRAINTS="[-disk=sata]" LEARNERS=1
      SCHEDULING_STATE: PENDING
| username: dba-kit | Original post link

However, I couldn’t find this monitoring. Could you provide details on which Grafana dashboard it is on?

| username: dba-kit | Original post link


After a while, the state of this region changed from pending to INPROGRESS.