How to Configure Backup CR in tidb-operator 1.5.x to Backup to Alibaba Cloud OSS?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb-operator 1.5.x 的Backup CR 该如何配置才能备份到阿里云OSS 呢？

| username: TiDBer_RywnG56h

[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version] 7.5.1
[Reproduction Path]
Configure Backup CR like this, POD reports an error: “Used STS type AccessKey ID, but did not include the SecurityToken field in the request to indicate the use of STS authentication method”

backupMode: snapshot
backupType: full
br:
  logLevel: debug
  cluster: basicai
  clusterNamespace: tidb-cluster
resources: {}
s3:
  bucket: bas....p
  endpoint: https://oss-cn-beijing.aliyuncs.com
  prefix: tidb_test/alidev
  provider: alibaba
  region: oss-cn-beijing
  secretName: s3-secret

secret: s3-secret exists, and the ak/sk inside is correct. Using this ak/sk, the BR command can complete the backup manually.

Refer to the BR command (specify ak/sk after the storage parameter value: --storage “s3://ba…p/tidb_test/alidev?access-key=LT…tN&secret-access-key=MD…K”), explicitly add ak/sk after the prefix field value to write the backup file to OSS:

  s3:
    bucket: ba...p
    endpoint: https://oss-cn-beijing.aliyuncs.com
    prefix: tidb_test/alidev?access-key=L...N&secret-access-key=M...JK
    provider: alibaba
    region: oss-cn-beijing

But then another problem arises. At the final stage of the backup, when Backup CR reads the backup metadata from OSS, it reports an error: error: read backup meta from bucket basicai-ops-backup and prefix tidb_test/alidev?access-key=LTA…N&secret-access-key=MD…K: backupmeta not exist.

How should this Backup CR be configured to correctly back up to Alibaba Cloud OSS?

2
| username: yiduoyunQ | Original post link

Taking v1.5.2 as an example, the code logic is as follows:

  1. Read the secret corresponding to secretName, tidb-operator/pkg/backup/util/util.go at master · pingcap/tidb-operator · GitHub
  2. After reading, add it to env, tidb-operator/pkg/backup/backup/backup_manager.go at v1.5.2 · pingcap/tidb-operator · GitHub
  3. Finally, set it uniformly to pod env, tidb-operator/pkg/backup/backup/backup_manager.go at v1.5.2 · pingcap/tidb-operator · GitHub

In simple terms, the key/value in the secret will be set to pod env. You can get pod -oyaml or describe to check the spec (BR supports configuring AK/SK in env to read and write S3).

POD error: “Using an STS type AccessKey ID, but did not include the SecurityToken field in the request to indicate the use of STS authentication”

You need to contact OSS technical support to confirm why a token is required.

However, in the final stage of the backup, when the Backup CR reads the backup metadata from OSS, it reports an error: error: read backup meta from bucket basicai-ops-backup and prefix tidb_test/alidev?access-key=LTA…N&secret-access-key=MD…K: backupmeta not exist

Same as above :sweat_smile:

3
| username: 友利奈绪 | Original post link

Waiting for the best response.

4
| username: TiDBer_RywnG56h | Original post link

Sigh, I also raised an issue on GitHub, but no one has paid attention to it yet: How to use Backup CR to backup tidb to aliyun OSS? · Issue #5611 · pingcap/tidb-operator · GitHub

5
| username: wangkk2024 | Original post link

Learned.

6
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.