Backup Error: S3 Bucket and Region Do Not Match

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: br 备份报错s3 bucket and region are not matched

| username: TiDBer_ZsnVPQB4

[TiDB Usage Environment] Production Environment
[TiDB Version] v6.5.0
[Reproduction Path] br backup full
[Encountered Problem: Phenomenon and Impact] Backup failed with error message: s3 bucket and region are not matched

[2023/05/22 11:06:57.761 +08:00] [INFO] [collector.go:73] [“Full Backup failed summary”] [total-ranges=0] [ranges-succeed=0] [ranges-failed=0]
Error: s3 bucket and region are not matched, bucket=db-back, input region=hazz, real region=us-east-1
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]
Previously, with br v5.2.1 version and tidb v5.2.1 version, there were no issues with backup. Now, with br updated to v6.5.0 and v7.0.0 versions, this error occurs.
The keys and other information are written in the script and there are no issues.

br backup full --send-credentials-to-tikv=true --pd 10.30.30.3:2379 --storage s3://db-back/tidb-back/20230522 --s3.endpoint http://oss-hazz.aliyun.cn --s3.region hazz --ratelimit 32 --log-file 20230522_backupfull.log

Why does the error message indicate real region=us-east-1? Is this the eastern United States?

| username: Billmay表妹 | Original post link

This error message is because the region of the specified S3 bucket does not match the region you entered. Please check your input to ensure that the region you entered matches the region where your S3 bucket is located.

If you are using AWS’s official S3, you need to specify the region and do not need to specify s3.endpoint. If you are using an object storage service compatible with the S3 protocol, you need to specify both s3.endpoint and region.

You can specify the region of the S3 bucket with the following command:

./br backup full --pd "pd_ip:pd_port" --storage "s3://bucket_name/dir" --s3.region "us-east-1"

Here, the --s3.region parameter specifies the region where the S3 bucket is located, and you need to replace it with the region where your S3 bucket is located.

| username: TiDBer_ZsnVPQB4 | Original post link

The backup command used Tianyi Cloud, and the command already specified --s3.endpoint and --s3.region. It worked before, but after upgrading the BR version, it doesn’t work anymore. The error message indicates a real region=us-east-1, which I have no idea where it came from. Shouldn’t it use the parameters I specified? How did this real region come up? I’m completely lost.

| username: TiDBer_ZsnVPQB4 | Original post link

With the same parameters, using br:v5.2.1 will prompt a version mismatch. Adding --check-requirements=false allows the backup to run.

| username: TiDBer_ZsnVPQB4 | Original post link

Similar to this post: br备份数据到ks3存储报错 - TiDB 的问答社区. Tried br v6.5.4, v6.5.0, v7.0.0, none of them worked. It must be a tool issue. Adding the --check-requirements parameter to br v5.2.1 allows for backup.

| username: Fly-bird | Original post link

Is the S3 configuration correct?

| username: cassblanca | Original post link

The name of the region where S3 is located is incorrect, right?

| username: TiDBer_ZsnVPQB4 | Original post link

Using the lower version v5.2.1 works, but higher versions v6.5.0, v6.5.4, v7.0.0, v7.3.0 br do not work. The bucket and region parameters have been adjusted many times, but errors keep occurring. The S3 configuration should be correct since the lower versions work, and we have also communicated with Tianyi Cloud.

| username: TiDBer_ZsnVPQB4 | Original post link

This tool refers to

Has this been fixed in version 6.5? I feel like this is the reason. If I don’t fill in s3.region, it reports this error:

Error: failed to get region of bucket db-back: BadRequest: Bad Request
status code: 400, request id: 0bccf13a128a4bc371d2dbecdfe8eaf0afb1b6a5a7a9abadaf, host id:

Sigh, just to save one configurable parameter…

| username: TiDBer_ZsnVPQB4 | Original post link

This post

| username: dba-kit | Original post link

It seems like there is an issue with your configuration. These three parameters should be sufficient for Alibaba Cloud:

--storage="s3://mysql-dts-migrate/***?access-key=*********&secret-access-key=*****" \
--s3.endpoint="https://oss-cn-shenzhen-internal.aliyuncs.com"  \
--s3.provider "alibaba"
| username: TiDBer_ZsnVPQB4 | Original post link

If you don’t specify s3.region, it will try to find it on its own.
Error: failed to get region of bucket db-back: BadRequest: Bad Request
Even if you specify it, it will still check and compare with your specified value. If it doesn’t match, it will still report an error.

I’m using Tianyi Cloud, and I’m not sure how it obtains the real region. It’s us-east-1. Even if I specify it, it doesn’t use it and insists on fetching this value on its own. The fetched value seems incorrect or unsupported.

| username: dba-kit | Original post link

I haven’t used Tianyi Cloud, but you can try adding the parameter &force-path-style=false to see if it still reports an error.

| username: dba-kit | Original post link

  • force-path-style: Use path-style instead of virtual-hosted-style, default is true

Due to historical reasons, S3 storage has two path styles. The default is path-style mode, but other vendors’ S3 storage may not support it. You can set it to false and try again.

| username: dba-kit | Original post link

I see you wrote aliyun.cn here? Isn’t this the endpoint for Tianyi Cloud??

| username: dba-kit | Original post link

This has been fixed in version 6.5. I’m currently using BR version 6.5.3 for backups, and it’s working fine.

| username: TiDBer_ZsnVPQB4 | Original post link

I modified it, it is actually Tianyi Cloud’s.

| username: TiDBer_ZsnVPQB4 | Original post link

Adding &force-path-style=false still results in an error. As long as --s3.region is not added, it will try to get the region by itself. The request doesn’t show what it is trying to concatenate. The request to query the region seems to be unsupported by Tianyi Cloud.
Error: failed to get region of bucket db-back: BadRequest: Bad Request

| username: TiDBer_ZsnVPQB4 | Original post link

BR version 6.5.5 can perform backups, but versions 6.5.1, 6.5.3, and 6.5.4 cannot.