Failed to access Alibaba Cloud OSS via S3 after TiDB 6.4, same configuration works successfully in 6.1 and 5.1

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiDB 6.4后通过S3访问阿里云OSS失败，同样配置在6.1, 5.1都可上传成功

| username: dba-kit 
--storage s3://test-bucket/tidb-backup-20221201?access-key=********&secret-access-key=******** --s3.provider alibaba --s3.endpoint https://oss-cn-shenzhen-internal.aliyuncs.com

Previously, setting it this way allowed using OSS for temporary storage in tools like BR, DM, dumping, and lightning without expanding local disk space. However, after switching to 6.4, the same parameters keep causing errors during reporting. The error message is:

Forbidden: Forbidden
	status code: 403, request id: 6389988AAE2EFC36368E99E8, host id:
failed to get region of bucket mysql-data-archive
github.com/pingcap/tidb/br/pkg/storage.newS3Storage
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/storage/s3.go:368
github.com/pingcap/tidb/br/pkg/storage.New
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/storage/storage.go:176
github.com/pingcap/tidb/br/pkg/task.GetStorage
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/task/common.go:631
github.com/pingcap/tidb/br/pkg/task.ReadBackupMeta
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/task/common.go:651
github.com/pingcap/tidb/br/pkg/task.RunRestore
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/task/restore.go:524
main.runRestoreCommand
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/restore.go:58
main.newFullRestoreCommand.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/restore.go:143
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:872
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:990
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:918
main.main
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/main.go:57
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594
create storage failed] [stack=github.com/pingcap/log.Error
	/go/pkg/mod/github.com/pingcap/log@v1.1.1-0.20221015072633-39906604fb81/global.go:46
main.runRestoreCommand
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/restore.go:59
main.newFullRestoreCommand.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/restore.go:143
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:872
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:990
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:918
main.main
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/main.go:57
runtime.main
	/usr/local/go/src/runtime/proc.go:250
2
| username: dba-kit | Original post link

It looks like s3manager.GetBucketRegionWithClient will set credential anonymous is the key point. I checked the 6.1 version and it does not use s3manager to get the region.

3
| username: dba-kit | Original post link

Oh, no one has replied yet…
I found the merge information: br: get bucket region from s3 to enable users not to input s3 region param by WangLe1321 · Pull Request #34419 · pingcap/tidb · GitHub. It seems that this is just to reduce the input of a region parameter. If this causes incompatibility with Alibaba Cloud’s OSS, it doesn’t seem worth it. Could the official team consider rolling it back?

4
| username: Lucien-卢西恩 | Original post link

Please wait a moment, we will confirm at the product level and keep you updated on the progress.

5
| username: TiDBer_V1su6yVQ | Original post link

This issue arises because in version 6.2 we introduced the automatic retrieval of the S3 bucket region feature. The AWS SDK sets S3ForcePathStyle to true by default in the implementation of this feature, which causes the generated URL to be in path style when requesting the Alibaba Cloud OSS interface, leading to request failures. A PR has already been submitted and will be fixed in version 6.5.

6
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.