BR Tool Backup Error: TiKV Storage I/O Error - I/O Permission Denied Error Occurs on TiKV Node

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: br工具备份报错 tikv storage occur I/O error: I/O permission denied error occurs on TiKV Node

| username: TiDBer_BwNZ5U9X

[TiDB Usage Environment] Testing
[TiDB Version] 6.3.0
Using the 6.3.0 ARM version database with a total of three TiKV nodes, performing a full backup to a shared directory using the BR tool with the command: br backup full --pd 172.18.55.20:12379 -s /data/myshare

Backup Error
Full Backup <…> 0.00% [2023/08/16 04:18:15.597 +08:00] [INFO] [collector.go:69] [“Full Backup failed summary”] [total-ranges=285] [ranges-succeed=0] [ranges-failed=285] [backup-total-ranges=285] [unit-name=“range start:7480000000000008135f720000000000000000 end:7480000000000008135f72ffffffffffffffff00”] [error=“rpc error: code = Canceled desc = context canceled”] [errorVerbose="rpc error: code = Canceled desc = context canceled\ngithub.com/tikv/pd/client.(*client).respForErr\n\t/root/go/pkg/mod/github.com/tikv/pd/client@v0.0.0-20220725055910-7187a7ab72db/client.go:1885\ngithub.com/tikv/pd/client.(*client).GetAllStores\n\t/root/go/pkg/mod/github.com/tikv/pd/client@v0.0.0-20220725055910-7187a7ab72db/client.go:1586\ngithub.com/pingcap/tidb/br/pkg/conn/util.GetAllTiKVStores\n\t/var/lib/docker/je…

br.log Error Information
[2023/08/21 04:18:18.418 +08:00] [ERROR] [push.go:201] [range-sn=1] [error=“[BR:KV:ErrKVStorage]tikv storage occur I/O error: I/O permission denied error occurs on TiKV Node(store id: 2; Address: 172.18.55.22:30160). work around: please ensure tikv has permission to read from & write to the storage.”] [stack=“github.com/pingcap/tidb/br/pkg/backup.(*pushDown).pushBackup\n\t/var/lib/docker/jenkins/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/backup/push.go:201\ngithub.com/pingcap/tidb/br/pkg/backup.(*Client).BackupRange\n\t/var/lib/docker/jenkins/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/backup/client.go:638\ngithub.com/pingcap/tidb/br/pkg/backup.(*Client).BackupRanges.func2\n\t/var/lib/docker/jenkins/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/backup/client.go:592\ngithub.com/pingcap/tidb/br/pkg/utils.(*WorkerPool).ApplyOnErrorGroup.func1\n\t/var/lib/docker/jenkins/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/utils/worker.go:76\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:75”]

[2023/08/21 04:18:18.419 +08:00] [ERROR] [client.go:1085] [“fail to backup”] [range-sn=2] [store-id=3] [StoreID=3] [retry=0] [stack=“github.com/pingcap/tidb/br/pkg/backup.SendBackup\n\t/var/lib/docker/jenkins/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/backup/client.go:1085\ngithub.com/pingcap/tidb/br/pkg/backup.(*pushDown).pushBackup.func2\n\t/var/lib/docker/jenkins/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/backup/push.go:92”]

Cluster Information

Seeking expert assistance to determine what went wrong!!!

| username: 像风一样的男子 | Original post link

Is the /data/myshare directory present on the server at 172.18.55.22:30160? Has it been given 777 permissions?

| username: cassblanca | Original post link

There is no permission to access the /data/myshare directory, so the backup file cannot be written completely. Please check if the directory exists and its permissions.

| username: tidb菜鸟一只 | Original post link

Try touching a file in the /data/myshare directory on all three TiKV nodes to test. It is likely a permissions issue.

| username: zhanggame1 | Original post link

The backup directory needs read and write permissions, at least give it 666.