BR Backup Error: Cannot Pass GC Safe Point Check

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: BR备份报错cannot pass gc safe point check

| username: TiDBer_yangxi

[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration]
[Attachments: Screenshots / Logs / Monitoring]

10:30 Set SET GLOBAL tidb_gc_enable=FALSE;
backupts ‘2023-09-13 10:35:00’,
13:37:58 (52.8% backup completed) error reported [“cannot pass gc safe point check, aborting”] [error=“GC safepoint 444222059096834048 exceed TS 444222013440000000: [BR:Backup:ErrBackupGCSafepointExceeded]backup GC safepoint exceeded”] [errorVerbose=“[BR:Backup:ErrBackupGCSafepointExceeded]backup GC safepoint exceeded\nGC safepoint 444222059096834048 exceed TS 444222013440000000\ngithub.com/pingcap/tidb/br/pkg/utils.CheckGCSafePoint\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/utils/safe_point.go:76\ngithub.com/pingcap/tidb/br/pkg/utils.StartServiceSafePointKeeper.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/utils/safe_point.go:133\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1571”] [safePoint=“{ID=br-92f713cb-58d2-4a1f-9c2d-208e296bfbdf,TTL=5m0s,BackupTime="2023-09-13 10:35:00 +0800 CST",BackupTS=444222013440000000}”] [stack=“github.com/pingcap/tidb/br/pkg/utils.StartServiceSafePointKeeper.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/utils/safe_point.go:134”]

| username: TiDBer_yangxi | Original post link

I don’t quite understand this GC check. After setting it, how long is the delay? Or it would be fine to check it once at startup. Reporting this error halfway through is very unreasonable.

| username: 大飞哥online | Original post link

Backup statement, take a look.

| username: 大飞哥online | Original post link

Take a look.

| username: cassblanca | Original post link

Starting from TiDB v4.0.8, BR backup already supports adaptive GC, so no adjustments are needed. Check the Grafana monitoring issue to see if it is caused by other problems.

| username: 大飞哥online | Original post link

The error says GC safepoint 444222059096834048 exceeds 444222013440000000.

| username: TiDBer_vfJBUcxl | Original post link

Other programs experiencing OOM can cause the BR backup tool to be killed, and the logs will report communication interruptions between TiKV and BR.

| username: TiDBer_yangxi | Original post link

I have already turned off the GC first.

| username: TiDBer_yangxi | Original post link

tiup br backup full --pd “172.18.xx.xx:2379” --backupts ‘2023-09-13 10:35:00’ --storage “s3://cairui-bigdata-temp?access-key=cairui-bigdata-temp&secret-access-key=xxx&endpoint=http://xxx:8333&force-path-style=true” --ratelimit 128

| username: 像风一样的男子 | Original post link

Why do you need to specify backupts? If not specified, it defaults to the current time for the backup.

| username: TiDBer_yangxi | Original post link

Specify the time to facilitate the subsequent CDC start time.

| username: 像风一样的男子 | Original post link

No need, the logs after the backup will also have ts.

| username: TiDBer_yangxi | Original post link

I haven’t used it, but specifying it might make you feel more at ease.

| username: TiDBer_yangxi | Original post link

GC is turned off, why does the safepoint keep changing?

| username: TiDBer_yangxi | Original post link

Version 7.1.1 does not have this issue.

| username: TiDBer_yangxi | Original post link

The safepoint issue has been moved to tikv gc safepoint 异常 - TiDB 的问答社区

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.