Is there a script for automatic daily backups with br, and can you share one?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 有br每天自动备份的脚本吗,能不能分享一个

| username: zhanggame1

[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version] 7.5
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Symptoms and Impact]

Planning to perform a full BR backup of the entire database daily and periodically delete backups older than a certain period. Looking for similar scripts for reference.

| username: Jiawei | Original post link

Just a few core logics:
Backup, then scan the documents to determine whether the files need to be deleted.

| username: 像风一样的男子 | Original post link

Write your own shell script

######### Full database backup #########################################################

Backup TiDB database

date1=date +'%y%m%d'
mkdir -p /data/tidbback/$date1
chmod -R 777 /data/tidbback/$date1
cd /opt/tidb-toolkit-v5.0.0-linux-amd64/bin
./br backup full --pd “xxxxx:2379” -s local:///data/tidbback/$date1 --ratelimit 256 --log-file backupfull.log

Compress and delete original data

cd /data/tidbback
tar -zcvf tidb_$date1.tar.gz $date1
rm -rf $date1

Delete backups older than 30 days
find /data/tidbback/ -maxdepth 1 -mtime +29 -type f -name “*.tar.gz” | xargs rm -rf

Set up a cron job on the server to execute at 1 AM every day
0 1 * * * sh /data/scripts/backup.sh &

Write the specific requirements in your shell script

| username: zhanggame1 | Original post link

Thank you, I’ll give it a try. This looks good.

| username: oceanzhang | Original post link

Does this require writing a crontab for each node?

| username: 像风一样的男子 | Original post link

I have a separate backup server with a disk mounted as NFS to all KV nodes. The script only needs to be placed on this backup server. When BR performs the backup, it connects to the cluster through PD, and the backup files will be automatically stored on this mounted disk. There is no need to execute the script on each node.

| username: zhanggame1 | Original post link

You don’t need to write a crontab for each node; generally, you can use tiup to call br.

| username: oceanzhang | Original post link

Got it, thanks.

| username: 有猫万事足 | Original post link

Compared to daily full snapshot backups, I think having a full snapshot backup once a week and keeping incremental PITR log backups running continuously would be more convenient.

| username: zhanggame1 | Original post link

Tested it, NFS shared directory /nfs, backup directory /nfs/date, automatically create tar package and delete backup files.

Content of br.sh file:
/root/.tiup/components/br/v7.6.0/br backup full --pd 192.168.19.207:2379 --storage local:///nfs/$(date +%Y%m%d) --log-file /nfs/b$(date +%Y%m%d).log --concurrency=10
cd /nfs
tar cvf br_$(date +%Y%m%d).tar $(date +%Y%m%d)
rm -rf $(date +%Y%m%d)

Crontab scheduled task
51 22 * * * /root/br.sh

Testing with gz compression is meaningless, very slow, and the compression effect is minimal.

| username: 像风一样的男子 | Original post link

I perform backups once a week, and it’s a bit more convenient to upload the package to OSS after it’s done.

| username: wfxxh | Original post link

#
# auth: wfxxh
# desc: Periodically perform full backups of production TiDB data and delete expired full and incremental backups, executed every 3 days
# name: br-fullbackup.sh

TODAY=`date +%Y%m%d`

source /home/tidb/.bash_profile

echo > /home/tidb/br/br-fullstatus.log
echo > /home/tidb/br/log/br-date.log

# Perform full backup
tiup br:v6.5.3 backup full --pd "pd address" --storage "s3://wfmodel/tidb-br/tidb-online/fullbackup/${TODAY}?endpoint=s3 address&access-key=your key&secret-access-key=your secret" --ratelimit 128 --log-file /home/tidb/br/log/br-fullbackup-${TODAY}.log

echo $TODAY >> /home/tidb/br/stats/fullbackup

SIX_DAY_AGO=`tail -3 /home/tidb/br/stats/fullbackup | head -1`

# Get the timestamp of the full backup from 6 days ago
FULL_BACKUP_TS=`tiup br:v6.5.3 validate decode --field="end-version" --storage "s3://wfmodel/tidb-br/tidb-online/fullbackup/${SIX_DAY_AGO}?endpoint=s3 address&access-key=your key&secret-access-key=your secret" --log-file /home/tidb/br/log/br-date.log | tail -1`

# Check if the timestamp obtained in the previous step matches the one in the log
LOG_FULL_BACKUP_TS=`tail -1 /home/tidb/br/log/br-fullbackup-${SIX_DAY_AGO}.log | awk -F 'BackupTS=' '{print $2}' | awk -F ']' '{print $1}'`

if [[ $FULL_BACKUP_TS == $LOG_FULL_BACKUP_TS ]]; then
  # Activate conda environment
  source /home/tidb/.bashrc

  # Switch to virtual environment
  conda activate aws-client

  # Delete expired full backups (9 days ago), keeping two historical versions
  NINE_DAY_AGO=`tail -4 /home/tidb/br/stats/fullbackup | head -1`

  aws --endpoint-url your s3 address s3 rm --recursive s3://wfmodel/tidb-br/tidb-online/fullbackup/${NINE_DAY_AGO}
  
  # Delete expired incremental backups (6 days ago)
  tiup br:v6.5.3 log truncate --until=${FULL_BACKUP_TS} --storage 's3://wfmodel/tidb-br/tidb-online/logbackup?endpoint=s3 address&access-key=your key&secret-access-key=your secret" -y --log-file /home/tidb/br/log/br-truncate-${TODAY}.log
else
  echo "`date +'%Y-%m-%d %H:%M:%S'` FAILED" > /home/tidb/br/br-fullstatus.log
fi
| username: dba远航 | Original post link

This is very simple, just write one casually and it will be fine.

| username: zhanggame1 | Original post link

Just use tar, don’t compress it into tar.gz, it’s 10 times slower. SST files are already compressed, so compressing them again is pointless.

| username: zhanggame1 | Original post link

Looks more high-end :grinning:

| username: 随便改个用户名 | Original post link

Learned.

| username: yulei7633 | Original post link

#!/bin/bash
#
timestr=`date +"%Y%m%d"`
host_group='tidb cluster'                          # Project or nucleus it belongs to
host_master='10.0.47.22'                  # Main database VIP, if no VIP, fill in the real IP of the main database
port_master=4000                          # Main database port
host_backup='10.0.47.22'                   # Backup database IP
port_backup=4000                          # Backup database port
Backup_msg=''                                     # Backup error message collection
backdir="/mnt/storageback/tidb_backup"
putdir=${backdir}/${timestr}                      # Backup database directory
#
/usr/bin/find /mnt/storageback/tidb_backup/* -maxdepth 1 -type d -mtime +30| xargs rm -fr

/root/.tiup/bin/tiup br backup db --pd "10.0.47.21:2379" --db custom_forms_db --storage "local:///mnt/storageback/tidb_backup/$timestr" --ratelimit 120 --log-file /mnt/storageback/tidb_backup/backupdb_${timestr}.log
| username: yulei7633 | Original post link

/mnt/storageback is a shared directory.

| username: wangccsy | Original post link

Try using the system’s cron.

| username: oceanzhang | Original post link

Does this script need to be deployed on every TiKV node or just one node?