Setting Up Automated Backups with rsync, borgbackup and

Data loss can be catastrophic for individuals and organizations alike. Implementing robust, automated backup solutions is essential for protecting valuable data against hardware failures, accidental deletion, ransomware, and other disasters. This comprehensive guide explores setting up automated backup systems on Linux using rsync for simple file synchronization, borgbackup for deduplicating encrypted backups, and systemd timers for reliable scheduling.

Backup Strategy Fundamentals

The 3-2-1 Backup Rule

A sound backup strategy follows the 3-2-1 rule:

3 copies of your data (original + 2 backups)
2 different media types (e.g., local disk + cloud storage)
1 copy offsite (protect against physical disasters)

Backup Types

Full Backup: Complete copy of all data

Pros: Simple restoration, complete snapshot
Cons: Time-consuming, storage-intensive

Incremental Backup: Only changes since last backup

Pros: Fast, efficient storage
Cons: Complex restoration (requires all increments)

Differential Backup: Changes since last full backup

Pros: Faster restoration than incremental
Cons: More storage than incremental

Continuous Backup: Real-time or near-real-time replication

Pros: Minimal data loss
Cons: Resource-intensive, complex setup

What to Back Up

Essential data:

User home directories (/home/)
System configuration (/etc/)
Web server content (/var/www/)
Database files or dumps
Application data
Custom scripts and configurations

Generally exclude:

Temporary files (/tmp/, /var/tmp/)
Cache directories
System binaries (reinstallable via package manager)
Virtual filesystems (/proc/, /sys/, /dev/)

Automated Backups with rsync

rsync is a versatile tool for synchronizing files and directories locally or over networks.

Understanding rsync

Key features:

Incremental transfer (only changed data)
Preserves permissions, ownership, timestamps
Compression during transfer
SSH support for secure remote transfers
Exclude patterns for fine-grained control

Basic syntax:

rsync [options] source destination

Local Backup with rsync

Simple local backup:

# Basic sync
rsync -av /source/ /backup/

## Options explained:
## -a (archive): Preserve permissions, timestamps, etc.
## -v (verbose): Show progress

Comprehensive backup script:

#!/bin/bash

## Configuration
SOURCE="/home/"
DEST="/mnt/backup/home-backup"
LOG="/var/log/backup.log"
DATE=$(date +%Y-%m-%d_%H-%M-%S)

## Create destination if it doesn't exist
mkdir -p "$DEST"

## Run rsync
rsync -av \
    --delete \
    --exclude='lost+found' \
    --exclude='.cache' \
    --exclude='.thumbnails' \
    --exclude='*.tmp' \
    --log-file="$LOG" \
    "$SOURCE" "$DEST/"

## Log completion
echo "[$DATE] Backup completed" >> "$LOG"

Incremental Backups with rsync

Use hardlinks for space-efficient incremental backups:

#!/bin/bash

BACKUP_DIR="/mnt/backup"
SOURCE="/home/"
DATE=$(date +%Y-%m-%d)
LATEST="$BACKUP_DIR/latest"
DEST="$BACKUP_DIR/$DATE"

## Create backup with hardlinks to previous backup
rsync -av \
    --delete \
    --link-dest="$LATEST" \
    "$SOURCE" "$DEST/"

## Update latest symlink
rm -f "$LATEST"
ln -s "$DATE" "$LATEST"

This creates full backups efficiently—unchanged files are hardlinked to previous backup, consuming no additional space.

Remote Backup with rsync over SSH

Push to remote server:

rsync -avz \
    --delete \
    -e "ssh -p 22 -i /home/user/.ssh/backup_key" \
    /home/ user@backup-server:/backups/hostname/

Pull from remote server:

rsync -avz \
    --delete \
    -e "ssh -p 22" \
    user@remote-server:/data/ /local/backup/

SSH key setup (for passwordless authentication):

## Generate SSH key
ssh-keygen -t ed25519 -f ~/.ssh/backup_key

## Copy to remote server
ssh-copy-id -i ~/.ssh/backup_key.pub user@backup-server

## Test connection
ssh -i ~/.ssh/backup_key user@backup-server

Advanced rsync Options

Bandwidth limiting:

rsync -av --bwlimit=5000 /source/ /dest/
## Limit to 5000 KB/s

Progress and statistics:

rsync -av --progress --stats /source/ /dest/

Partial transfer resume:

rsync -av --partial --progress /source/ /dest/

Exclude patterns:

rsync -av \
    --exclude='*.log' \
    --exclude='cache/' \
    --exclude-from='/etc/backup-exclude.txt' \
    /source/ /dest/

Dry run (test without changes):

rsync -avn --delete /source/ /dest/

rsync Backup Script with Rotation

#!/bin/bash

## Configuration
SOURCE="/home/"
BACKUP_BASE="/mnt/backup/rsync"
KEEP_BACKUPS=7
LOG="/var/log/rsync-backup.log"
DATE=$(date +%Y-%m-%d)
BACKUP_DIR="$BACKUP_BASE/$DATE"

## Create backup directory
mkdir -p "$BACKUP_DIR"

## Find latest backup for hardlinking
LATEST=$(find "$BACKUP_BASE" -maxdepth 1 -type d -name "20*" | sort -r | head -n 1)

## Perform backup
echo "$(date): Starting backup to $BACKUP_DIR" >> "$LOG"

if [ -n "$LATEST" ] && [ "$LATEST" != "$BACKUP_DIR" ]; then
    rsync -av \
        --delete \
        --link-dest="$LATEST" \
        --exclude-from=/etc/backup-exclude.txt \
        "$SOURCE" "$BACKUP_DIR/" 2>&1 | tee -a "$LOG"
else
    rsync -av \
        --delete \
        --exclude-from=/etc/backup-exclude.txt \
        "$SOURCE" "$BACKUP_DIR/" 2>&1 | tee -a "$LOG"
fi

## Rotate old backups
echo "$(date): Removing backups older than $KEEP_BACKUPS days" >> "$LOG"
find "$BACKUP_BASE" -maxdepth 1 -type d -name "20*" -mtime +$KEEP_BACKUPS -exec rm -rf {} \;

echo "$(date): Backup completed" >> "$LOG"

Automated Backups with BorgBackup

BorgBackup is a modern, efficient backup solution featuring deduplication, compression, and encryption.

Why BorgBackup?

Advantages:

Deduplication: Only stores unique data chunks
Compression: Multiple compression algorithms
Encryption: Built-in AES-256 encryption
Verification: Cryptographic integrity checks
Efficient: Fast backups and restores
Space-efficient: Deduplication saves significant space

Installing BorgBackup

## Debian/Ubuntu
sudo apt install borgbackup

## Fedora/RHEL
sudo dnf install borgbackup

## Arch Linux
sudo pacman -S borg

Initializing a Borg Repository

Local repository:

borg init --encryption=repokey /path/to/repo

Remote repository:

borg init --encryption=repokey user@backup-server:/path/to/repo

Encryption modes:

repokey: Key stored in repo, backed up with passphrase
keyfile: Key stored in ~/.config/borg/keys/
none: No encryption (not recommended)

Set passphrase:

export BORG_PASSPHRASE='your-secure-passphrase'

Or use a key file:

export BORG_KEY_FILE='/path/to/keyfile'

Creating Backups with Borg

Basic backup:

borg create /path/to/repo::backup-name /data/to/backup

Comprehensive backup:

borg create \
    --verbose \
    --stats \
    --progress \
    --compression lz4 \
    /path/to/repo::{hostname}-{now} \
    /home \
    /etc \
    --exclude '/home/*/.cache' \
    --exclude '*.tmp'

Placeholders:

{hostname}: System hostname
{now}: Current timestamp
{user}: Current username

Borg Backup Script

#!/bin/bash

## Configuration
REPO="/mnt/backup/borg-repo"
HOSTNAME=$(hostname)
DATE=$(date +%Y-%m-%d)
ARCHIVE="${HOSTNAME}-${DATE}"

## Passphrase
export BORG_PASSPHRASE='your-secure-passphrase'

## Backup sources
SOURCES="/home /etc /var/www"

## Exclusions
EXCLUDES=(
    '/home/*/.cache'
    '/home/*/.thumbnails'
    '*.tmp'
    '*.log'
)

## Build exclude arguments
EXCLUDE_ARGS=""
for pattern in "${EXCLUDES[@]}"; do
    EXCLUDE_ARGS="$EXCLUDE_ARGS --exclude '$pattern'"
done

## Create backup
echo "Starting backup: $ARCHIVE"
borg create \
    --verbose \
    --stats \
    --compression lz4 \
    $EXCLUDE_ARGS \
    "$REPO::$ARCHIVE" \
    $SOURCES

## Prune old backups
echo "Pruning old backups"
borg prune \
    --verbose \
    --list \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 6 \
    "$REPO"

## Verify backup integrity
echo "Verifying backup integrity"
borg check "$REPO::$ARCHIVE"

echo "Backup completed: $ARCHIVE"

Restoring from Borg

List archives:

borg list /path/to/repo

List contents of archive:

borg list /path/to/repo::archive-name

Extract entire archive:

borg extract /path/to/repo::archive-name

Extract specific files:

borg extract /path/to/repo::archive-name path/to/file

Mount archive as filesystem:

mkdir /mnt/borg
borg mount /path/to/repo::archive-name /mnt/borg
## Browse files
ls /mnt/borg
## Unmount when done
borg umount /mnt/borg

Borg Maintenance

Compact repository (reclaim space):

borg compact /path/to/repo

Check repository integrity:

borg check /path/to/repo

Get repository info:

borg info /path/to/repo
borg info /path/to/repo::archive-name

Remote Borg Backups

SSH-based remote backup:

#!/bin/bash

REPO="user@backup-server:/backups/borg-repo"
HOSTNAME=$(hostname)
ARCHIVE="${HOSTNAME}-$(date +%Y-%m-%d)"

export BORG_PASSPHRASE='your-secure-passphrase'
export BORG_RSH="ssh -i /home/user/.ssh/backup_key"

borg create \
    --verbose \
    --stats \
    --compression lz4 \
    "$REPO::$ARCHIVE" \
    /home /etc /var/www

borg prune \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 6 \
    "$REPO"

Scheduling with systemd Timers

systemd timers provide a robust alternative to cron for scheduling tasks.

Creating a Backup Service

Service file: /etc/systemd/system/backup.service

[Unit]
Description=Backup Service
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup.sh
User=root
Nice=19
IOSchedulingClass=idle

Creating a Timer Unit

Timer file: /etc/systemd/system/backup.timer

[Unit]
Description=Daily Backup Timer

[Timer]
OnCalendar=daily
OnCalendar=*-*-* 02:00:00
Persistent=true
RandomizedDelaySec=30min

[Install]
WantedBy=timers.target

Options explained:

OnCalendar=daily: Run daily
OnCalendar=*-*-* 02:00:00: Specifically at 2:00 AM
Persistent=true: Catch up missed runs
RandomizedDelaySec=30min: Random delay up to 30 minutes

Enabling and Managing Timers

## Reload systemd
sudo systemctl daemon-reload

## Enable timer
sudo systemctl enable backup.timer

## Start timer
sudo systemctl start backup.timer

## Check timer status
systemctl status backup.timer

## List all timers
systemctl list-timers

## View timer logs
journalctl -u backup.service
journalctl -u backup.timer

## Manually trigger backup
sudo systemctl start backup.service

Complete Automated Backup System

Backup script: /usr/local/bin/borg-backup.sh

#!/bin/bash

## Configuration
REPO="/mnt/backup/borg"
HOSTNAME=$(hostname)
ARCHIVE="${HOSTNAME}-$(date +%Y-%m-%d-%H%M)"
LOG="/var/log/borg-backup.log"

## Passphrase
export BORG_PASSPHRASE='your-secure-passphrase'

## Logging function
log() {
    echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG"
}

## Error handling
handle_error() {
    log "ERROR: $1"
    exit 1
}

log "Starting backup: $ARCHIVE"

## Pre-backup hook (e.g., database dump)
log "Running pre-backup tasks"
mysqldump --all-databases | gzip > /tmp/mysql-dump.sql.gz || handle_error "MySQL dump failed"

## Create backup
log "Creating Borg archive"
borg create \
    --verbose \
    --stats \
    --compression lz4 \
    --exclude '/home/*/.cache' \
    --exclude '/tmp/*' \
    --exclude '*.tmp' \
    "$REPO::$ARCHIVE" \
    /home \
    /etc \
    /var/www \
    /tmp/mysql-dump.sql.gz 2>&1 | tee -a "$LOG" || handle_error "Backup creation failed"

## Prune old backups
log "Pruning old backups"
borg prune \
    --verbose \
    --list \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 6 \
    --keep-yearly 2 \
    "$REPO" 2>&1 | tee -a "$LOG" || log "WARNING: Prune failed"

## Compact repository
log "Compacting repository"
borg compact "$REPO" 2>&1 | tee -a "$LOG"

## Verify backup
log "Verifying backup integrity"
borg check "$REPO::$ARCHIVE" 2>&1 | tee -a "$LOG" || handle_error "Verification failed"

## Post-backup cleanup
rm -f /tmp/mysql-dump.sql.gz

log "Backup completed successfully: $ARCHIVE"

## Send notification (optional)
## mail -s "Backup successful: $HOSTNAME" [email protected] < /dev/null

Make executable:

sudo chmod +x /usr/local/bin/borg-backup.sh

Service: /etc/systemd/system/borg-backup.service

[Unit]
Description=BorgBackup Service
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/borg-backup.sh
User=root
Nice=19
IOSchedulingClass=idle
StandardOutput=journal
StandardError=journal

Timer: /etc/systemd/system/borg-backup.timer

[Unit]
Description=Daily BorgBackup Timer

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true
RandomizedDelaySec=15min

[Install]
WantedBy=timers.target

Enable:

sudo systemctl daemon-reload
sudo systemctl enable --now borg-backup.timer

Monitoring and Alerting

Email Notifications

Install mail utilities:

sudo apt install mailutils

Add to backup script:

## At end of script
if [ $? -eq 0 ]; then
    echo "Backup completed successfully" | mail -s "Backup Success: $HOSTNAME" [email protected]
else
    echo "Backup failed. Check logs at $LOG" | mail -s "Backup FAILED: $HOSTNAME" [email protected]
fi

Monitoring with systemd-cron-next

Check next scheduled run:

systemctl list-timers backup.timer

Log Rotation

Configure log rotation: /etc/logrotate.d/backup

/var/log/backup.log {
    weekly
    rotate 12
    compress
    delaycompress
    missingok
    notifempty
}

Best Practices

Security

Encrypt backups: Always use encryption for sensitive data
Secure passphrases: Use strong, unique passphrases
Protect credentials: Restrict access to scripts with credentials
Use dedicated SSH keys: Separate keys for backups
Limit access: Backup user should have minimal permissions
Offsite backups: Store copies remotely
Test encryption: Verify you can decrypt backups

Reliability

Test restores regularly: Verify backups are recoverable
Monitor backup jobs: Alert on failures
Check disk space: Ensure adequate storage
Verify integrity: Run periodic integrity checks
Document procedures: Maintain restoration documentation
Automate verification: Script integrity checks
Keep multiple versions: Retain historical backups

Performance

Schedule during off-hours: Minimize production impact
Use compression: Balance speed vs. size
Bandwidth limiting: Don’t saturate network
Nice priority: Lower backup process priority
Incremental backups: Minimize backup time
Exclude unnecessary data: Don’t backup temp files

Maintenance

Prune regularly: Remove old backups
Monitor growth: Track backup size trends
Update software: Keep backup tools updated
Review exclusions: Ensure relevant data is backed up
Audit logs: Review backup logs periodically
Test disaster recovery: Practice full restoration

Troubleshooting

rsync Issues

Permission denied:

## Use sudo or appropriate user
sudo rsync -av /source/ /dest/

Partial transfers:

## Resume with --partial
rsync -av --partial /source/ /dest/

Slow transfers:

## Check network bandwidth
## Adjust compression
rsync -avz --bwlimit=5000 /source/ /dest/

Borg Issues

Repository locked:

## Break lock if backup crashed
borg break-lock /path/to/repo

Passphrase forgotten:

If using repokey, passphrase is essential
Keep secure backup of passphrase
No recovery without passphrase

Corrupted repository:

## Check and repair
borg check --repair /path/to/repo

Conclusion

Implementing automated backups is critical for data protection. rsync provides simple, efficient file synchronization suitable for basic backup needs. BorgBackup offers advanced features like deduplication, compression, and encryption for comprehensive backup solutions. systemd timers ensure reliable scheduling with better logging and management than traditional cron.

A robust backup strategy combines appropriate tools, regular automation, offsite storage, and periodic testing. The scripts and configurations provided in this guide establish a solid foundation for protecting valuable data against loss, corruption, and disasters.

Remember: backups are only valuable if they can be restored. Regularly test your restoration procedures to ensure your backup strategy actually protects your data when needed.

References

rsync Documentation: https://rsync.samba.org/documentation.html
BorgBackup Documentation: https://borgbackup.readthedocs.io/
systemd.timer Manual: https://www.freedesktop.org/software/systemd/man/systemd.timer.html
Linux Backup Best Practices: https://www.kernel.org/doc/html/latest/admin-guide/