Linux offsite backups made easy

Srdjan Marjanovic

• 5 min read
Linux offsite backups made easy
Photo by benjamin lehman / Unsplash

Several of years ago I finally decided to invest time and money and build a home NAS. I had a bunch of photos from the DSLR scattered across numerous USB hard drives and decades worth of different "backups" so I wanted to centralize them. I did not want to vendor-lock myself and pay for the Google or Microsoft storage plans or similar cloud backup solutions and besides that, I did not want to family photos to their servers. So the only option remaining was to build or buy a NAS server. As I'm a web developer and ocassionally like to play with ops and tinker with linux servers, I decided to buy a cheap PC with good CPU, enough of RAM and throw in some WD RED drives. Although i have 2 x 4tb WD RED drives paired in RAID1 I still wanted to have some additional layer of redundancy.

So I bought another PC....

Few months ago, I moved into my own office. And what better place to put the backup server in than that?

Now I was facing only 2 problems:

  1. servers being on different networks
  2. which backup software to use

Servers being on different networks

This is the problem because servers can not communicate directly when they are on behind different routers, behind a NAT and/or don't' have a static IP.

TAILSCALE TO THE RESCUE!

Tailscale is kind of VPN but instead of on device acting as a central server, all devices registered are part of a network mesh. All devices will get a public IP so any device can connect to it over the internet without any need for port forwarding. Also, all devices can be pronounced as "exit nodes" towards the internet. By installing and setting up the tailscale on both servers, I was now able to make them "see eachother".

Which backup software to use

This was a tough one. I wanted to have incremental backups so I can backtrack files to some point in time but also in order to minimize the overhead space used. I also did not want to have some special backup file format from which I would have to restore the files, but rather just a plain and simple directory and file structure which I would be able to access from the backup server anytime without need for additional software or process.

At first I tried to set up urbackup.com for several days, but I struggled to do so for a number of reasons both on server (office backup server) and on client (home NAS server in this case) side. I looked at couple of different tools but none of them fit the descrption. I also tried to look for some existing rsync scripts that would fit my needs but didn't trust any of them. So I turned to ChatGPT...

rsync TO THE RESCUE!
I asked ChatGPT the following prompt:

I have two linux servers. One at home which serves as home NAS and the other in the office, which should serve as a backup server for some files of the first one. Both servers are using tailscale. I want to have scheduled remote backups for some folders of the home server to the office server. I want to keep monthly backups, one for each of 12 months in a year, i want to keep weekly backup, one for each week in a month, where last weekly backup should become monthly backup for that month, and I also want to keep daily backup for each day in a week, where last backup of the week will also become weekly backup for that week. I dont want to keep full backups. I want to have one full backup and then only add differences to subsequent backups. I dont want files on the backup to be in some weird fileformat for which i would need some software to restore. I just want them to be plain old files and dirrectories organized in some logical manner so I can simply go in to a daily backup and access the whole file list as if it was a full backup.

And this is what it's answer was:

You can achieve this setup using rsync with hard links to efficiently create incremental backups while keeping a fully browsable directory structure.

After some trial and error we were able to make up with the following process:

Step 1: install rsync on both servers:
sudo apt update && sudo apt install rsync

Step 2: Set up SSH access

  1. Generate SSH keys on your home NAS:
    ssh-keygen -t ed25519
  2. Copy the public key to the backup server:
    ssh-copy-id user@office-server-tailscale-ip
  3. Test passwordless SSH:
    ssh user@office-server-tailscale-ip

Step 3. Create the Backup Script:

We'll use rsync with --link-dest to create incremental backups while making them appear as full backups.

#!/bin/bash

#Configuration - You will need to change the values below to suit your needs
SOURCE_DIR="/path/to/home/nas/folders/"  # Directory to backup
BACKUP_BASE="/path/to/office/server/backups"  # Base backup location on the office server
SSH_USER="home_server_user"
REMOTE_HOST="office-server-tailscale-ip"
BACKUP_USER="backup_server_user"  # User who should own the backups on the remote server

# Get current date components
DAY=$(date +%A)           # Monday, Tuesday, ..., Sunday
DATE=$(date +%d-%m-%Y)    # Format: dd-mm-yyyy
WEEK=$(date +%U)          # Week number (00-53)
MONTH=$(date +%m)         # Month number (01-12)
YEAR=$(date +%Y)          # Year

# Define backup directories with date appended
DAILY_BACKUP="$BACKUP_BASE/daily/${DAY}_${DATE}"
WEEKLY_BACKUP="$BACKUP_BASE/weekly/week_${WEEK}_${DATE}"
MONTHLY_BACKUP="$BACKUP_BASE/monthly/${YEAR}-${MONTH}_${DATE}"

# Find the previous daily backup (yesterday’s backup)
PREVIOUS_BACKUP=$(ssh "$SSH_USER@$REMOTE_HOST" "ls -d $BACKUP_BASE/daily/* 2>/dev/null | tail -n 1")

# Rsync options
RSYNC_OPTS="-a --delete --link-dest=$PREVIOUS_BACKUP"

# Perform daily backup
rsync -e ssh $RSYNC_OPTS "$SOURCE_DIR" "$SSH_USER@$REMOTE_HOST:$DAILY_BACKUP"

# Set correct permissions and ownership on the backup server
ssh "$SSH_USER@$REMOTE_HOST" << EOF    
    chown -R $BACKUP_USER:$BACKUP_USER "$DAILY_BACKUP"    
    chmod -R u+rw,g+r,o+r "$DAILY_BACKUP"
EOF

# Every Sunday, create a weekly backup from the last daily backup

if [ "$DAY" == "Sunday" ]; then
    ssh "$SSH_USER@$REMOTE_HOST" "cp -al $DAILY_BACKUP $WEEKLY_BACKUP"     ssh "$SSH_USER@$REMOTE_HOST" "chown -R $BACKUP_USER:$BACKUP_USER $WEEKLY_BACKUP && chmod -R u+rw,g+r,o+r $WEEKLY_BACKUP"
fi

# Check if today is the last day of the month
TOMORROW=$(date -d "tomorrow" +%d)
if [ "$TOMORROW" == "01" ]; then    
    ssh "$SSH_USER@$REMOTE_HOST" "cp -al $DAILY_BACKUP $MONTHLY_BACKUP"     ssh "$SSH_USER@$REMOTE_HOST" "chown -R $BACKUP_USER:$BACKUP_USER $MONTHLY_BACKUP && chmod -R u+rw,g+r,o+r $MONTHLY_BACKUP"
fi

# Clean up old backups (keeping retention policy)
ssh "$SSH_USER@$REMOTE_HOST" << EOF    
    # Keep only the last 7 daily backups    
    cd "$BACKUP_BASE/daily" && ls -t | tail -n +8 | xargs -I {} rm -rf {}  
    # Keep only the last 5 weekly backups
    cd "$BACKUP_BASE/weekly" && ls -t | tail -n +6 | xargs -I {} rm -rf {}
    # Keep only the last 12 monthly backups    
    cd "$BACKUP_BASE/monthly" && ls -t | tail -n +13 | xargs -I {} rm -rf {}
EOF

Step 4. Create the destination directory structure on backup server

mkdir -p /path/to/office/server/backups/daily
mkdir -p /path/to/office/server/backups/weekly
mkdir -p /path/to/office/server/backups/monthly

Step 5: Automate with Cron

crontab -e on home server

add: 0 2 * * * /path/to/backup_script.sh

This will:

  • Run the script every day at 2 AM
  • Log output to /var/log/backup.log (useful for debugging)

Step 6: Verify Everything Works

Test the script by running on home server:

bash /path/to/backup_script.sh

Then verify on the office backup server:

ls -l /path/to/office/server/backups/daily

Example directory structure you should end up with

/backups/
├── daily/
│   ├── Monday_27-03-2025
│   ├── Tuesday_28-03-2025
│   ├── Wednesday_29-03-2025
│   ├── Thursday_30-03-2025
│   ├── Friday_31-03-2025  <-- Last daily backup of the month
│
├── weekly/
│   ├── week_10_10-03-2025
│   ├── week_11_17-03-2025
│   ├── week_12_24-03-2025
│
├── monthly/
│   ├── 2025-01_31-01-2025  <-- Last daily backup of January
│   ├── 2025-02_29-02-2025  <-- Last daily backup of February (leap year)
│   ├── 2025-03_31-03-2025  <-- Last daily backup of March

Final thoughts:
✅ rsync's --delete option will delete all the files on the backup server that are missing or have been deleted on the home server. You might want to remove this one, it's up to you
✅ Ensures first backup is always a full backup (when no previous backup exists).
✅ Uses incremental backups (--link-dest) only when a previous backup exists.
✅ Automatically creates missing backup directories (mkdir -p $DAILY_BACKUP).
✅ Prevents rsync from failing due to missing previous backup folders.
✅ Maintains correct ownership (chown user:user) and permissions (chmod -R u+rw,g+r,o+r).

Sign up for more like this.

Enter your email
Subscribe