Simple backup system for Linux with Bash, Tar, Rsync and Crontab pt.1/2

Introduction

I have spent many hours for backingup my servers files. I used to have one text file which contained predefined tar and rsync commands and I was copy-pasting them. Sometimes I did this once in a week and sometimes once in a month. This wishy washy thing was totally ok but it was waste of a time. After all this mess I realised I have to do something for this. I decided to automize it with simple bash-scripts and crontab.

Tested on Ubuntu 14.04 LTS.

This first part contains information how to make backups. Second part will show how to recover the backups.

This is how it works in a nutshell

First of all I have two Linux servers. Production node and node for storing my backups. I also have setuped public key authentication between them(nice guide for public key authentication). For example if I am logged in my production node I am able to login to backup-node like this(without asking for a password):

[raw]$ ssh backup-node[/raw]

The backup process is handled by these two files:

[raw]backup-tar.sh backup-rsync.sh[/raw]

backup-tar.sh is the script which will make mysql dump(backup of a database) and compress my full-system(/) in single tar.gz file. This script is executed by root user crontab.

backup-rsync.sh is executed by normal user crontab. It contains credentials for sending backup-files to backup node via rsync.

PLEASE NOTICE! There are two separate files for a reason. Tar process has to be done with root privilleges(sudo) since it will compress files owned by root aswell. But if we want to rsync without password authentication it is not possible with sudo. If you try to do: $sudo rsync … it will always ask for password.

backup-tar.sh

[raw]#!/bin/bash
#Purpose = Backup of Important Data
#Created on 05-05-2015
#Author = Tuukka Merilainen
#Version 1.0

#START
# Tar credentials
DATE=`date +%d-%b-%Y`                  # This Command will add date in Backup File Name.
FILENAME=fullbackup-$DATE.tar.gz       # Here I define Backup file name format.
SRCDIR=/                               # Location of Important Data Directory (Source of backup).
DESDIR=/example/please/change        # Destination of backup file.

# Database credentials
user=”root”                             # username for database eg. root
password=”example-password”   # password for database
db_name=”fulldbbackup”                  # name for db backupfile
backup_path=”/example/please/change” # path where backup is stored

# Dump database into SQL file
mysqldump –user=$user –events –ignore-table=mysql.event –password=$password –all-databases > $backup_path/$db_name-$DATE.sql

# Make tarball of /
tar -cpzf $DESDIR/$FILENAME –directory=/ –exclude=proc –exclude=sys –exclude=dev/pts –exclude=$DESDIR $SRCDIR
#END[/raw]

backup-rsync.sh

[raw]#!/bin/bash
#Purpose = Sync backup files to an another server
#Created on 05-05-2015
#Author = Tuukka Merilainen
#Version 1.0
#START

rsync -a –bwlimit=5000 -e ssh –hard-links –inplace sourcefolder destinationuser@example.com:/full-backup

#END[/raw]

How to make it work

Download and extract the scripts

[raw]$ wget http://tuukkamerilainen.com/files/backup.tar.gz[/raw]
[raw]$ tar -xf backup.tar.gz[/raw]

Edit the scripts to match your environment

Take care of filepaths and address for a backup server. Atleast modify everything marked with something like “example”.

Make two different crontabs

Firsr for backup-tar.sh
[raw]$ sudo crontab -e[/raw]

Add line:
[raw]00 08 * * 7 /bin/bash /path/to/backup-tar.sh[/raw]
This will run the backup-tar.sh script every sunday at 08:00.

Then for backup-rsync.sh
[raw]$ crontab -e[/raw]

Add line:
[raw]00 23 * * 7 /bin/bash /home/tuukka/backup-rsync.sh[/raw]
This will run the backup-rsync.sh script every sunday at 23:00.

Sources

http://www.broexperts.com/2012/06/how-to-backup-files-and-directories-in-linux-using-tar-cron-jobs/

https://ukk.kapsi.fi/questions/164/synologyn-rsyncilla-siiloon