Home

Simple Backup Script Using rsync

Submitted by Vince on Thu, 2007-08-30 11:00.
Tags:

Here's a simple bash script I created for backing up items to a remote machine. It uses rsync. Rsync is a very powerful tool used primarily for backing up and synchronizing files/directories over the network. If you don't have rsync, you can obtain it here. Or, do one of the following.

For Debian based systems:

apt-get install rsync

For RedHat based systems:

yum install rsync

My laptop is split into a Windows and a Linux partition. It is assumed that you know how to mount a read-only Windows partition (or if you're using captive-ntfs, read/write). I use the Linux partition 80% of the time. The Windows partition is used mainly for financial tasks like Quicken or if I have to run something that is specific to Windows.

Finally, to fully automate this, public/private key pairs must be generated on the host and remote machines in order for password-less logins to occur. Setting up public/private key pairs is beyond the scope of this article. Here's the code.

#!/usr/bin/bash

DUMPDATE=`date +"%Y%m%d"`
LOG=/var/log/laptop.backup/backup_${DUMPDATE}.log
umask 066

{
   # Ensure backup log directory exists. If not, create it.
   if [[ ! -e "/var/log/laptop.backup" ]]
   then
      mkdir /var/log/laptop.backup 
   fi
   
   echo "`date +\"%Y%m%d %H:%M:%S\"`: Starting incremental backup." 
   echo "`date +\"%Y%m%d %H:%M:%S\"`: " 

   echo "`date +\"%Y%m%d %H:%M:%S\"`: Start backup of /home." 
   rsync --verbose  --progress --stats --compress --rsh=/usr/bin/ssh \
      --recursive --times --perms --links --delete \
      --exclude "*bak" \
      --exclude "*~" \
      --exclude "/home/vmancini/BOINC" \
      --exclude "/home/vmancini/fedora" \
      --exclude "Cache/*" \
      --exclude ".xine/*" \
      --exclude ".opera/*" \
      /home backup@192.168.2.6:/extra/laptop.backup
   echo "`date +\"%Y%m%d %H:%M:%S\"`: End backup of /home." 

   echo "`date +\"%Y%m%d %H:%M:%S\"`: Start backup of Windows partition." 
   rsync --verbose  --progress --stats --compress --rsh=/usr/bin/ssh \
      --recursive --times --perms --links --delete \
      --exclude "*bak" \
      --exclude "*~" \
      --exclude "Cache/*" \
      /mnt/win/Document*/root/My\ Documents /mnt/win/Program*/Quicken/VINCE* \ 
      backup@192.168.2.6:/extra/laptop.backup
   echo "`date +\"%Y%m%d %H:%M:%S\"`: End backup of Windows partition." 

   echo "`date +\"%Y%m%d %H:%M:%S\"`: " 
   echo "`date +\"%Y%m%d %H:%M:%S\"`: Completed incremental backup." 

   # Remove log files older than 10 days
   /usr/bin/find /var/log/laptop.backup -name "*.log" -atime +10 -exec rm -f {} \; 

} 2>&1 | tee ${LOG}

exit

Let's break this code down to see what it's doing. We'll skip explanation of the echo statement since they're self-explanatory.

   if [[ ! -e "/var/log/laptop.backup" ]]
   then
      mkdir /var/log/laptop.backup 
   fi

The above code block ensures that the backup log directory is present.

   rsync --verbose  --progress --stats --compress --rsh=/usr/bin/ssh \
      --recursive --times --perms --links --delete \
      --exclude "*bak" \
      --exclude "*~" \
      --exclude "/home/vmancini/BOINC" \
      --exclude "/home/vmancini/fedora" \
      --exclude "Cache/*" \
      --exclude ".xine/*" \
      --exclude ".opera/*" \
      /home backup@192.168.2.6:/extra/laptop.backup

Let's break down rsync's parameters from above.

  • --verbose - Output extra information on what rsync is doing.
  • --progress - Shows the progress during transfer.
  • --stats - Show some file transfer statistics.
  • --compress - Compresses the data during a transfer.
  • --rsh - Specify which remote protocol to use. Ssh is the way to go for security.
  • --recursive - Recurse into the directories. Ensures all subdirectories are being looked at.
  • --times - Preserves the times of the files and directories.
  • --perms - Preserves the permissions of the files and directories.
  • --links - Preserves the state of a symbolic link (it will not expand a symlink).
  • --delete - Deletes files that do not exist on the sending side.
  • --exclude - Files or directories to exclude from transfer.

The last line finally says backup the /home directory (of course, also exclude everything that was excluded in the --exclude parameter list) to a machine on the local LAN using the backup user.

   
   rsync --verbose  --progress --stats --compress --rsh=/usr/bin/ssh \
      --recursive --times --perms --links --delete \
      --exclude "*bak" \
      --exclude "*~" \
      --exclude "Cache/*" \
      /mnt/win/Document*/root/My\ Documents /mnt/win/Program*/Quicken/VINCE* \ 
      backup@192.168.2.6:/extra/laptop.backup

The second rsync job above is very similar to the first one except that it is backing up two directories in the Windows partition.

Here's a little trick that logs the entire process. Everything that within the curly braces, below, will be logged to a log file.

{
  ...
} 2>&1 | tee ${LOG}

This allows items to be written to the log file and to STDOUT. Thus, if ran manually, you'll be able to see what's going on. The 2>&1 means to put STDERR into STDOUT. By piping this to tee ${log}, it will in turn put STDERR into the log file.

I hope this helps. If there are errors or discrepancies, please leave a comment or contact me here to let me know.

If you liked this, please donate a byte or two.