LinuxPhoto.org

The place for all photography and linux fans

BACKUP UNDER LINUX

by Thomas Andersson

email: thomas.b@mossnet.net

    Backup - or not?

    Why? Just for a second, imagine that you loose everything you have stored on your PC(s). Think about it for a moment. What do I have stored on my pc(s)? All my (digital) pictures of my kids, wife, wedding, next of kin lost!!!! Are you scared enough ? You should be..... I read the other day about a family that had their PC stolen during a burglary - the kind of thing you read about in the news papers. They lost all their pictures of their first kid, now 6 years old. Nothing they could do about it - the PC among other things was lost. Now they burn CD's and store with family and friends. Other things, merely a small selection, that could happen are:
     
    • Disk crash
    • Power fluctuations - lightning
    • Fire
    • Flooding - a burst pipe
    • Virus or trojan infection
    • Software bugs

     
    I do not know about you but now adays I take backups regularely - at least once a week. I really want to keep all my picture memories intact. Thus I have even scanned most of my old slides in order to protect them from time and fire.

    What?

    There are a few different ways of doing backups - I do have extensive experience working in the IT industry for over 30 years. Hopefully I have learnt something by making all of those mistakes over the years...
     
    What issues do we need to adress ?
     
    • In house problems like unintentionally deleting a file, file system corruption etc.
    • Hardware errors like disk crashes
    • Disasters like fire, flooding, theft

     
    A and B are less complicated than C since they merely requires a recently updated copy somewhere near the computer. C requires a copy stored somewhere outside of your own home. In short - C requires a media that could easily be moved and stored outside your home like at a friends place or in your bank safe deposit box.
     
    So, let us have a look at our options from a Soho (Small Office, Home Office) environment view with the above issues taken into account:
     

    1. DVD

    • Most of todays PC's are equipped with a DVD burner
    • Small and cost effective media (i.e. the DVD disc)
    • Easy to transfer offsite
    • Time consuming to burn and to verify integrity of backup copy
    • Non incremental (need to take full backups every time)
    • Need to backup in +4Gb chunks, i.e. need for picture library structuring
    • Questionable time resiliance, need to recycle now and then (every 5 years ?)
    • Easily rendered unreadable - a scratch may be enough
    • Many DVD's may be required - hard to keep track/ administer

    2. Online service

    • Off site by default
    • (most are) Easy to use
    • Professional operations
    • Incremental storage (i.e. backups only files changed since last backup)
    • Backup accessible from where ever you are (providing a laptop PC)
    • Requires a very fast Internet connection
    • Backup times of large volumes prohibiting
    • Cost is incremental and volume dependent
    • Restore times are prohibiting

    3. Cassette tape

    • Small media size
    • Easy to transfer offsite
    • Cassettes are relatively well protected
    • Incremental backup
    • Requires (expensive) hardware and software
    • Media is costly
    • Rather time consuming to make backups
    • Tapes need to be recycled and maintained
    • Many cassettes may be required - hard to keep track/ administer
    • Requires some skill to create backup plans

    4. Portable disks

    • Relatively small media size
    • Easy to transfer offsite
    • Environment protected disks can be purchased
    • Not to high investment cost
    • Software required is free and standard with operating system
    • Fast backups (if the sync method is used) - only files that changed since last backup are copied
    • Good time resiliance
    • Easy to verify
    • No incremental backup
    • Requires some Linux technical knowledge

     
    You may not agree to all of the above advantages and disadvantages - make your own selection list according with your requirements and see what is best for you. I have chosen option 4. Portable disks since it is very fast and also relatively cost efficient. As a runner up I would choose 2. Online service with a concern about the running cost.

    How?

    Now, this is a Linux photography site. Therefor I will only cover technicalities in Linux though the metodologies mentioned here works in Windows (Mac also ?) as well. I will show how to implement Option 4 with file syncronization which means that you have a full copy of the data you choose to sync on the external disk.
     
    First you need to purchase at least one external disc - I use two, one is in my bank safe deposit box and the other is connected to a PC. It is switched off until I need to use it. I recommend a 3,5" USB disk since they are faster than the 2,5" and does not take a whole lot of space. The disk size will vary depending on how much data you have - I currently use 2 * 300 Gb USB disks. The USB disk is easy to install - connect the USB cable to your PC and switch it on. Most Linux implementations will automatically mount the USB drive and show it as a drive on your desktop.
     
    Option 4 is easily implemented in Linux - there are two ways: graphical interface utility and script with text interface and file sync program (I use mirrordir). Since the graphical interface utilities varies depending on Linux package and also works a bit differently I have choosen mirrordir and implemented it in a script which you will find at the end of this article. Use your Linux implementation package manager to install mirrordir (in Ubuntu use either Synaptic or text mode: sudo apt-get install mirrordir).
     
    To proceed, ensure that your USB drive is connected, switch'd on and mounted (it should pop up as a drive on your desktop). You need to be logged in as a user with authority to read all directories you want to back up (if you are not sure, use sudo) and also to write them onto the USB drive file system.
     
    Make a list of the directories you want to backup. Use the file manager to display the USB drive - it should be empty. Create the directories you want to backup on the USB drive file system. Edit the backup script at: #HERE etc - put one echo/ mirrordir pair for each directory you want to backup - save it as backup.sh in your home directory. The script in this article assumes that all directories you backup are in your home directory. If they are not you need to fully qualify the path names (i.e. /home/name/dir) in the script.
     
    To run the backup:
     
    • Ensure that your USB disk is switched on and has been mounted (the disk displayed on your desktop)
    • Open a command window and ./backup.sh "USB diskname"
    • When the script completed - check the log (less backup.log)

     
    You can now dismount the USB disk and swith it off. Do not forget to swap the USB disk at home with the one in you bank safety deposit box (or at a friends or...) now and then. I do it once a month (payday ?) which means that in a worst case scenario I can loose one months worth of pictures/ data.

    The backup script - example 1

    If another USB mountpoint than /media/something is used, do either: global change of /media to the mountpoint used on your system (i.e. /mnt) or use script example 2 that allows for a fully qualified mount point. Now - the script example:

    #!/bin/sh
    # backup.sh - Synchronize all directories to backup with the USB hard drive
    # input prompt = USB disk mount point under /media (in case it changes with more than one USB drive)

     
       echo "Now starting backup......"
       rm backup.log
       if [ "${1}" = "" ]
       then
          DISK="usbdisk"
       else
          DISK="${1}"
       fi
       echo "Backup --> /media/${DISK}"
       cd /home/scan

     
    # HERE IS THE LIST OF SYNCHRONIZE COMMANDS - ONE FOR EACH DIRECTORY

     
       echo "..--> RAW"
       mirrordir -v --no-netrc --no-chown --no-chmod RAW /media/$DISK/RAW >>backup.log
       echo "..--> download"
       mirrordir -v --no-netrc --no-chown --no-chmod download /media/$DISK/download >>backup.log
       echo "Backup done - see result: backup.log"


     

    The backup script - example 2

    This example is edited to:
    • Any USB disk mountpoint can be used - MUST BE GIVEN fully qualified i.e. /somedir/otherdir....
    • Fully qualified directories to be synchronized

     
    Now - the script example:
     

    #!/bin/sh
    # backup.sh - Synchronize all directories to backup with the USB hard drive
    # input prompt = USB disk mount point anywhere in format /mountpoint/subdir
    # (in case it changes with more than one USB drive)

     
       echo "Now starting backup......"
       rm backup.log
       if [ "${1}" = "" ]
       then
          DISK="/mnt/usbdisk"
       else
          DISK="${1}"
       fi
       echo "Backup --> ${DISK}"
       cd /home/scan

     
    # HERE IS THE LIST OF SYNCHRONIZE COMMANDS - ONE FOR EACH DIRECTORY

     
       echo "..--> /home/user/RAW"
       mirrordir -v --no-netrc --no-chown --no-chmod /home/user/RAW $DISK/RAW >>backup.log
       echo "..--> /usr/local/confident"
       mirrordir -v --no-netrc --no-chown --no-chmod /usr/local/confident $DISK/confident >>backup.log
       echo "Backup done - see result: backup.log"


 

Copyright 2007, Jacek Góźdź
template by : David Herreman