Recent events gave me cause to consider my personal data backup and redundancy strategy for my Debian installs. Or, more accurately, it caused me to amend my half-baked and semi-implemented existing approach so that I won't lose data or have to reconfigure things from memory/scratch in the event of a hard disk failure.
My present “backup” approach is really somewhere between a time-limited backup and redundant storage. Essentially, I use Unison to synchronize my home folder (with certain sub-folders ignored, e.g. certain git repositories, config and thumbnail cache folders) between my desktop and my netbook. I have to run Unison manually, so I end up synchronizing my data every week, give or take. This effectively kills two birds with one stone: I get to have local copies of my important data as up-to-date as my last sync for when I'm on the road and using my netbook, and the delayed redundancy gives me some protection should anything go wrong with either hard drive.
I'll be clear: what I'm doing here is something not-quite a backup (safe storage of historical editions of files) and not quite proper redundancy (up-to-date duplication of data). The frequency of synchronization is the key factor, here. Too frequent, and it's possible that I may unintentionally change or delete something on one system that I will not be able to restore because the change was already propagated to the other. A proper backup solution would not suffer from this. On the other hand, too infrequent, and it's possible that a drive will fail long enough since last sync that I lose some data. A proper redundant storage solution would not suffer from this.
That point made, I argue that a few days to a week is quite reasonable for this to work well: it gives sufficient time to notice almost all accidents, it's frequent enough that I'm not likely to lose much, if any, new data, and it's not so frequent that doing it becomes prohibitively annoying.
Of course, this approach only solves part (well, most) of the problem. The other part is preservation of the base system, i.e. the packages and configuration. One can, in fact, reduce this down to the system configuration and any non-repository or locally-created packages, since repository packages can always be re-downloaded.
So what does that leave? Basically, get-selections and a particular set of folders that contain configuration and local-package data one would need in order to restore a system back to its former glory. Since these are things I'd want to backup on a regular basis, I wrote a script—basicsysbackup.sh—that will tar and compress (with xz) all the relevant locations and package selections. I can then keep the resulting tarball backed-up/stored redundantly alongside my personal data, dovetailing nicely with the method above. Restoration, to be fair, is certainly not automated in this approach, but certainly it is doable.
#! /bin/sh
# Brendon L. Higgins <brendon@quantumfurball.net>
# 2014-01-11
# This is a script to backup the basic system files, excluding standard package
# data that could be downloaded, and excluding user home directories. The
# intent is that the system could in principle be reconstructed by hand using
# this backup and a base install, with user data restored from a separate
# redundant copy.
# Run this as a normal user - sudo will prompt for your password.
# Turn globbing off
set -f
TIMESTAMP=`date +%F+%H_%M`
HOSTNAME=`hostname`
# Only stat existing dirs (could just ignore tar error messages, but tar could
# give other errors - this is cleaner). /home is intentionally absent.
DIRS=`ls -fd /etc /opt /root /srv /usr/local /var 2> /dev/null`
INCL=""
for x in ${DIRS}; do
# Necessary to remove newlines
INCL="${INCL} ${x}"
done
# Exclude some particular dirs/files
DIRS=`ls -fd /var/cache /var/crash /var/lib /var/lock /var/run /var/swap /var/tmp 2> /dev/null`
EXCL=""
for x in ${DIRS}; do
EXCL="${EXCL} --exclude ${x}/* --exclude ${x}"
done
USRGRP=`id -u`:`id -g`
BACKUPNAME="${TIMESTAMP} ${HOSTNAME} basic sys backup.tar.xz"
SELSNAME="${TIMESTAMP} package selections.txt"
dpkg --get-selections > "${SELSNAME}"
# Must run as root. We can then set the owner of the file to the user, this way.
sudo sh -fc "tar ${EXCL} -vcpf \"${BACKUPNAME}\" --xz \"${SELSNAME}\" ${INCL} ; chown ${USRGRP} \"${BACKUPNAME}\""
rm "${SELSNAME}"
Perhaps what remains to consider, now, is the case of total failure. I was curious to work out just how likely it would be, despite my efforts, that I could actually lose all my data. Consider: let x be the probability that a hard drive will fail within a year. The probability of either one out of the two hard drives failing is 2x, and both failing in the same year is x2. But suppose that, if a hard drive fails, I can get a replacement within, say, a week. Assuming the hard drive failure probability is roughly constant, the probability of the second drive failing in the same week as the first would be x ⁄ 52. So overall, the probability of finding myself in a situation where both of my drives have failed is 2x×x ⁄ 52 = x2 ⁄ 26.
Therefore, if the probability that a hard drive will fail in a given year is one in ten (which is probably an overestimate, but then again, we're ignoring any variation in failure probability over time, here), then the likelihood that all my data will be lost in a given year is less than 0.04%. Good enough odds, I think.
Comment to add? Send me a message: <brendon@quantumfurball.net>