Simple yet powerful backup solution for Linux user data

One of the things I love about Linux is that it comes with a huge suite of software and utilities that can be very powerful. Some of these utilities go un-noticed or are not widely known. In this example I'll focus on backing up user data; I am a strong believer of making good backups of data at all times, to prepare for the worst. There's nothing worse than losing data and having to live with older copies or missing files.

With Linux, the entire user's profile folders are stored in /home. What's even better about this, is all applications that the user runs also store settings in this profile folder. So, my main focus is to get good backups of the user's profile folders that are stored in /home, and this will ensure that if the hard drive goes up in flames, that all data can be restored allowing the user to be back up and running with little effort.

For doing these backups, I use a lightweight but extremely powerful utility called "rsync". It's a command-line based utility, but can handle gigs of data across slow links, making it very effective. In fact, it's one of the leading utilities used to replicate large public software mirrors among sites. What makes rsync most appealing to me is it can identify changes in files, and will only transfer data that has changed since the last sync, making it extremely efficient. I've used various similar programs in Windows to rsync, and have NEVER found anything as effective as rsync on Linux. Mainly, issues like "files in use" and other problems crept up in Windows, which caused incomplete backups and errors, and never gave us 100% reliable backups. Not to mention, the ability to run a sync in Windows silently seemed to be a challenge also, where rsync on Linux can be run silently in the background while the user continues to work, and there are never any file locking issues which is extremely important for getting a complete backup.

Using rsync involves a little setup and knowledge of editing text files on a Linux system. You must have rsync running on a host/server system where the files are stored, and also on the client where the files are backed up from. And the beauty about Linux is that the host/server and client computers can be any sort of machines because Linux distributions contain the same software and rsync is relatively light on system load.

On the host/server:

Make sure the "xinetd" and "rsync" packages are installed. Edit the file /etc/xinetd.d/rsync, and make sure it has contents similar to below:

# default: off # description: The rsync server is a good addition to an ftp server, as it \ # allows crc checksumming etc. service rsync { disable = no # flags = IPv6 socket_type = stream wait = no user = root server = /usr/bin/rsync server_args = --daemon log_on_failure += USERID }

This makes xinetd start the rsyncd service. You can double check by running "ntsysv" and making sure "rsyncd" is checked so that it starts automatically on boot.

Then edit the file /etc/rsyncd.conf and add a backup point where the data will be stored, similar to the sample below:

[backup] path = /ext/rsync/backupfolder uid = localuser gid = localgroup read only = false comment = Workstation backup auth users = localuser secrets file = /etc/rsyncd.secrets

Edit the file /etc/rsyncd.secrets (or create it if it doesn't exist), add a line with the username/password specified in the file above, as "username:password". In this example, "localuser:backuppassword" would suffice. You can create mutiple backup points (one per user) and specify separate usernames and passwords by creating multiple entries in both of these files.

On the client system:

Create a file (I usually store this in the user's folder, i.e. /home/localuser/.rsync.passwd). Putting the dot in front of the file makes it hidden. In that file, put in the same password specified in the server's settings file above, in this case "backuppassword"; you do not need the username in this file.

Next, you simply need to add a cron job to run the backup job on the client machine. I do this by creating a new file in /etc/cron.hourly, which cron will run on the hour. I usually name it : /etc/cron.hourly/z-rsync-backup, so that cron will run it last. Be sure to "chmod 775" to this file to allow cron to run it. Then in this file, put a line in it similar to below:

rsync -aq /home/localuser/ localuser@rsyncserverip::backupfolder/profilefolder --prune-empty-dirs --delete --delete-excluded --exclude-from=/home/localuser/.rsync-exclude --password-file=/home/localuser/.rsync.passwd

This creates a live "copy" or sync of the user's profile folder on the host/server.

The final step is to create the file referenced in the run command above, /home/localuser/.rsync-exclude , and put in entries for items that you do NOT want to back up. I usually add the following entries:

.mozilla/**/Cache *.o temp/* .gvfs

This skips over Firefox's cache, any object files, any files if there's a folder called "temp", and the .gvfs folder.

Conclusion

The possibilities are endless for rsync. Read the man page by using "man rsync" if you want to read what all of the options do.

This solution can be added or enhanced for added functionality. I use one scenario like this and on the host/server system, I set rsync on the host/server to back up to the path "/media/backupdrive" (which is a mounted ext4 USB drive named "backupdrive"). This is done with the "path =" line in /etc/rsyncd.conf. This way the USB drive can be removed and stored in a safe place. Another solution I use is to have rsync back up remote systems to another site over the Internet, using ssh tunnelling to provide a secure connection.

I currently run my own backups like this with multiple workstations that save the files to a server, then on the server the files are backed up to tape. I have put it to the test by formatting a hard drive and installing Fedora from scratch again, and restoring the data backed up with rsync. When logging in, all profile data is read like normal and everything works just like it did before. In fact, an rsync command could be written to do the opposite where it connects to the host/server and restores the data back to the client. These are all examples of how GNU/Linux is really excellent stuff, and just works every time without a lot of hassle like we have in Windows.