Greg,
I actually do my backups on Windows 2000, and so do not currently have any Linux scripts - however let me give you a layout of such a beast!
First - and this is really cool, I don't use ANY foreign file formats. No archived tar files and no compression except on Windows 2000 since it does it transparently.
By this I mean...backups are simply COPIES of their originals - in the exact same directory tree structure! This is VERY important, as it lets any authorized operator restore a damaged or erased file simply by drag-copying the backup copy back to where it originated, or some other location if they're timid about doing otherwise (which I ususally am).
So imagine your backup as a complete clone or photocopy of your original. You can clone the whole thing, OS and all, or just your custom utilities and scripts, user files and network shared folders. With Windows, I only clone the network shares and my utilities. With Linux, I might be more inclined to clone the whole thing, but you must backup each file system except of course, the swap partition.
And if you expect the backup copy to be bootable, you'd better copy that too! You'll need to create partitions for each file system of the exact same type for it to boot. Otherwise, you can redirect each file system to a folder (forgive me, I meant to say "directory") on your backup media.
With Windows, I use xcopy or another utility called xxcopy, both of which [finally] have excellent command-line sets that allow you to backup files and directories en masse complete with permissions.
With Linux, I'd use the cp, copydir or mirrordir commands. The latter two do not come with all distributions, but you can find them using Google. I found mirrordir at Tucows, and found the link to
www.sourceforge.net where it's hosted as an open source project under the GNU license.
Mirrordir clones directory trees with permissions and hard links. Your backup procedure needs to be well written using tests to make sure the media is available and mounted, as mirrordir can copy the wrong way if your source and destinations are not named correctly. (Same with any "copy" command.)
Note that mirrordir by default, deletes files on the destination (your backup drive) that no longer exist on the source. cp on the other hand, will not. I am unaware if mirrordir has an option to leave existing files intact. I happen to like doing so, as I use BIG or compressed drives to backup smaller ones, allowing me to keep the old junk as long as I can. (You can just imaging the look of my basement from this admission.)
There's nothing nicer than making your customer downright exhuberant when you're able to recover last year's tax returns in 10 seconds - because they were never deleted from the backup media. Very cool!
You still need archives if you want a periodic snapshot of all the firm's files - but you can do that on tape or CD-R if you like. I prefer CD-R since it's life expectancy is 10 times that of tape and since every PC I mannage has the CD drive to read the archives. Splitting the files across multiple CD-R media is not a lot of fun, but some backup commands allow spanning. I REALLY hate tape. (Can you tell?)
cp used to be the absolute worst and slowest file copy command under Unix, however it's not the same animal in Linux, and now has some good command-line options that make it an strong choice for cloning file systems complete with permissions and links. Look at cp's "-dfpRuv" switches as a good example of the options you'd want to use to produce a great backup.
A cron job starts my script - called "bkup.cmd" and includes code for the following tasks...
1) Create a log file to monitor the process. Use a unique name generated with the "date" command.
2) Verify the backup media is mounted. Do this with the "test" command. If not, post a message to the log file and using the "mail" command, e-mail the log to the backup administrator and then exit, otherwise continue.
3) One at a time, stop or pause any processes that might have a data base open, copy the data out using cp, dump or an appropriate SQL command and then resume/restart each process. This step lets you have clones of your in-process files that would otherwise be locked and therefore skipped.
You pretty much have to custom design your script file here. If you have no special data bases that would be open exclusively, just skip this step.
4) Mirror your desired folders using the cp, copydir or mirrordir command, one folder at a time. Set the --verbose switch if you like, and redirect (append) the status out to the log file.
5) Copy the now completed log file to the backup media. (Any earlier copy was incomplete.) Using the "mail" command, e-mail the log to the backup administrator.
That's it. Sorry I don't have a decent Linux based script for you at this time, but if I create one soon, I'll e-mail you a copy.
I'd also take a long look at the various backup programs mentioned in this thread, since some may have options to perform file-to-file copies discussed here.
I prefer non-proprietary file formats as I mentioned before. Backup files should just be files.
There's nothing worse than not being able to read the contents of a backup media without first having to install special software or hardware. And since nearly all PCs now have USB ports of one kind or another, you can always read these external drives on any workstation.
USB 2.0 drives can be read using either USB 1 or 2 connections. Mind you, they're 12 times slower on USB 1, but you can still access them at a usable rate. The only requirement is...the OS has to be able to see the partition and file system.
To obtain the best pricing for hardware, I simply use Google.com and Pricewatch.com to research out vendors.
I purchase IBM and Seagate "top-of-the-line" IDE drives with 3-year warranties, paying (as of October 1,2002) $103 for an 80 gig and $153 for a 120 gig from Googlegear.com.
You need to be careful about warranties. Most drive manufacturers are reducing them to one year for much of their line. Some will sell you extended warranties on their web sites. The top-of-the-line still has 3-year warranties as of this writing.
I buy something called an ME-320 USB 2.0 external drive housing (I don't know the manufacturer) from audioexchange.com for $43 each, and a generic PCI USB 2.0 controller for $14 each.
I hope all this helps.
dave...
21 Replies
6109 Views