Topic: smeserver-rdiff-backup

[dmay]

Gordon Rowell has an excellent contrib and build of rdiff-backup available here:

http://www.gormand.com.au/smeserver/WIP/smeserver-rdiff-backup
rdiff-backup-1.1.5-1sme01.i386.rpm
smeserver-rdiff-backup-0.0.1-18.noarch.rpm

I spoke with Gordon about adding a server-manager panel to his contrib in support of backup to local disk. I added a panel and then dived in and made some significant code changes as well. My revised build will be available as soon as mirrors sync here:

http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/testing
smeserver-rdiff-backup-0.0.1-18dmay.noarch.rpm

If you wish to try and comment on my highly experimental changes, please do so at your own risk!

To explain what my build is all about the panel states:

This backup method uses a software package called rdiff-backup to create a copy of your server configuration and user data files, and copies it to an attached local hard drive every night. The initial backup will match your current data size. Subsequent backups will synchronize your data and only copy changes which will make your actual backup time very efficient. In addition incremental changes are stored so you can recover old data overwritten some time ago. The idea is to combine the best features of a mirror and an incremental backup. This method allows you to restore your configuration and user data files easily using command-line tools.

Darrell

[psoren]

Hi Darrel,

This looks interesting.
I am for the moment using rsync on a USB drive and a script that excludes some (big) files that don't need to be backed up.

Are you planing on making a panel or so to be able to exclude files, or can it be done in command line?

Per

[dmay]

Are you planing on making a panel or so to be able to exclude files, or can it be done in command line?

No plans, however I'm listening...

Darrell

[psoren]

No plans, however I'm listening...

Darrell

OK, I really like dar2 because it's easy to backup exactly what you want.
I really like rdiff because you can go back in time and it's incremental, it's also easy to just pick one file if needed.

A combination of the two must be ideal, (in my needs...), but i also see the advance in having dar2 as a replacement for the SME backup system as it is. Great work... I wish i could do stuff like that.

Per

[dmay]

Give release 19 a try. Same highly experimental, do so at your own risk warning!

# rpm -qp --changelog smeserver-rdiff-backup-0.0.1-19dmay.noarch.rpm
* Fri Oct 27 2006 Darrell May <dmay@myezserver.com> 0.0.1-19dmay
- added Include/Exclude db entries
- updated panel, templates, do_rdiff-backup to support above
- smeserver-rdiff-backup-0.0.1-19dmay.patch

Darrell

[jpl]

Darrell,

what's advantage of rdiff over dar for incremental backups ?

[psoren]

Give release 19 a try. Same highly experimental, do so at your own risk warning!

Darrel,

It's not in your download area.

Per

[dmay]

Darrell, what's advantage of rdiff over dar for incremental backups ?

rdiff-backup creates a current at time of backup snapshot in standard file system format. Restores are as easy as copying back whatever you want to restore using standard command line tools. Also due to the standard file system format the available restore from disk action works to perform a full restore. Incrementals allow you to go backward from the current snapshot. There are no backup sets to create or administer. You always have a current full backup in standard format and incrementals going back as long as you have set the duration for.

With dar it works in the opposite direction. You first create a full backup and then create incrementals that go forward from your last full backup snapshot. You build up a backup set then save the entire set and repeat the process to build a new backup set. This is a tedious process to administer and support. In addition the process provides a high potential for failure. Any corruption in the full backup file in a backup set may render that set broken.

Having said this rdiff-backup does not work going to a Windows workstation as all ownership and file rights would be lost. This is why I see a fit for both dar and rdiff-backup.

Darrell

[dtech]

Darrell;

Thanks again for your work, this contrib and your dar contrib address one of the biggest issues I have with SME, backup and easy restore.

I see a combination of your dar contrib and this rdiff contrib as the all around solution to backup, is that your vision?

-P

[dmay]

It's not in your download area.

I uploaded rpms into my contribs ftp area yesterday. They are definitely there however it looks like contribs.org syncing to mirrors isn't working.  

Darrell

[dmay]

I see a combination of your dar contrib and this rdiff contrib as the all around solution to backup, is that your vision?

I expect most people will use one or the other based on which one meets their needs and hardware environment best. However having said this I'm sure someone will have their own personal reasons for using both at the same time. Which is entirely possible.  

Darrell

[jpl]

With dar it works in the opposite direction. You first create a full backup and then create incrementals that go forward from your last full backup snapshot. ...
 In addition the process provides a high potential for failure.

Could you explain what you mean ?

Any corruption in the full backup file in a backup set may render that set broken.

Not so trivial. There are two possibilities for a full backup to be corrupt :
1. failure during backup : dar informs you about it if you want. So there is a simple way - like in e-smith-backup with dar - to do a full backup in place of incremental backup on the next day. And you have at one's disposal preceding good full or/and incremental backups (I speek about e-smith-backup with dar), whereas rdiff seems to leave you without full backup until next day.
2. failure after good backup : it occurs if you have a problem on the backup support. I think there is no real difference between rdiff and dar, although dar has a possibility to use parchive to recover errors and dar-cp (copy a file with io errors) to read corrupt backup file.

[psoren]

It's not in your download area.

I uploaded rpms into my contribs ftp area yesterday. They are definitely there however it looks like contribs.org syncing to mirrors isn't working.  

Darrell

Darrel,

It's still not there... Could i ask you to mail it to me?

per at per-s dot dk

Per

[dmay]

It's not in your download area.

I uploaded rpms into my contribs ftp area yesterday. They are definitely there however it looks like contribs.org syncing to mirrors isn't working.  

Darrell

Darrel,

It's still not there... Could i ask you to mail it to me?

per at per-s dot dk

Per

I did notify staff@contribs.org. However until syncing is fixed I've put copies here:

http://myezserver.com/smeserver/7.x/testing/

Darrell

[psoren]

Darrel,

My USB drive does not auto mount, but i can mount i manually with:

# mount -t ext3 /dev/sda1 /media/usbdisk

How ever, when rdiff starts, it somehow unmounts it again and the backup never starts.

It would also be nice to have a "backup now" option in the panel.

Have you maybe thought about backup over internet with rdiff. That is where it gets VERY interesting since it's incremental.

Per