Koozali.org formerly Contribs.org

[UPDATE 7.2] e-smith-backup with dar

Offline jpl

  • *
  • 112
[UPDATE 7.2] e-smith-backup with dar
« on: July 25, 2007, 03:56:38 PM »
What is e-smith-backup with dar ?

e-smith-backup with dar is an upgrade of current e-smith-backup sme core rpm which provides panels for daily full or incremental backup on LAN workstation (via nfs, smbfs or cifs) or local usb disk, and full or selective restore  with use of dar program.

What provides e-smith-backup with dar ?

e-smith-backup with dar completes core e-smith-backup with four panels :
    - configure daily full or incremental backup to smbfs/cifs/nfs share on a LAN workstation or on local usb disk
    - verify backup on LAN workstation or local usb disk
    - restore full backup from LAN workstation or local usb disk
    - selective restore of files or directories from LAN workstation or local usb disk

New backup options are provided with additional perl functions included in backup panel script, a new perl backup script /etc/e-smith/events/actions/workstation-backup-dar, and a new backupwk record in configuration db. All are written with the same logic as present backup scripts.

What are specific problems of SME Server LAN backups managed by e-smith with dar ?

With experience of more than four years backuping sme servers with dar on LAN workstations (in production environments) the most important of encountered problems is that, when the backup session is started, you cannot rely upon a perfect working of the LAN or the distant workstation.
Inevitably, one day the LAN is too busy and/or the connection to the distant share hacks. After the failure sme may keep zombies backup or mount processes, and with some contribs you can be left with incomplete or buggy backup until hopped next good daily backup. These problems are problems of real world with LAN, that backup scripts must take into account.
Another common problem with backup through a LAN is that your lan disponibility for backup flow is not 24/24, but more probably something like "only from 20h to 6h, except on sunday where nobody works on the LAN..." So in case of large volume of data to backup, beeing able to backup all the datas during the week-end and doing only incremental backup on the week days really makes sense.
All these problems are considered with e-smith-backup with dar.
Scripts verify the absence of old share mounts, and alert admin about such situation. Test is made that the distant  backup share is available and writable. Of course with daily backup a maximum backup session time of only 24 h is available, but if this limit occurs e-smith-backup with dar cleanly stops the backup session  (maybe all is ok and the backup has not enough time to complete), the lan share is unmounted, and daily process can restart anyway. If you have enabled incremental backup and the last backup (full or incremental) was not completed because of a timeout, the backup can then safely restart and complete in an incremental way.
The admin can also set a timeout limit which keeps the session inside a specified number of hours, excluding traffic of LAN backup sessions when users work on the LAN.
Admin user can also set backup configuration so that this time limit occurs only on incremental backup sets. Then if you have a lot of data and your full backup time exceeds time of LAN inactivity, you can always backup all your data each end of week and preserve LAN bandwith for users. If you  program a full backup on saturday night, you get 24 hours to do full backup and even if the full backup time exceeds 24 hours, the first incremental backup on sunday night will backup remaining data. You begin your week with full data saved, and incremental backup session time is limited to 6 or 8 hours during week days.
Full backup sessions can also be allowed on every day, or limited to a specific day of week.

Are other new functions provided ?

Yes. Aside use of session timeout, the first is incremental backup.
This means that you can backup and restore data for the period of time you want : one day, three days, one week, one month, 100 days... and restore your system at any state it was during this period of time. This probably has no utility to do full restore of the system as it was one month ago, but restoring a file lost by a user two or three weeks ago can be useful. And restoring a safe system more than one day old can be needed.

The second function is keeping more than one set of backup (a set is full backup data and all datas of next daily incremental backups) with automatic rotation. e.g. you can do only nightly full backups but keep three sets of backup for security reason  (as being able to restore the system as it was 72h ago).

The third function is selective restore of any backuped file or directory, exactly as it was for any of the backuped days in your sets. Not only you can restore a lost file at his last state, but also say : make restore of the most recent version of the file before this given date... Selective restore is not an easy thing to manage by hand, and providing this in a simple way with panels is useful. Dar permits to manage selective restore and e-smith-backup with dar panels tries to keep this function as simple as possible to use.

Where to find e-smith-backup with dar ?

Presently beta version of e-smith with dar can be uploaded at http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-16eos1.el4.sme.noarch.rpm

SRPMS available at http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-16eos1.el4.sme.src.rpm
Dar rpm is available at http://beta.free-eos.org/sme/RPMS/dar-2.3.2-1.i386.rpm or mirrored at http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/dar-2.3.2-1.i386.rpm

I read about problems about smbfs or cifs support with sme, and nfs packages are not on my sme...

Before sme 7.2, smb and kernel versions of sme have bugs in smb mount and cifs support.
For smb mount bug, e-smith-backup with dar includes necessary work-around.
For cifs support, corrected 2.6.9-55 kernel is packaged with sme 7.2.
For nfs support, e-smith-backup doesn't need install of any nfs package on the sme server (not even portmap).

Preventing automatic upgrades from sme yum updates...
You can prevent sme yum upgrade system from upgrading e-smith-backup with dar with new e-smith-backup core version.
Just enter as root :
Code: [Select]

# echo e-smith-backup > /etc/e-smith/db/yum_repositories/defaults/smeupdates/Exclude
# /etc/e-smith/events/actions/initialize-default-databases
# /sbin/e-smith/expand-template /etc/yum.conf

Manual yum upgrade is always available.

Comments are welcomed.

/edit/
sme 7.2 version
Jul 26 : corrected rpms release and link

Offline Tib

  • *
  • 571
    • http://www.tibors.net
[UPDATE 7.2] e-smith-backup with dar
« Reply #1 on: July 26, 2007, 05:20:24 AM »
I was reluctant to update my server because prior breakages for this great contrib ... Now I can update all servers.

Great Job jpl ... I love your contrib.

Regards,

Tib

Offline jpl

  • *
  • 112
[UPDATE 7.2] e-smith-backup with dar
« Reply #2 on: July 26, 2007, 10:22:24 AM »
Seems there where a problem while generating rpm.
Updated version available in a few moments...

Offline jpl

  • *
  • 112
[UPDATE 7.2] e-smith-backup with dar
« Reply #3 on: July 26, 2007, 11:30:51 AM »
Corrected rpms are now available for download (release 16eos1).
Some scripts were no more included in first 16eos release, so only upgrades from e-smith-backup with dar older releases where OK, not upgrades from core e-smith-backup.
Should be now corrected. Apologies...

[UPDATE 7.2] e-smith-backup with dar
« Reply #4 on: July 26, 2007, 04:42:02 PM »
I think you need to update your dar links... I found one at http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/dar-2.3.2-1.i386.rpm

Also, are there any docs pages or pages more clearly explaining the options on the workstation backup page. I'm not sure I understand exactly what the # of sets and dailys in a set mean.

I *think* that by setting it to 4 sets with 7 dailys per set with Friday set as the only full day should give me 4 sets of 7 files with one large file per set bing the Friday full and the rest being daily incrementals.
Elliott

Offline jpl

  • *
  • 112
[UPDATE 7.2] e-smith-backup with dar
« Reply #5 on: July 26, 2007, 05:55:20 PM »
Thanks for the dar link.

In fact, Darrel originally  made a copy of my rpm, and  I have not seen that he provides now an updated version  :)

You are right about sets.

Sorry for the doc, it has yet to be made on the wiki. But panels are almost (almost only I know) self explainatory.

Offline jpl

  • *
  • 112
[UPDATE 7.2] e-smith-backup with dar
« Reply #6 on: July 26, 2007, 06:42:12 PM »
Quote from: "jpl"
Thanks for the dar link.

In fact, Darrel originally  made a copy of my rpm, and  I have not seen that he provides now an updated version  :)


Sorry again, it's not a new rpm, it's the one I provided to Darrel.

Just my links that are wrong, as you say. I correct them.

[UPDATE 7.2] e-smith-backup with dar
« Reply #7 on: July 27, 2007, 03:34:06 PM »
Ok... so I got this:

Quote
==================================
DAILY BACKUP TO WORKSTATION REPORT
==================================
Backup started at Thu Jul 26 20:15:02 2007
Backup of mysql databases has been done.
Mounting backup shared directory netshare/backup
*** No backup allowed or error during backup ***
Error while mounting netshare:backup :
mount error: could not find target server. TCP name netshare/backup not found  rc = 134526947
No ip address specified and hostname not found


My settings are:
Quote
Backup is enabled.
Backup is made on LAN workstation netshare via cifs
Destination backup share folder is backup
Login is backup
Password is ********
Number of rotating backup sets is 4
Number of daily backups contained in each set is 7
Compression level (0-7) of backup is 4
Daily backup occurs at 20:15
Each daily backup session is cleanly timed out after 10h except full backups which are cleanly timed out after 24 hours
Full backup session (new backup sets) is allowed only on Friday


This SME Server is 10.11.12.14 on my LAN and "netshare" is 10.11.12.3 on the same segment. I changed the workstation name to 10.11.12.3 for tonights attempt at a backup. Can anyone suggest anything else or possibly a way to test it now without actually starting a backup since I've got users hitting it?

Thanks,
E
Elliott

Offline jpl

  • *
  • 112
[UPDATE 7.2] e-smith-backup with dar
« Reply #8 on: July 27, 2007, 10:50:04 PM »
Not at home for eigth days, and don't have acess to server.

You can look at /etc/e-smith/events/actions/workstation-backup-dar script, find the command which connect your cifs target, and try to debug it directly as a shell command.

I think it is :

/bin/mount -t cifs $smbhost:$smbshare $mntdir -o user=$login,pass=$password

in your case :

/bin/mount -t cifs netshare:backup /mnt/smb -o user=backup,pass=backup

You can also look at your /var/log/messages log for some more information about your cifs mount.

Offline gerd

  • ***
  • 107
[UPDATE 7.2] e-smith-backup with dar
« Reply #9 on: August 03, 2007, 04:43:55 PM »
..I must be blind today...
I have made a fresh install of the SME7.2 and installed the "dar-2.3.2-1.i386.rpm" contrib.
But is my understanding correct that for this contrib there is no user-interface displayed in the web-panel??

[UPDATE 7.2] e-smith-backup with dar
« Reply #10 on: August 03, 2007, 04:48:41 PM »
Quote from: "gerd"
But is my understanding correct that for this contrib there is no user-interface displayed in the web-panel??


It adds more options to the standard backup panel.
Elliott

Offline gerd

  • ***
  • 107
[UPDATE 7.2] e-smith-backup with dar
« Reply #11 on: August 03, 2007, 06:10:42 PM »
Elliot,
a stupid question: which contribs have to installed in order to see
the see the additional these options??? For me here nothing has changed, strange ?? (& I used signal-event post-upgrade // signal-event reboot)

thanx

gerd

Offline gerd

  • ***
  • 107
[UPDATE 7.2] e-smith-backup with dar
« Reply #12 on: August 03, 2007, 06:10:59 PM »
Elliot,
a stupid question: which contribs have to installed in order to see
the see the additional these options??? For me here nothing has changed, strange ?? (& I used signal-event post-upgrade // signal-event reboot)

thanx

gerd

[UPDATE 7.2] e-smith-backup with dar
« Reply #13 on: August 03, 2007, 06:31:12 PM »
Follow the directions at the start of this post. That's it.

Then go to the "Backup or restore" tab on the SME Server Console. That's it. The new options are in the drop down list on the bottom.
Elliott

Offline gerd

  • ***
  • 107
[UPDATE 7.2] e-smith-backup with dar
« Reply #14 on: August 04, 2007, 06:11:37 PM »
...yep, indeed - I was blind....it works perfect.

thanx

gerd