Koozali.org: home of the SME Server

[beta] e-smith-backup with dar

Offline jpl

  • *
  • 112
  • +0/-0
[beta] e-smith-backup with dar
« on: October 13, 2006, 06:38:20 PM »
What is e-smith-backup with dar ?

e-smith-backup with dar is an upgrade of current e-smith-backup sme core rpm based upon use of dar program.

What provides e-smith-backup with dar ?

e-smith-backup with dar completes core e-smith-backup with four panels :
    - configuring daily backup to smbfs/cifs/nfs share on a LAN workstation or on local usb disk
    - verifying backup on lan or local usb disk
    - restoring full backup from lan or local usb disk
    - selective restoring of files or directories from lan or local usb disk


New backup options are provided with additional perl functions included in backup panel script, a new perl backup script /etc/e-smith/events/actions/workstation-backup-dar, and a new backupwk record in configuration db. All are written with the same logic as present backup scripts, for seamlessly integration.

What are specific problems with LAN backups ?

With more than one year use of e-smith backuping with dar on LAN workstations (in production environments) the most important of encountered problems is that, when the backup session is started, you cannot rely upon a perfect working of the LAN or the distant workstation.
Inevitably, one day the LAN is too busy and/or the connection to the distant share hacks. The result could be zombie backup or mount process, or even backups that do not complete before next daily backup session. These problems are problems of real world with LAN, that backup scripts must take into account.
Another common problem with backup through a LAN is that your lan disponibility for backup flow is not 24/24, but more probably something like "only from 20h to 6h, except on sunday where nobody works on the LAN..." So in case of large backup volume of datas, beeing able to backup all the datas during the week-end and doing only incremental backup on the week days really makes sense.
All these problems are considered with e-smith-backup with dar.
Scripts aim to verify absence of old zombie mounts of share, and alerts admin about such situation. No other assumption than "the distant  backup share is available and writable" is made (and verifications are made for each access). With daily backup only a maximum backup time of 24 h is possible, and if this limit occurs the script cleanly stop the backup session  (maybe the backup is working and in progress but has not enough time to complete), the lan share is unmounted, and daily process can restart anyway. If a precedent full backup was not completed because of a timeout (and if incremental backup is enabled) the backup can then safely restart and complete in an incremental way.
To limit when backup session occurs on the LAN, the admin user can set a timeout which keeps the session into a specified number of hours, allowing LAN backup sessions at periods where users don't work on the LAN. Admin user can also query for this time limit to occur only on incremental backup sessions. This permits to use, for instance, 24h on sunday to do full backup, and limit incremental backup sessions to 6 or 8 hours during week days. Full backup sessions can also be allowed on every day, or limited to a specific day of week.

Are other new functions provided ?

Yes. Aside session timeout use, the first is incremental backup.
This means that you can backup and restore datas for periods you want : one day, three days, one week, one month, 100 days... and restore your system at any state it was during this period of time. This probably has no utility to do full restore of the system as it was one month ago, but restoring a file lost by a user  two or three weeks ago can be useful. And restoring a safe system more than one day old can be needed.

The second function is keeping more than one set of backup (a set is a full backup and all next daily incremental backups before a new full backup) with automatic rotation. In the simplest case you can, for example, do only nightly full backups but keep three sets of backup for security reason  (as being able to restore the system as it was 72h ago).

The third function is selective restore of any backuped file or directory, exactly as it was for any of the backuped days. Not only you can restore a lost file at his last state, but also say : make restore of the most recent version of the file before this given date... Selective restore is not an easy thing to manage and to provide in a simple way with panels. Dar permits to manage it. e-smith-backup with dar panels try to keep this function as simple as possible to use, without reducing offered capabilities.

Where to find e-smith-backup with dar ?

Presently beta version of e-smith with dar can be uploaded at http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-14jplb01.el4.sme.noarch.rpm

SRPMS available at http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-14jplb01.el4.sme.src.rpm
Dar rpm is available at http://beta.free-eos.org/sme/RPMS/dar-2.3.1-1.i386.rpm or http://mirror.contribs.org/smeserver/contribs//dmay/smeserver/7.x/testing/dar-2.3.1-1.i386.rpm

I read problems about smbfs or cifs support with sme, and nfs packages are not on my sme...

Yes, present sme7 smb and kernel versions of have bugs for smb and cifs support. For smb mount bug, e-smith-backup with dar includes necessary work-around. For cifs, if the problem occurs, update to SME 2.6.9-55 kernel (this is a bug in older SME kernel). For nfs support, e-smith-backup doesn't need install of any nfs package on the sme server (not even portmap).

Comments are welcomed.

/edit/
sme 7.1 version
sme 7.1.1 version
sme 7.1.3 version
7 Jun 2007 : updated version

Offline Tib

  • *
  • 571
  • +0/-0
    • http://www.tibors.net
[beta] e-smith-backup with dar
« Reply #1 on: October 16, 2006, 02:39:33 AM »
Hello jpl,

Thanks a lot for this contrib ... I have been looking for these features for a long time.

Loaded it on the test server and will start testing from today.

Will report my findings as the come.

Regards,

Tib

Offline Tib

  • *
  • 571
  • +0/-0
    • http://www.tibors.net
[beta] e-smith-backup with dar
« Reply #2 on: October 18, 2006, 03:31:14 AM »
ok

Backup of just over 9 gig went fine .... one strange thing that is happeneing though is.

Backup files in \\Backup-server\Mailserver\tmp_dir\server5.cooltemp.com.au get created no probs ... full-20061018.1.dar through to full-20061018.13.dar.

Once job is finished it is supposed to transfere the files to \\Backup-server\Mailserver\server5.cooltemp.com.au\set0.

Well this does not happen ... but what does happen is the file full-20061018.1.dar dissapears and the rest are left there .... full-20061018.1.dar does not appear in \\Backup-server\Mailserver\server5.cooltemp.com.au\set0.

There is plenty of room on the Backup server as well as on the test server so lack of hdd space should not be the case here.

I will load this on another machine and test there as well as I had issues with the backup contribs on this machine ... that will tell me if it's a corrupt  Test server ... although I uninstalled the backup contribs and re-loaded everything as well as the base backup.

When I first tested this contrib in the early stages all the transfers where going fine .... this whent wierd after my problems with the backup contrib issues.

I will report on the bug tracker as well after I have confirmed if it's this machine or not.

Regards,

Tib.

Offline Tib

  • *
  • 571
  • +0/-0
    • http://www.tibors.net
[beta] e-smith-backup with dar
« Reply #3 on: October 18, 2006, 11:53:04 PM »
jpl

I have confirmed that the 2nd server is having the same problem .... I will follow this up in the bug tracker.

Regards,

Tib

Offline jpl

  • *
  • 112
  • +0/-0
[beta] e-smith-backup with dar
« Reply #4 on: October 23, 2006, 11:03:59 AM »
Corrected new beta version version available at http://bugs.contribs.org/attachment.cgi?id=591

Offline Tib

  • *
  • 571
  • +0/-0
    • http://www.tibors.net
[beta] e-smith-backup with dar
« Reply #5 on: October 24, 2006, 12:55:24 AM »
jpl

Have confirmed ... file transfers work as expected now ...  :D

This is a great contrib.

Will do further testing as incremental backups should occure properly now.

Regards,

Tib

Offline Tillebeck

  • ***
  • 76
  • +0/-0
sounds great!
« Reply #6 on: October 24, 2006, 03:06:29 PM »
This sounds great!
I am just about to set up new backup procedures. So I will join in on the testing and try to post any findings.

BR. Anders

Offline Tillebeck

  • ***
  • 76
  • +0/-0
howto configure workstataion backup
« Reply #7 on: October 24, 2006, 03:47:15 PM »
Hi. I ran into trouble when trying to setup the workstation backup

Can you please provide a short howto for setting up workstation backup. In my case I use an ibay on another sme server.

Type: smbfs (is this correct when destination is another sme server?)
Path on LAN is: \\192.168.2.51\snapshot\files
User: [username]
Password: [pasword]

I get this error:
Unable to mount workstation shared folder 3576: session setup failed: ERRDOS - ERRnoaccess (Access denied.) SMB connection failed

when trying to use:
workstation IP: 192.168.2.51
Backup share: snapshot/files

Offline cactus

  • *
  • 4,880
  • +3/-0
    • http://www.snetram.nl
[beta] e-smith-backup with dar
« Reply #8 on: October 24, 2006, 07:15:37 PM »
There is a little glitch in the panel, the second page specifies compression level to range fropm 0 to 9 (when specifying workstation backup to USB), but if I enter 9 I get an error that 7 is the maximum allowed value.
Be careful whose advice you buy, but be patient with those who supply it. Advice is a form of nostalgia, dispensing it is a way of fishing the past from the disposal, wiping it off, painting over the ugly parts and recycling it for more than its worth ~ Baz Luhrmann - Everybody's Free (To Wear Sunscreen)

Offline Tib

  • *
  • 571
  • +0/-0
    • http://www.tibors.net
[beta] e-smith-backup with dar
« Reply #9 on: October 25, 2006, 02:54:30 AM »
ok

The first incremental has run today on both servers and all looks good ... all file transfers are going well.

Next week I will trash the test server and try to re-build it and see what happens.

Tillebeck

Are you sure you have the Ibay setup correctly for user rights.

A little more info on how you setup your Ibay could be good as well.

Regards,

Tib

Offline Tillebeck

  • ***
  • 76
  • +0/-0
It is working now
« Reply #10 on: October 25, 2006, 12:33:40 PM »
I am backing up to another sme server.

This did not work:
I tried to backup to the files folder in an ibay (servername/ibay/files)
First try:
 - server -> servername
 - folder -> ibay/files
Second try:
 - server -> servername/ibay
 - folder -> files

This did work:
I created a backup user and did the backup to the users folder. This way there is no need for more than one folder level
 - server -> servername
 - folder -> backupuser

Obviously it must be due to me trying to add an extra level in the folder path. So now it seems to be working just fine.

Follow-up question. I do not know the difference on smbfs, cifs and nfs. I use smbfs and that seems to work. Would one of the other be better from sme server to sme server?

BR. and thanks for the feedback
Anders

Offline jpl

  • *
  • 112
  • +0/-0
Re: It is working now
« Reply #11 on: October 25, 2006, 03:05:29 PM »
Quote from: "Tillebeck"
I am backing up to another sme server.

This did not work:
I tried to backup to the files folder in an ibay (servername/ibay/files)
First try:
 - server -> servername
 - folder -> ibay/files
Second try:
 - server -> servername/ibay
 - folder -> files


Quite normal. With SME  ibay/files folder is shared is shared as ibay smb ressource, not ibay/files. You must use folder ibay.

Quote

This did work:
I created a backup user and did the backup to the users folder. This way there is no need for more than one folder level
 - server -> servername
 - folder -> backupuser


yes, it's an option.

Quote

Obviously it must be due to me trying to add an extra level in the folder path. So now it seems to be working just fine.


It was due to the fact that ibay share points to ibay/files and ibay/files share points to ibay/file/files folder.

Quote

Follow-up question. I do not know the difference on smbfs, cifs and nfs. I use smbfs and that seems to work. Would one of the other be better from sme server to sme server?


To be simple : cifs is the name of new smbfs versions. If you want to backup to XP or 2003 server, using cifs is the way.

nfs is another problem : use nfs if you have a nfs server and if you want better throughput during the backup.

Offline ronnie_dk

  • 19
  • +0/-0
Kernel requirement for this Contrib?
« Reply #12 on: November 03, 2006, 02:55:04 PM »
Hi there.
I´m having issues regarding CIFS mount point on my Win2003 server.
JPL wrote Wait for the next kernel release  . OK but i think there is a newer kernel than the one at my production server right now:
"2.6.9-34.0.2"
I have seen topic saying "in the smeupdates-testing repo is 2.6.9-42.0.2.EL" does this kernel suit my needs to make CIFS work for this Contrib?
...

Offline dmay

  • *
  • 450
  • +0/-0
    • http://myezserver.com
Re: Kernel requirement for this Contrib?
« Reply #13 on: November 03, 2006, 03:27:45 PM »
Quote from: "ronnie_dk"
Hi there.
I´m having issues regarding CIFS mount point on my Win2003 server.

What are your issues?

Darrell

Offline ronnie_dk

  • 19
  • +0/-0
[beta] e-smith-backup with dar
« Reply #14 on: November 03, 2006, 06:44:01 PM »
Meanwhile I have updated the kernel. I will try to cut+paste the "CIFS mount denied message" if backup fails again tonight..
Best Regards Ronnie
...