Koozali.org: home of the SME Server
Obsolete Releases => SME 7.x Contribs => Topic started by: jpl on October 13, 2006, 06:38:20 PM
-
What is e-smith-backup with dar ?
e-smith-backup with dar is an upgrade of current e-smith-backup sme core rpm based upon use of dar program.
What provides e-smith-backup with dar ?
e-smith-backup with dar completes core e-smith-backup with four panels :
- configuring daily backup to smbfs/cifs/nfs share on a LAN workstation or on local usb disk
- verifying backup on lan or local usb disk
- restoring full backup from lan or local usb disk
- selective restoring of files or directories from lan or local usb disk
New backup options are provided with additional perl functions included in backup panel script, a new perl backup script /etc/e-smith/events/actions/workstation-backup-dar, and a new backupwk record in configuration db. All are written with the same logic as present backup scripts, for seamlessly integration.
What are specific problems with LAN backups ?
With more than one year use of e-smith backuping with dar on LAN workstations (in production environments) the most important of encountered problems is that, when the backup session is started, you cannot rely upon a perfect working of the LAN or the distant workstation.
Inevitably, one day the LAN is too busy and/or the connection to the distant share hacks. The result could be zombie backup or mount process, or even backups that do not complete before next daily backup session. These problems are problems of real world with LAN, that backup scripts must take into account.
Another common problem with backup through a LAN is that your lan disponibility for backup flow is not 24/24, but more probably something like "only from 20h to 6h, except on sunday where nobody works on the LAN..." So in case of large backup volume of datas, beeing able to backup all the datas during the week-end and doing only incremental backup on the week days really makes sense.
All these problems are considered with e-smith-backup with dar.
Scripts aim to verify absence of old zombie mounts of share, and alerts admin about such situation. No other assumption than "the distant backup share is available and writable" is made (and verifications are made for each access). With daily backup only a maximum backup time of 24 h is possible, and if this limit occurs the script cleanly stop the backup session (maybe the backup is working and in progress but has not enough time to complete), the lan share is unmounted, and daily process can restart anyway. If a precedent full backup was not completed because of a timeout (and if incremental backup is enabled) the backup can then safely restart and complete in an incremental way.
To limit when backup session occurs on the LAN, the admin user can set a timeout which keeps the session into a specified number of hours, allowing LAN backup sessions at periods where users don't work on the LAN. Admin user can also query for this time limit to occur only on incremental backup sessions. This permits to use, for instance, 24h on sunday to do full backup, and limit incremental backup sessions to 6 or 8 hours during week days. Full backup sessions can also be allowed on every day, or limited to a specific day of week.
Are other new functions provided ?
Yes. Aside session timeout use, the first is incremental backup.
This means that you can backup and restore datas for periods you want : one day, three days, one week, one month, 100 days... and restore your system at any state it was during this period of time. This probably has no utility to do full restore of the system as it was one month ago, but restoring a file lost by a user two or three weeks ago can be useful. And restoring a safe system more than one day old can be needed.
The second function is keeping more than one set of backup (a set is a full backup and all next daily incremental backups before a new full backup) with automatic rotation. In the simplest case you can, for example, do only nightly full backups but keep three sets of backup for security reason (as being able to restore the system as it was 72h ago).
The third function is selective restore of any backuped file or directory, exactly as it was for any of the backuped days. Not only you can restore a lost file at his last state, but also say : make restore of the most recent version of the file before this given date... Selective restore is not an easy thing to manage and to provide in a simple way with panels. Dar permits to manage it. e-smith-backup with dar panels try to keep this function as simple as possible to use, without reducing offered capabilities.
Where to find e-smith-backup with dar ?
Presently beta version of e-smith with dar can be uploaded at http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-14jplb01.el4.sme.noarch.rpm
SRPMS available at http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-14jplb01.el4.sme.src.rpm
Dar rpm is available at http://beta.free-eos.org/sme/RPMS/dar-2.3.1-1.i386.rpm or http://mirror.contribs.org/smeserver/contribs//dmay/smeserver/7.x/testing/dar-2.3.1-1.i386.rpm
I read problems about smbfs or cifs support with sme, and nfs packages are not on my sme...
Yes, present sme7 smb and kernel versions of have bugs for smb and cifs support. For smb mount bug, e-smith-backup with dar includes necessary work-around. For cifs, if the problem occurs, update to SME 2.6.9-55 kernel (this is a bug in older SME kernel). For nfs support, e-smith-backup doesn't need install of any nfs package on the sme server (not even portmap).
Comments are welcomed.
/edit/
sme 7.1 version
sme 7.1.1 version
sme 7.1.3 version
7 Jun 2007 : updated version
-
Hello jpl,
Thanks a lot for this contrib ... I have been looking for these features for a long time.
Loaded it on the test server and will start testing from today.
Will report my findings as the come.
Regards,
Tib
-
ok
Backup of just over 9 gig went fine .... one strange thing that is happeneing though is.
Backup files in \\Backup-server\Mailserver\tmp_dir\server5.cooltemp.com.au get created no probs ... full-20061018.1.dar through to full-20061018.13.dar.
Once job is finished it is supposed to transfere the files to \\Backup-server\Mailserver\server5.cooltemp.com.au\set0.
Well this does not happen ... but what does happen is the file full-20061018.1.dar dissapears and the rest are left there .... full-20061018.1.dar does not appear in \\Backup-server\Mailserver\server5.cooltemp.com.au\set0.
There is plenty of room on the Backup server as well as on the test server so lack of hdd space should not be the case here.
I will load this on another machine and test there as well as I had issues with the backup contribs on this machine ... that will tell me if it's a corrupt Test server ... although I uninstalled the backup contribs and re-loaded everything as well as the base backup.
When I first tested this contrib in the early stages all the transfers where going fine .... this whent wierd after my problems with the backup contrib issues.
I will report on the bug tracker as well after I have confirmed if it's this machine or not.
Regards,
Tib.
-
jpl
I have confirmed that the 2nd server is having the same problem .... I will follow this up in the bug tracker.
Regards,
Tib
-
Corrected new beta version version available at http://bugs.contribs.org/attachment.cgi?id=591
-
jpl
Have confirmed ... file transfers work as expected now ... :D
This is a great contrib.
Will do further testing as incremental backups should occure properly now.
Regards,
Tib
-
This sounds great!
I am just about to set up new backup procedures. So I will join in on the testing and try to post any findings.
BR. Anders
-
Hi. I ran into trouble when trying to setup the workstation backup
Can you please provide a short howto for setting up workstation backup. In my case I use an ibay on another sme server.
Type: smbfs (is this correct when destination is another sme server?)
Path on LAN is: \\192.168.2.51\snapshot\files
User: [username]
Password: [pasword]
I get this error:
Unable to mount workstation shared folder 3576: session setup failed: ERRDOS - ERRnoaccess (Access denied.) SMB connection failed
when trying to use:
workstation IP: 192.168.2.51
Backup share: snapshot/files
-
There is a little glitch in the panel, the second page specifies compression level to range fropm 0 to 9 (when specifying workstation backup to USB), but if I enter 9 I get an error that 7 is the maximum allowed value.
-
ok
The first incremental has run today on both servers and all looks good ... all file transfers are going well.
Next week I will trash the test server and try to re-build it and see what happens.
Tillebeck
Are you sure you have the Ibay setup correctly for user rights.
A little more info on how you setup your Ibay could be good as well.
Regards,
Tib
-
I am backing up to another sme server.
This did not work:
I tried to backup to the files folder in an ibay (servername/ibay/files)
First try:
- server -> servername
- folder -> ibay/files
Second try:
- server -> servername/ibay
- folder -> files
This did work:
I created a backup user and did the backup to the users folder. This way there is no need for more than one folder level
- server -> servername
- folder -> backupuser
Obviously it must be due to me trying to add an extra level in the folder path. So now it seems to be working just fine.
Follow-up question. I do not know the difference on smbfs, cifs and nfs. I use smbfs and that seems to work. Would one of the other be better from sme server to sme server?
BR. and thanks for the feedback
Anders
-
I am backing up to another sme server.
This did not work:
I tried to backup to the files folder in an ibay (servername/ibay/files)
First try:
- server -> servername
- folder -> ibay/files
Second try:
- server -> servername/ibay
- folder -> files
Quite normal. With SME ibay/files folder is shared is shared as ibay smb ressource, not ibay/files. You must use folder ibay.
This did work:
I created a backup user and did the backup to the users folder. This way there is no need for more than one folder level
- server -> servername
- folder -> backupuser
yes, it's an option.
Obviously it must be due to me trying to add an extra level in the folder path. So now it seems to be working just fine.
It was due to the fact that ibay share points to ibay/files and ibay/files share points to ibay/file/files folder.
Follow-up question. I do not know the difference on smbfs, cifs and nfs. I use smbfs and that seems to work. Would one of the other be better from sme server to sme server?
To be simple : cifs is the name of new smbfs versions. If you want to backup to XP or 2003 server, using cifs is the way.
nfs is another problem : use nfs if you have a nfs server and if you want better throughput during the backup.
-
Hi there.
I´m having issues regarding CIFS mount point on my Win2003 server.
JPL wrote Wait for the next kernel release . OK but i think there is a newer kernel than the one at my production server right now:
"2.6.9-34.0.2"
I have seen topic saying "in the smeupdates-testing repo is 2.6.9-42.0.2.EL" does this kernel suit my needs to make CIFS work for this Contrib?
-
Hi there.
I´m having issues regarding CIFS mount point on my Win2003 server.
What are your issues?
Darrell
-
Meanwhile I have updated the kernel. I will try to cut+paste the "CIFS mount denied message" if backup fails again tonight..
Best Regards Ronnie
-
Hi there.
I have seen topic saying "in the smeupdates-testing repo is 2.6.9-42.0.2.EL" does this kernel suit my needs to make CIFS work for this Contrib?
No it doesn't.
cifs correct patch has been committed by Jason Baron (maintainer of RHEL4 kernel) in 2.6.9-42.10 release of RHEL4 kernel.
All preceding versions of RHEL4 kernel have problem with cifs when listing directories.
For a test server you can try to use lastest dev kernel available at http://people.redhat.com/~jbaron/rhel4/ but for a production server, you are advised to wait next stable CentOS kernel update before using cifs.
-
JPL
Thanks for info...
Ronine
-
Meanwhile I have updated the kernel. I will try to cut+paste the "CIFS mount denied message" if backup fails again tonight..
Best Regards Ronnie
Ronnie,
This code works for me for mounting using cifs to a W2K3 share:
/bin/mount -t cifs -o username=blah,password=blah //ipaddress/share /mnt/smb
Example:
/bin/mount -t cifs -o username=dar2,password=logical //10.1.0.20/dar2 /mnt/smb
See if it works for you. If it does try editing JPL's workstation-backup-dar to match and test. Something like this might work as long as you make sure you input //ipaddress in JPL's panel entry for smbhost:
/bin/mount -t cifs -o username=$login,password=$password //$smbhost/$smbshare $mntdir
Darrell
-
See if it works for you. If it does try editing JPL's workstation-backup-dar to match and test. Something like this might work as long as you make sure you input //ipaddress in JPL's panel entry for smbhost:
/bin/mount -t cifs -o username=$login,password=$password //$smbhost/$smbshare $mntdir
e-smith-backup with dar uses a more recent syntax, but Darrell's proposition should be supported the same way (this is the case for my tests).
Tell if it changes anything. I don't think it will, but I hope to be wrong.
You could also verify this bug related to cifs use with Windows 2003 Server : https://bugzilla.samba.org/show_bug.cgi?id=3850
-
Hi again.. I have managed to log onto the 2003 server now and here it gets funny..
I have 4 new lines in panel "Administration/Backup or restore"
I´ll try to generate a backup job:
It looks like this
NB: I have my 2003 server in hosts file so 192.168.0.2 wich is my 2003 server is named "filserver"
Configure Workstation Backup
This panel displays the present workstation backup configuration. You can change it in this panel and the next one.
Backup is enabled.
Backup is made on LAN workstation filserver via cifs
Destination backup share folder is Serverbackup
Login is username "used local superuser account"
Password is XXXX
Number of rotating backup sets is 1
Number of daily backups contained in each set is 1
Compression level (0-7) of backup is 4
Daily backup occurs at 14:00
Each daily backup session is cleanly timed out after 1h
Full backup sessions (new backup set) are allowed everyday
at 14:00 the following comes up in messages
14:00:01 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: OLD 1162645201=(undefined)
Nov 4 14:00:01 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: NEW 1162645201=backup_record
Nov 4 14:00:01 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: OLD 1162645201=backup_record
Nov 4 14:00:01 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: NEW 1162645201=backup_record|StartEpochTime|1162645201
Nov 4 14:00:01 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: OLD 1162645201=backup_record|StartEpochTime|1162645201
Nov 4 14:00:01 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: NEW 1162645201=backup_record|BackupType|workstation|StartEpochTime|1162645201
Nov 4 14:00:01 router esmith::event[17562]: Processing event: pre-backup
Nov 4 14:00:01 router esmith::event[17562]: Running event handler: /etc/e-smith/events/pre-backup/S10mysql-delete-dumped-tables
Nov 4 14:00:01 router esmith::event[17562]: S10mysql-delete-dumped-tables=action|Event|pre-backup|Action|S10mysql-delete-dumped-tables|Start|1162645201 773364|End|1162645201 778396|Elapsed|0.005032
Nov 4 14:00:01 router esmith::event[17562]: Running event handler: /etc/e-smith/events/pre-backup/S20mysql-dump-tables
Nov 4 14:00:01 router esmith::event[17562]: S20mysql-dump-tables=action|Event|pre-backup|Action|S20mysql-dump-tables|Start|1162645201 779083|End|1162645201 893544|Elapsed|0.114461
Nov 4 14:00:01 router esmith::event[17562]: Running event handler: /etc/e-smith/events/pre-backup/S50rewind-tape
Nov 4 14:00:02 router esmith::event[17562]: S50rewind-tape=action|Event|pre-backup|Action|S50rewind-tape|Start|1162645201 894239|End|1162645202 174943|Elapsed|0.280704
Nov 4 14:00:04 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: OLD 1162645201=backup_record|BackupType|workstation|StartEpochTime|1162645201
Nov 4 14:00:04 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: NEW 1162645201=backup_record|BackupType|workstation|EndEpochTime|1162645204|StartEpochTime|1162645201
Nov 4 14:00:04 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: OLD 1162645201=backup_record|BackupType|workstation|EndEpochTime|1162645204|StartEpochTime|1162645201
Nov 4 14:00:04 router /sbin/e-smith/do_backupwk[17561]: /home/e-smith/db/backups: NEW 1162645201=backup_record|BackupType|workstation|EndEpochTime|1162645204|Result|backup:7424|StartEpochTime|1162645201
But no backupfile is made on 2003 server... hmmmm...
OK another way to make backups is to use the other New button in my SME panel "Backup Disk ARchive"
I have managed to create a job called "stdbackup"
I fired it up by manually clicking "Backup" and the result is here.. fine
Nov 4 14:06:52 router e-smith-bg: Creating /mnt/dar2/stdbackup dir
Nov 4 14:06:52 router e-smith-bg: Connecting /mnt/dar2/stdbackup to //filserver/Serverbackup
Nov 4 14:06:52 router e-smith-bg: Expanding /etc/dar2/stdbackup template
Nov 4 14:06:52 router e-smith-bg: Creating destination /router.linwin.dk/stdbackup dir
Nov 4 14:06:52 router e-smith-bg: Expiring old backups
Nov 4 14:06:52 router e-smith-bg: Executing pre-backup event
Nov 4 14:06:52 router esmith::event[17800]: Processing event: pre-backup
Nov 4 14:06:52 router esmith::event[17800]: Running event handler: /etc/e-smith/events/pre-backup/S10mysql-delete-dumped-tables
Nov 4 14:06:52 router esmith::event[17800]: S10mysql-delete-dumped-tables=action|Event|pre-backup|Action|S10mysql-delete-dumped-tables|Start|1162645612 649623|End|1162645612 654883|Elapsed|0.00526
Nov 4 14:06:52 router esmith::event[17800]: Running event handler: /etc/e-smith/events/pre-backup/S20mysql-dump-tables
Nov 4 14:06:52 router esmith::event[17800]: S20mysql-dump-tables=action|Event|pre-backup|Action|S20mysql-dump-tables|Start|1162645612 655594|End|1162645612 737923|Elapsed|0.082329
Nov 4 14:06:52 router esmith::event[17800]: Running event handler: /etc/e-smith/events/pre-backup/S50rewind-tape
Nov 4 14:06:52 router esmith::event[17800]: S50rewind-tape=action|Event|pre-backup|Action|S50rewind-tape|Start|1162645612 738683|End|1162645612 920405|Elapsed|0.181722
Nov 4 14:06:52 router e-smith-bg: Verbose logging is off
Nov 4 14:06:52 router e-smith-bg: Performing stdbackup backup
Nov 4 14:06:52 router e-smith-bg: No terminal found for user interaction. All questions will be assumed a negative answer (less destructive choice), which most of the time will abort the program.
Nov 4 14:07:28 router e-smith-bg:
Nov 4 14:07:28 router e-smith-bg:
Nov 4 14:07:28 router e-smith-bg: --------------------------------------------
Nov 4 14:07:28 router e-smith-bg: 1770 inode(s) saved
Nov 4 14:07:28 router e-smith-bg: with 0 hard link(s) recorded
Nov 4 14:07:28 router e-smith-bg: 0 inode(s) changed at the moment of the backup
Nov 4 14:07:28 router e-smith-bg: 0 inode(s) not saved (no inode/file change)
Nov 4 14:07:28 router e-smith-bg: 0 inode(s) failed to save (filesystem error)
Nov 4 14:07:28 router e-smith-bg: 240 inode(s) ignored (excluded by filters)
Nov 4 14:07:28 router e-smith-bg: 0 inode(s) recorded as deleted from reference backup
Nov 4 14:07:28 router e-smith-bg: --------------------------------------------
Nov 4 14:07:28 router e-smith-bg: Total number of inode considered: 2010
Nov 4 14:07:28 router e-smith-bg: --------------------------------------------
Nov 4 14:07:28 router e-smith-bg:
Nov 4 14:07:28 router e-smith-bg: Listing stdbackup archives on target:
Nov 4 14:07:28 router e-smith-bg: total 104932
Nov 4 14:07:28 router e-smith-bg: -rw-rw-rw- 1 root admin 107450147 Nov 4 2006 2006.11.04.1.dar
Nov 4 14:07:28 router e-smith-bg:
Nov 4 14:07:28 router e-smith-bg: Total target disk space usage:
Nov 4 14:07:28 router e-smith-bg: Filesystem Size Used Avail Use% Mounted on
Nov 4 14:07:28 router e-smith-bg: //filserver/Serverbackup
Nov 4 14:07:28 router e-smith-bg: 112G 106G 6.2G 95% /mnt/dar2/stdbackup
Nov 4 14:07:28 router e-smith-bg:
Nov 4 14:07:28 router e-smith-bg: Executing post-backup event
Nov 4 14:07:29 router esmith::event[17829]: Processing event: post-backup
Nov 4 14:07:29 router esmith::event[17829]: Running event handler: /etc/e-smith/events/post-backup/S10mysql-delete-dumped-tables
Nov 4 14:07:29 router esmith::event[17829]: S10mysql-delete-dumped-tables=action|Event|post-backup|Action|S10mysql-delete-dumped-tables|Start|1162645649 492291|End|1162645649 497845|Elapsed|0.005554
Nov 4 14:07:29 router esmith::event[17829]: Running event handler: /etc/e-smith/events/post-backup/S50rewind-tape
Nov 4 14:07:29 router esmith::event[17829]: S50rewind-tape=action|Event|post-backup|Action|S50rewind-tape|Start|1162645649 498562|End|1162645649 674327|Elapsed|0.175765
Nov 4 14:07:29 router e-smith-bg: Dismounting /mnt/dar2/stdbackup
Nov 4 14:07:29 router e-smith-bg:
Nov 4 14:07:29 router e-smith-bg: Sat Nov 4 14:06:51 CET 2006 - backup started
Nov 4 14:07:29 router e-smith-bg: Sat Nov 4 14:07:29 CET 2006 - backup finished
Year i have finally created a Dar file on my 2003 server with the following name:"2006.11.04.1.dar" filesize among 107 MB..
Now I want to verify my backup with Panel "Administration/Backup or restore" click "Verify Workstation Backup"
stdbackup() is preselected and i click "verify"
Result is here:
The following files are contained in the backup file:
Aborting program. User refused to continue while asking: /mnt/smb/router.linwin.dk/stdbackup/2006.1.dar is required for further operation, please provide the file.
Error while reading files from : /mnt/smb/router.linwin.dk/stdbackup/2006
What is wrong with the filename here.
Hmm . Is the mountpoint missing? nope
[root@router router.linwin.dk]# ls
dar-catalog stdbackup
[root@router router.linwin.dk]# cd stdbackup/
[root@router stdbackup]# ls
2006.11.04.1.dar
[root@router stdbackup]#
I now have 2 new folders on SME . /mnt/dar2 and /mnt/smb..
One of them is mounted to my 2003 server /mnt/smb and the other one /mnt/dar2 has been umounted by the backupscript..
How can i verify this backup ?
What am I doing wrong...?
I´m getting a little confused here...
Darrel/jpl:
Pls bear with me as I'm a newbe and doesn't know where to edit those files.
NB: My mtab is growing "1 line" each time i try to perform the verify:
/dev/mapper/main-root / ext3 rw,usrquota,grpquota 0 0
none /proc proc rw 0 0
none /sys sysfs rw 0 0
none /dev/pts devpts rw,gid=5,mode=620 0 0
usbfs /proc/bus/usb usbfs rw 0 0
/dev/md1 /boot ext3 rw 0 0
none /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
//filserver/Serverbackup /mnt/smb cifs rw,mand 0 0
//filserver/Serverbackup /mnt/smb cifs rw,mand 0 0
//filserver/Serverbackup /mnt/smb cifs rw,mand 0 0
//filserver/Serverbackup /mnt/smb cifs rw,mand 0 0
//filserver/Serverbackup /mnt/smb cifs rw,mand 0 0
//filserver/Serverbackup /mnt/smb cifs rw,mand 0 0
//filserver/Serverbackup /mnt/smb cifs rw,mand 0 0
-
Hi again.. I have managed to log onto the 2003 server now and here it gets funny..
Few important things according to your tests :
1. e-smith-backup with dar and smeserver-dar2 are two different ways to do backup, even they share use of dar. You cannot verify (as you tried) smeserver-dar2 made backup with verify option provided by e-smith-with dar.
2. I don't advise to simultaneously tests the two programs, or you can be confused.
3. smeserver-dar2 and e-smith-backup with dar are not equal in functionalities (incremental backups, selective restore, security, etc.). Just look at what are your needs and select only one of them for backup.
4. Concerning e-smith-backup with dar, one explaination is surely that there are zombies left in memory from preceeding tests (it is what suggest your mtab file). So in this case e-smith-backup see there is already a mount and doesn't do backup.
5. You normally are informed about by reading mail send to admin. What do these mails say ?
6. Before trying a new backup, you must kill all remaining smbd zombies in memory. A simple way to do it is to reboot the server.
-
Hi again.. I have managed to log onto the 2003 server now and here it gets funny..
Did you make the changes I suggested to JPL's workstation-backup-dar?
Did this work or fail for you?
As JPL and I have stated before, our two contribs are completely separate. Separate rpms, panels and they do not share any code. The only thing in common is they both use 'dar' which in itself is a separate rpm.
As you have tested and reported my contrib and code works, I've simply suggested you update JPL's workstation-backup-dar script with my code and see if this fixes things and permits his script to work. We know it will permit the mount to function. After that there may be more changes required. What you really should do is open a Bug Tracker ticket on this issue and we can work together with JPL there to find a resolution.
Darrell
-
Dmay Wrote:
Did you make the changes I suggested to JPL's workstation-backup-dar?
Did this work or fail for you?
Actually i don´t know where to edit what... so my answer is no.. i hope it will solve the problem when edited..
What you really should do is open a Bug Tracker ticket on this issue and we can work together with JPL there to find a resolution.
I will do that if it fails again after reboot
jpl Wrote:
6. Before trying a new backup, you must kill all remaining smbd zombies in memory. A simple way to do it is to reboot the server.
I will reboot the server later this evening. Time is now 5 PM here in Denmark so i will get back with the test results later this evening..
-
Actually i don´t know where to edit what... so my answer is no.. i hope it will solve the problem when edited..
This is even more reason to open a Bug Tracker ticket. It is easier to share code in the Bug Tracker so please open a ticket here:
http://bugs.contribs.org/enter_bug.cgi?product=SME%20Contribs
Component: e-smith-backup update to use DAR
Darrell
-
Hi again.
I have rebooted the SME and guess what... I should have done that first.. The thing is actually working :idea: .. verify backup is ok too....Whining happily :o
Gee man. I must admit that rebooting servers is not one of my favorites but the reboot made it work.
Thanks for helping me finding the right way to archive perfect backups, and replying to my foolish questions..
I could have waited for a howto on this contrib, but no i just fooled around with it and waisted Your time instead..
Sorry and thanks again :D
-
I have tested a bit on my a test server at home. I have not tried to do a restore yet. Hope to have time to play around with that this weekend.
I have these two findings and an entry to the wish-list:
Findings/Issues:
1) For the first many days backup failed for me. It was due to a " in th the password for the backup share. So I changed this example password fgrt!"3B to fgrt!23B and backup started to run smoothly.
2) I started a backupjob with two sets and full backup tuesday and incrementals the other days. I later changed full backup only to be allowed on sundays but that seems not to affect the current setup as a full backup still occours tuesday. I did try to disable and reenable the backup but no change. Maybe if I delete all backups in the backup share and restart the backup job it will work. This has not yet been tested. I guess this is not a bug but there may be needed a little info on how to reset the backupsettings (e.g. disbal backup, rename folder with server name on backup destination folder, reconfigure backup, reenable backup- if that is the way to do it)
Wish-list:
1) It would be really nice to have some sort of exclude list of folders not to be backed up. My testserver contains nearly 90GB opf data where 60-70 GB are static data not needed to be backed up.
BR. Anders
-
Wish-list:
1) It would be really nice to have some sort of exclude list of folders not to be backed up. My testserver contains nearly 90GB opf data where 60-70 GB are static data not needed to be backed up.
BR. Anders
You can exclude files as you want because the backup configuration uses templating.
To exclude files you must use a custom template and create a file /etc/e-smith/custom-templates/etc/dar/DailyBackup.dcf/50prune to satisfy your needs. You must write one prune line for each excluded file or directory.
Syntax (from man dar) is :
--prune <path>
Do not consider file or directory sub-tree given by the path. The path mask is not applied only to the filename, but also include the path. Moreover it applies also to directories. Note that <path> may contains wild-cards like * or ? see glob(7) man page for more informations.
Dar template expansion is made automagically before each backup.
-
You can exclude files as you want because the backup configuration uses templating.
Thanks jpl
That is really great and usefull info. I will try to exclude some folders.
BR. Anders
-
Is it also possible to use this contrib to backup to an usb REV drive?!
The drive is automagically being mounted on /media/cdrom (after reboot), but using the backup to 'local usb disk' option i get a 'no drives found' sort of error.
Can anyone shed some light?!
-
Is it also possible to use this contrib to backup to an usb REV drive?!
The drive is automagically being mounted on /media/cdrom (after reboot), but using the backup to 'local usb disk' option i get a 'no drives found' sort of error.
Can anyone shed some light?!
"The drive is automagically being mounted on /media/cdrom"
Here is the explaination. e-smith-backup excludes and doesn't show /media/cdrom... because a cdrom should not be writable !
If you cannot change the mount name, and if your usb drive mounts as /media/cdrom, you could try to comment line 1565 ("next if /cdrom/;") from /etc/e-smith/web/functions/backup script.
-
Could we get the backup software to mount the USB disk when the backup starts and unmount it when it is finished.
I noticed that the process goes as such:
- Check to see if network mount is already mounted
- If not mount network point
- If mounted end with error
- Run backup
- Unmount mount point
But this only happens if it is using a smb/cifs mount point.
It is important to have this happen for usb disks as well so that the data stays intact in the case of a power failure.
Thanks,
Daniel Davis
-
Usb disks file systems are hotplugged.
We could unmount usb disk after backup, but are you sure that it provides more security (i'm not a specialist of usb disks) ?
-
Usb disks file systems are hotplugged.
In practice this doesn't work well with ext3 drives. Always best to unmount so ext3 journalling terminates prior to disconnect.
Darrell
-
Where should I go looking to resolve this ?
Backup base file name is full-20061201
Making backup on temporary dir...
using a backup session timeout of : 88500 seconds
*** No backup allowed or error during backup ***
Dar error during backup : 4
Thanks
Peter
-
The code 4 is return by the dar command.
Here is the man explanation for dar error 4 :
4 Code issued when the user has aborted the program upon dar question from dar. This also happens when dar is not run from a terminal (for example launched from crontab) and dar has a question to the user. In that case, dar aborts the same way as if the user pressed the escape key at the question prompt.
A way to debug is to launch /etc/e-smith/events/actions/workstation-backup-dar from a root console. If this is really an error 4 from dar you should have some question asked to you by dar about the backup job.
-
jpl
You may want to do what dmay did with his contrib and put it under a diff heading etc.
After the server update I lost your ver of the dar backup out of the server manager panel and my backups stopped as well ... it was working really well till then.
I will try and re-load it on the test server and see what happens.
Regards,
Tib
-
jpl
Re-load didn't work:
Preparing... ########################################### [100%]
package e-smith-backup-1.14.0-7.el4.sme (which is newer than e-smith-backup-1.14.0-06jplb01) is already installed
Can you please help with this ... I love your ver and would like to keep using it.
Regards,
Tib
-
My answer on the bugtrack this morning :
you can wget :
http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-7jplb01.el4.sme.noarch.rpm
and
http://beta.free-eos.org/sme/SRPMS/e-smith-backup-1.14.0-7jplb01.el4.sme.src.rpm
tell me if cleanly upgrades
Will see later to change heading
I left in beta, but no bug detected since last release.
-
Hello jpl
Everything seems to be fine on install.
Picked up all the prior settings so we will see I guess.
Backup will start this afternoon.
Thanks ... I love this contrib :)
Regards,
Tib
-
I'm very pleased with the DAR tool as it saved my life/server last weekend.
After reinstalling version 7.1 all data i-bays were restored in a breeze !!!
Also all users were restored, all directories exist, however a few of my users (including me) are not able to access their e-mails anymore (IMAP) neither through Horde nor through outlook express.
I don't know what i might have done wrong; all directories including maildir were restored beautifully, all "files" are still there; however after trying to connect to the account with outlook express, i get the following error:
Internal error: could not set UID/GID
Configuratie:
Account: Taco op nieuwe server
Server: fileserver
Gebruikersnaam: twijtzes
Protocol: IMAP
Poort: 143
Beveiligd(SSL): 0
CODE: 800ccc0f
What is this; is it resolvable ?
Kindest regards
Taco
-
SME rules...
My hat's off to Darrell May for the dar2 contrib. It worked flawlessly. I migrated to new hardware in about 2 hours and all 60gigs of data are intact.
Thank you!!!!!
I did this to upgrade my server to new hardware. These are the exact steps I used to test a server upgrade. Before using
this, try it out on test hardware. I'm still pretty much a n00b, but after *READING* and *LEARNING*, this is the method I
came up with. This method is considered a "full backup", in that it backs up e-smith's config files, passwords, groups,
users, and user data. Keep in mind that this method will NOT move your contribs over too. After restoring the server to
the new hardware, you will have to reinstall any contribs you want. This is the method that I used:
To set up an external hard drive for use with linux and ext3, see dmay's
tutorial:
http://mirror.contribs.org/smeserver//contribs/dmay/smeserver/7.x/smeserver-usb-disk-howto.htm
1. Install fresh copy of SME 7.1 on machine1.
2. Set up 4 users and copy some data > 10GB. Mess around with the users, make some disk quotas, copy some data, etc.
3. Do a yum update:
yum update
signal-event post-upgrade
signal-event reboot
4. Install dar:
wget http://mirror.contribs.org/smeserver//contribs/dmay/smeserver/7.x/dar-2.3.2-1.i386.rpm
wget http://mirror.contribs.org/smeserver//contribs/dmay/smeserver/7.x/smeserver-dar2-0.0.1-1dmay.noarch.rpm
rpm -i dar-2.3.1-1.i386.rpm
rpm -i smeserver-dar2-0.0.1-0dmay.noarch.rpm
*perl-CGI-FormMagick-0.92-08.noarch.rpm
*(probably won't need this, a newer version is already on SME 7.1)
5. Mount external USB hard drive. (I had to add an entry to /etc/fstab and reboot. Editing fstab was probably
unnecessary, but I was being impatient.)
Don't forget to mkdir to create a mount point:
mkdir /media/usbdisk
mount /dev/sda1 /media/usbdisk
6. signal-event pre-backup
This prepares mySQL data for backup.
7. Perform dar backup from your admin web console. If it screws up, you can use the web console to view the log files;
just look under "messages". When dar completes, it will automatically umount the usb drive.
If you want to verify, just:
mount /dev/sda1 /media/usbdisk
ls /media/usbdisk
Being the careful geek that I am, I made a directory called "This_Is_The_Backup_Drive" on the external drive so I know
that it is actually mounted when I do an ls on it, helpful if the drive is really empty.
You can still ls /media/usbdisk but if its not mounted, nothing will show up. When you do a backup using dar2, it will
create a directory of the backup date that contains the backup data. When the backup is done, unmount the usb drive.
(dar does an umount anyway, but if you remounted it to verify the backup went okay, don't forget to unmount it):
umount /media/usbdisk
*("sda1" may differ for your machine)
------------------------------------------------
8. Install fresh copy of SME 7.1 on machine2.
9. Do a yum update:
yum update
signal-event post-upgrade
signal-event reboot
10. Install dar2 per instructions above.
11. Mount external USB hard drive per instructions above.
12. Perform restore using dar2. *
"cd" to the directory that contains the dar files.
ex: /media/usbdisk/yoursevername.yourservername/2007.02.06
Do an ls. You should see a bunch of files that end in ".dar".
Use this command example to restore:
dar -x yyyy.mm.dd -w -R /
ex: dar -x 2007.02.06 -w -R /
*(ensure the new server has the same name and the same IP address as the old one.
you can change this later...dont forget to unplug the ehternet cable from machine1,
so IP address dont conflict... duhhh)
^^= I really don't think this matters, but I did do it just in case...careful geek.
13. signal-event reboot
14. Log in as admin and go thru the "Configure This Server" process again.
15. Test, test, test!
Notes:
*Directories to backup (dar2's default):
etc/e-smith/templates-custom
etc/e-smith/templates-user-custom
etc/group
etc/gshadow
etc/passwd
etc/samba/secrets.tdb
etc/samba/smbpasswd
etc/shadow
etc/smbpasswd
etc/ssh
etc/sudoers
home/e-smith
root
machine1:
HP D510, P4 2.0Ghz, 256MB RAM, Seagate 160GB IDE/ATA100 Hard Drive
machine2:
HP D7100, P4 2.4GHz, 512MB RAM, Seagate 160GB SATA Hard Drive
Western Digital 160 GB USB External Hard Drive Enclosure.
I hope this helps someone. I'm open to any suggestions!
-
SME rules...
My hat's off to Darrell May for the dar2 contrib. It worked flawlessly. I migrated to new hardware in about 2 hours and all 60gigs of data are intact.
Please post dar2 contrib comments to dar2 subject, not on this subject. Posting here dar2 comments is confusing. This subject is for e-smith-backup with dar.
JPL
-
I have installed this contrib on a few servers. But the latest one gave some problems. I have set up the parameters to be exactly as on another server where the contrib can do the backup.
[root@ronja ~]# /etc/e-smith/events/actions/workstation-backup-dar
Error while mounting //192.168.2.22/backup :
timeout connecting to 192.168.2.22:445
timeout connecting to 192.168.2.22:139
Error connecting to 192.168.2.22 (Operation already in progress)
6314: Connection to 192.168.2.22 failed
SMB connection failed
I have checked the firewall. And the server the traffic to the 192.168.2.22 will passing.
when searching the forum for (Operation already in progress) I get three hits with no good explanations. The only post there leads to a solution is a reinstall. If noone knows anything about the error please let me know how to uninstall DAR and this backup contrib so I can try to reinstall them.
BR. Anders
-
Problem solved. The server I did the backup to (also a SME server) had not been configured with the extra "local network" containing the server to be backup. Therefore the error. I just did not understand the error. But all works once again :-)
BR. Anders
-
I have 3 questions:
1 - when I start a backup (USB hdd) without de usb disk mounted the backup is made to the hdd in the server. Result: hdd full and the server will crash. Is it possible to check whether the usb disk has been mounted / check available diskspace before de actual backup is made?
2 - I had problems restoring a file with spaces in the filename. Does anyone else have this problem?
3 - I have to schedule a backup now, but a button 'Backup now' would be welcome. I do not have the usb disk installed permanently, so scheduling will not always work (with a full disk as a result).
Regards
Marcel
-
I have put bug report through for this again ...
After the latest update the dar backup panel dissapears.
Same problem as the last post about the update and dar.
Regards,
Tib
-
what directory is the database stored so i can backup the hole database every hour and backup of the hole server or the default directories every 3 hours?
-
I have put bug report through for this again ...
After the latest update the dar backup panel dissapears.
Tib
New updated release available. See first post for updated links.
Just built a few minuts ago. Please do some tests before using on production server.
-
Install went fine .... Backups will start tonight ... I will report tomorrow.
Regards,
Tib
-
I have 3 questions:
...
2 - I had problems restoring a file with spaces in the filename. Does anyone else have this problem?
Regards
Marcel
Bug found... updated rpm will follow soon...
-
OK ... the backups all went well.
I have not done any restores as yet but will try that out later.
Regards,
Tib
-
I have 3 questions:
...
2 - I had problems restoring a file with spaces in the filename. Does anyone else have this problem?
Regards
Marcel
Bug found... updated rpm will follow soon...
e-smith-backup-1.14.0-13jplb02.el4.sme.noarch.rpm is available
Should provide correct selective restoration for files with spaces in names.
Confirmation welcomed.
See updated first post for download links.
-
hi, i have installed this Contrib but i dont see a Menu on the Server Manager
yum localinstall e-smith-backup-1.14.0-7jplb01.el4.sme.noarch.rpm
==============================================================
WARNING: Additional commands may be required after running yum
==============================================================
Loading "smeserver" plugin
Loading "fastestmirror" plugin
Setting up Local Package Process
Examining e-smith-backup-1.14.0-7jplb01.el4.sme.noarch.rpm: e-smith-backup - 1.14.0-7jplb01.el4.sme.noarch
e-smith-backup-1.14.0-7jplb01.el4.sme.noarch.rpm: does not update installed package.
Nothing to do
I have restart and Reconfigurig the Server but not changes.
-
You tried to install e-smith-backup-1.14.0-7jplb01.el4.sme.noarch.rpm, but version to install is e-smith-backup-1.14.0-13jplb02.el4.sme.noarch.rpm.
See http://forums.contribs.org/index.php?topic=34187.0
-
Thank You it works now but the Descriptions not work in German:
(http://img89.imageshack.us/img89/807/zwischenablage02bj4.th.jpg) (http://img89.imageshack.us/my.php?image=zwischenablage02bj4.jpg)
-
2 - I had problems restoring a file with spaces in the filename. Does anyone else have this problem?
e-smith-backup-1.14.0-13jplb02.el4.sme.noarch.rpm is available
Should provide correct selective restoration for files with spaces in names.
Confirmation welcomed.
See updated first post for download links.
Just got back from a skiiing holiday: I don't have time to test this now, but I'll try to do it asap...
-
Sorry for this offtopic, but I found a good text about dar.
http://gradha.sdf-eu.org/textos/dar-differential-backup-mini-howto.en.pdf
-
2 - I had problems restoring a file with spaces in the filename. Does anyone else have this problem?
e-smith-backup-1.14.0-13jplb02.el4.sme.noarch.rpm is available
Should provide correct selective restoration for files with spaces in names.
Confirmation welcomed.
I had problems making the backup now, and found out why:
I use an usbdisk, mounted at /media/usbdisk. I can access the disk without problems, but when the backup started my normal disk started to fill up, until the system crashed. The reason for this is that the .dar files were stored in /mnt/smb/tmp_dir, and not on /media/usbdisk. I did not have enough space on the HDD, only on the usbdisk. I made another mount: /mnt/smb/tmp_dir to my usbdisk and everything worked fine.
One point: in case of an usbdisk, this contrib is storing the backup in the tmp_dir first, and then all .dar files are copied into the target directory (set0). The copying is not nessecary I think: when I can just use the usbdisk to store the dar files I think the copying is obsolete because I do not use the network.
To mount the usbdisk I use the usbdisk contrib: the combination is working perfectly.
Tomorrow I will try to recover a file with spaces in the name: my pc has just started copying 85 dar files, and I am going to sleep now...
Regards,
Marcel
-
Restoring a file with spaces in the pathname is working well now, thanks!
Regards,
Marcel
-
jpl
Question: ... where do I find the settings to include other directories in the backup.
I wanted to do a restore on some files in the /opt dir and I found that it is not being backed up.
Lucky this was my test server.
Regards,
Tib
-
1. Copy /etc/e-smith/templates/etc/dar/DailyBackup.dcf/40go-into to /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/40go-into
2. make your adds in /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/40go-into
3. expand /etc/dar/DailyBackup.dcf
-
Hi
marceld, had the problem:
I use an usbdisk, mounted at /media/usbdisk. I can access the disk without problems, but when the backup started my normal disk started to fill up, until the system crashed.
I have exactly the same problem, although when I try to mount the the tmp_dir is says it dont exist.
Why does it do this? is there a way to stop it because what is strange is i was watchin the usb drive and it appeared to increase in size so i dont think it does it after them all.
Thanks
k0r54
-
Because my system crashed (HDD full) I searched where the .dar files were. They were in tmp_dir: normally the backup program will probably erase the dir when finished, but because the process ended abnormally it was still there on my system.
What I don't understand in your post: did your HDD fill up or your usb disk? And did you succeed making a backup to the usbdisk?
Regards
Marcel
-
Hi,
No the backup did not succeed and it was my hdd that was full not the usb. The usb has plenty of space.
When i rebooted the server though it did come back to life fine but its just the backup did not finish.
Thanks
k0r54
-
When doing a selective restore of a folder I get:
Error while calling execvp:Argument list too long
and the restore doesn't work....Any ideas?
thanks,
-
How do you do so you dont have to mount the usb disk every time the backup is running before i reinstalled SME it was working fine but cant find anymore
i did this db configuration setprop /mnt and reconfigured so we will see if that did the trick
-
With new 2.6.9-55 SME kernel, cifs support in e-smith-backup with dar should now be correct.
Older SME kernels had bug in cifs.
Now e-smith-with dar fully support backups through local usb mount, smbfs, nfs and cifs.
-
Hi
I am hoping to backup my whole server to an external USB disk, prior to upgrade to version 7. We are currently running 6.0.1 but all the backup usin dar contribs seem to be for version >7.
Is there a way to get the backup down to a USB drive before upgrade?
Thanks in advance
Gordon
-
Can I backup installed contribs & SME updates?? Is it safe todo so? Can I run into problem when I do a full restore after a fresh installation?
-
Can I backup installed contribs & SME updates??
search for what is backed up, these generally aren't
contribs can install files anywhere on the filesystem
reinstall contribs
yum update to get updates