Topic: [Announce] Dar - Disk Archive (alpha build)

[byte]

Over on...

http://bugs.contribs.org/show_bug.cgi?id=1880

there is some great work being made to incorporate the excellent work of Jean-Paul Leclère & Free-EOS (http://free-eos.org) also with Darrell May who brought us Backupws.

Please only report back on the bug shown above as they will probably not see any posts here, so make it easy and post feedback there.

[icpix]

byte----
I installed the dar rpm referenced at:
wget http://212.234.119.227/dar-2.3.1-1.i386.rpm
rpm -ivh dar-2.3.1-1.i386.rpm
...then updated the standard e-smith-backup with...
http://bugs.contribs.org/attachment.cgi?id=547
rpm -Uvh e-smith-backup-1.14.0-05jpltest03.noarch.rpm
SME7.0 control panel (for 'Backup or restore') reports that:
'Your server has too much data for a reliable backup to desktop.'
Brief specs:
* SME7 (OS) -  two 75GB in s/w RAID1
* SME7 (data) - 2TB in h/w RAID5 mounted in an iBay.
----best wishes, Robert

[jpl]

'Your server has too much data for a reliable backup to desktop.'
Brief specs:
* SME7 (OS) -  two 75GB in s/w RAID1
* SME7 (data) - 2TB in h/w RAID5 mounted in an iBay.
----best wishes, Robert

Yes... but it's true, no ?
Is this message preventing workstation backup ?

[icpix]

jpl----

Cannot agree;~| I have distributed pockets of workstation GBs available.
It is a failing of the calling application that it cannot so distribute the much
needed backup. Darrell's original contrib (Backup2) manages to distribute
backup sets appropriately without issue though it could do with going at it
faster and using incrementals/partials.

No it doesn't 'stop' me from going ahead with a backup attempt despite me
having to so do against a red line warning! Ten hours ago I set the thing
off... I gave it a single hour's window to see what happened. There is
nothing at all on the target/destination share. My CPU util has been showing
circa 95% for a dar process continuously. My OS drive array had some
50 or 60GB space on it originally. 'Something' has filled this up and now
there is about 2GB remaining. Presumably the backup run is now using
my (small) OS drive array to aggregate the backup run for a 2TB (data)
drive array... so I can see why I was given a red line text warning initially.

Presumably I now have to try to abort this thing somehow before the
entire OS drive array space remaining hits the floor... is there a big
red 'OFF' button somewhere?

[postedit] ...too late it's hit the floor, the OS array has 0 bytes left.

----best wishes, Robert

[icpix]

( duplicate posting )
----best wishes, Robert

[icpix]

To recover working space on the OS array I deleted some 95 .dar
backup dataset files (CD-sized? ~718MB each) I found remaining on...
/mnt/smb/servername.mydomain/set0/
...and then deleted the /mnt/smb. At no point did I receive any
admin or warning emails nor any indication(s) at the control panel
about what might be or actually was happening.
----best wishes, Robert

[icpix]

Believe I misinterpreted the function of the box for...
Timeout only incrementals
...which earlier I had ticked (hence the 10hr never-ending marathon?).
Attempted another 1hr (backup window duration) test.
Shortly after the start received these two emails:

Subject: Cron <root@teri> /sbin/e-smith/do_backupwk
Could not resolve mount point /mnt/smb
No terminal found for user interaction. All questions will be assumed a negative answer (less destructive choice), which most of the time will abort the program.
Received signal: Terminated
Archive delayed termination engaged
Disabling signal handler, the next time this signal is received the program will end immediately
Program has been aborted for the following reason: Thread cancellation requested, aborting as properly as possible
umount2: Invalid argument
umount: /mnt/smb: not mounted

Subject: Daily Backup Report
==================================
DAILY BACKUP TO WORKSTATION REPORT
==================================
Backup started at Tue Sep 19 11:30:01 2006
Backup of mysql databases has been done.
Mounting backup shared directory //workstation/sme7-f
Backup shared directory mounted on /mnt/smb
Using set number 0 of 1
Making full backup
Backup directory teri.mydomain/set0 created
Backup base file name is full-20060919
Making backup on temporary dir...
Backup success on shared folder temporary dir
Deleting files on target backup directory
Moving backup files to target directory //workstation/sme7-f/teri.mydomain/set0
Updating backup configuration data
Backup successfully terminated at : Tue Sep 19 11:30:03 2006

This time I've left the /mnt/smb stuff in situ.
There's about 2MB of a single .dar file stored (/mnt/smb/...)
but absolutely nothing has appeared on the target drive share.
Not been brilliant so far;~/
a) never-ending, system filling, backup run that didn't work.
b) A barely started backup run that didn't work.
Did I break something somewhere!?

----best wishes, Robert

[icpix]

That red line warning was right about the unreliability.
Have unsuccessfully tried three locations on my intranet.
My own workstation, a spare workstation and a NAS box.
All abort shortly after starting:

Subject: Cron <root@teri> /sbin/e-smith/do_backupwk

Code: [Select]

No terminal found for user interaction. All questions will be assumed a negative answer (less destructive choice), which most of the time will abort the program.
Received signal: Terminated
Archive delayed termination engaged
Disabling signal handler, the next time this signal is received the program will end immediately
Program has been aborted for the following reason: Thread cancellation requested, aborting as properly as possible

The file size left at the destination share is ~1.7MB and
which reminds me of the old SCSI days of an unterminated
drive causing errors on the chain but still allowing the first
1MB or 2MB through into the buffer. Seems clear I have
broken something... just let me know what you need.

----best wishes, Robert

[jpl]

Believe I misinterpreted the function of the box for...
Timeout only incrementals
...which earlier I had ticked (hence the 10hr never-ending marathon?).
Attempted another 1hr (backup window duration) test.
Shortly after the start received these two emails:

Subject: Cron <root@teri> /sbin/e-smith/do_backupwk
Could not resolve mount point /mnt/smb
No terminal found for user interaction. All questions will be assumed a negative answer (less destructive choice), which most of the time will abort the program.
Received signal: Terminated
Archive delayed termination engaged
Disabling signal handler, the next time this signal is received the program will end immediately
Program has been aborted for the following reason: Thread cancellation requested, aborting as properly as possible
umount2: Invalid argument
umount: /mnt/smb: not mounted

Please could you answer next questions :

- What is the value you give to timeout ? 3600 ?
- logs indicates that the server was unable to mount the share. Are you sure that the share configuration was ok, and no other mount deamon was left on the share point ?

[jpl]

jpl----

Cannot agree;~| I have distributed pockets of workstation GBs available.
It is a failing of the calling application that it cannot so distribute the much
needed backup. Darrell's original contrib (Backup2) manages to distribute
backup sets appropriately without issue though it could do with going at it
faster and using incrementals/partials.

The message that makes you angry is MESSAGE FROM ORIGINAL SME E-SMITH-BACKUP package. Not issued by my modifications.

If you think it must not be issued, create a bug about it on the bugtrack.

[icpix]

Here's some stuff in the server's message log:

Code: [Select]

Sep 19 13:57:01 teri esmith::event[9383]: S50rewind-tape=action|Event|pre-backup|Action|S50rewind-tape|Start|1158670621 513701|End|1158670621 612080|Elapsed|0.098379
Sep 19 13:57:02 teri kernel: init_special_inode: bogus i_mode (0)
Sep 19 13:57:02 teri kernel: smb_retry: no connection process
Sep 19 13:57:32 teri kernel: smb_add_request: request [f1ec7e00, mid=0] timed out!
Sep 19 13:57:32 teri kernel: smb_delete_inode: could not close inode 2
Sep 19 13:57:32 teri mount.smbfs[9392]: [2006/09/19 13:57:32, 0] client/smbmount.c:send_fs_socket(410)
Sep 19 13:57:32 teri mount.smbfs[9392]:   mount.smbfs: entering daemon mode for service \\workstation\sme7dar, pid=9392
Sep 19 13:57:34 teri kernel: smb_writepage_sync: failed write, wsize=4096, write_ret=-512
Sep 19 13:57:34 teri esmith::event[9408]: Processing event: post-backup  
Sep 19 13:57:34 teri esmith::event[9408]: Running event handler: /etc/e-smith/events/post-backup/S10mysql-delete-dumped-tables
Sep 19 13:57:34 teri esmith::event[9408]: S10mysql-delete-dumped-tables=action|Event|post-backup|Action|S10mysql-delete-dumped-tables|Start|1158670654 378105|End|1158670654 380457|Elapsed|0.002352
Sep 19 13:57:34 teri esmith::event[9408]: Running event handler: /etc/e-smith/events/post-backup/S50rewind-tape
Sep 19 13:57:34 teri esmith::event[9408]: S50rewind-tape=action|Event|post-backup|Action|S50rewind-tape|Start|1158670654 380775|End|1158670654 479030|Elapsed|0.098255
Sep 19 13:57:34 teri /sbin/e-smith/do_backupwk[9382]: /home/e-smith/db/backups: OLD 1158670621=backup_record|BackupType|workstation|StartEpochTime|1158670621
Sep 19 13:57:34 teri /sbin/e-smith/do_backupwk[9382]: /home/e-smith/db/backups: NEW 1158670621=backup_record|BackupType|workstation|EndEpochTime|1158670654|StartEpochTime|1158670621
Sep 19 13:57:34 teri /sbin/e-smith/do_backupwk[9382]: /home/e-smith/db/backups: OLD 1158670621=backup_record|BackupType|workstation|EndEpochTime|1158670654|StartEpochTime|1158670621
Sep 19 13:57:34 teri /sbin/e-smith/do_backupwk[9382]: /home/e-smith/db/backups: NEW 1158670621=backup_record|BackupType|workstation|EndEpochTime|1158670654|Result|0|StartEpochTime|1158670621


Possibly it may be that old issue (smb/samba/login-passwords)... possibly.

----best wishes, Robert

[icpix]

jpl----

- What is the value you give to timeout ? 3600 ?
- logs indicates that the server was unable to mount the share. Are you sure that the share configuration was ok, and no other mount deamon was left on the share point ?

What timeout?
Maximum allowed backup time (hours) = 1

Share config was/is OK.
Your application makes new directory structure...
Other mount daemons? I don't know, I'm just the human;~)

----best wishes, Robert

[icpix]

jpl----
The red line message didn't make me angry.
My ;~| (pursed lips) is for annoyance!
I could not see the point of a backup package that tells
me I cannot backup because I have too much data.
eg "You can't bank here at Barclays/Lloyds bank,
you have too much money... try at Coutts!"
;~))
----best wishes, Robert

[jpl]

jpl----

No it doesn't 'stop' me from going ahead with a backup attempt despite me
having to so do against a red line warning! Ten hours ago I set the thing
off... I gave it a single hour's window to see what happened. There is
nothing at all on the target/destination share. My CPU util has been showing
circa 95% for a dar process continuously. My OS drive array had some
50 or 60GB space on it originally. 'Something' has filled this up and now
there is about 2GB remaining. Presumably the backup run is now using
my (small) OS drive array to aggregate the backup run for a 2TB (data)
drive array... so I can see why I was given a red line text warning initially.

No, the backup run does not use your OS drive to aggregate anything.

Explication may be more trivial : share is not mounted and the backup was correctly running and saving datas on the server itself... 50 or 60 Gb of compressed data saved in 10 hours seems a normal value for that.

The questions in this case are : why was the share not correctly mounted, and why backup script does not detect it ?

Another point : how workstation dar backup will do to backup 2Tb of datas ?

In theory, it can do it, but not in one night through lan share. You must try to use one set of backup and incremental backup for a lot of days...
The first night the first backup will be stopped by the timeout you set (value is in seconds, not in hours). Not Terabytes of datas will be saved, just tens of Gb. The backup should restart and continue on next nights... until all datas are saved. It's possible only if the daily amount of changed datas is inferior to the amount of datas saved every night.

I think you understand we have not make tests for such amount of datas. But if it works it's a promising test

[icpix]

jpl----

It must have used my OS drive to aggregate because that is where I found
all the .dar files sitting, whereas they SHOULD have been across the LAN
on my workstation. For some reason the samba/smb/login-password stuff
is not working...

My 2TB array is not yet full. About 1TB at the moment. It is built up in lots
of directories. These various directories I have to backup over the LAN. It
is not at all important to do it all in one go... a little bit all the time is good.
Much of the data does not change for days or even months.

The control panel says hours so I put in '1'. I will now try 3600;~)

I will work DAR very hard if I can get it to work for me;~)

----best wishes, Robert