Topic: HUGE PROPS - Backup2 Coder

[orcusomega]

I have to admit, I am remiss in finding the contributor of Backup2, but THANKS!

I had kind of written off SME for a bit since I could not find a good way to backup my SME box automagically, but this not only fits the bill, it exceeds it!

You have saved me a huge amount of time and grief, if you are in the Philly are,        on me!

I just wish I had the time and skill to come up with some contribs myself  Anyone need help?

Bob

[dmay]

Thanks Bob.

FYI to find a contrib author and all contributors involved in the rpm development simply query the rpm:

If local but not installed:

# rpm -qip --changelog smeserver-backup2-0.0.1-b1dmay.noarch.rpm | more

Once installed, drop the p (package) and version/release/arch to query the rpm name only:

# rpm -qi --changelog smeserver-backup2 | more

[orcusomega]

Thanks for the extra info...

...but...

LOL you get more props.  I just got your Asterisk install up and (mostly) running, and between that, Backup2, and SME7admin, this has saved me hundreds of dollars for my small home office.

I did find that the script that was included in the tarball was not installing for some reason, but the logging of the error was not there... and the buffer for my SSH session did not catch it either.  There is a script on the board that I found that fixed the problem... but you rock.

   

Do you have anything cooking that 1) you can share or 2) need some inexperienced help with?

Bob


Bob

[orcusomega]

Here is a link to the script I found here that solved the problem:

http://forums.contribs.org/index.php?topic=29275.0

Bob

[stocky]

I looked at the backup2 contrib and thought it looked great!

Then i tried to install it...it wanted RAR (which it said it was based on, so i realised that...not that n00b!) so i tried to install RAR...whcih then told me i needed a heap on dependancies (I am running SME7pre3).

Where do I find a list of all the modules so I can find these required dependancies?

help for a poor n00b....I am still learning...i have been brainwashed by M$ for too long.....

[p-jones]

Ditto on failed dependancies.

[orcusomega]

What problems did you encounter?

Can you give me the SME version you are running?
Can you give me a screen shot or a cut and paste of the error?

One way I isolated the problem was to pull apart the three scripts:

install.sh
install_parts.sh
install_fin.sh

and run chunks of the script sequentially to better isolate the problem.

You need to export the LOAD_LOC variable, so before you start pulling them apart, copy the tarball to /tmp (or wherever), and untar it

Code: [Select]


cd /tmp
wget ftp://anonymous@firewall-services.com/Asterisk/A4SME7.tar.gz
tar zxvf A4SME7.tar.gz
cd A4SME7
export LOAD_LOC=pwd


Then go through chunks of the script.  When/if you find the failed dependancies, let me know, I will help if I can.

Bob

[orcusomega]

Oh wait, did you need this library?

Code: [Select]


compat-libstdc++-33.i386 3.2.3-47.3


If so, then run

Code: [Select]


yum install compat-libstdc++


The OS (Centos) will go get the libraries, and that should help.  I forgot I needed to do that on SME7pre1 updated to SME7pre3 via yum.

Also, if you run into dependancy problems, you can use yum to go look for what is available.  For example, if you get an error saying you need dependancies, Yum will usually recommend a package.  To find it, run

Code: [Select]


yum search [package name]


if it finds it in the yum repository, then you can install it using

Code: [Select]


yum install [package name]


Hope this helps,

Bob

[dmay]

Correct. A simple forum search on 'rar depend', and then 'Show all results' would have returned the answer for the posters having issues. For SME 7.x you should install rpms using yum. The command below installs all required rpms and automagically all dependencies:

Code: [Select]

# yum localinstall rar-3.5.1-7dmay.i386.rpm smeserver-backup2-0.0.1-b1dmay.noarch.rpm

==============================================================
WARNING: Additional commands may be required after running yum
==============================================================
Setting up Local Package Process
Examining rar-3.5.1-7dmay.i386.rpm: rar - 3.5.1-7dmay.i386
Examining smeserver-backup2-0.0.1-b1dmay.noarch.rpm: smeserver-backup2 - 0.0.1-b1dmay.noarch
Marking rar-3.5.1-7dmay.i386.rpm to be installed
Marking smeserver-backup2-0.0.1-b1dmay.noarch.rpm to be installed
Resolving Dependencies
--> Populating transaction set with selected packages. Please wait.
---> Package smeserver-backup2.noarch 0:0.0.1-b1dmay set to be updated
---> Package rar.i386 0:3.5.1-7dmay set to be updated
--> Running transaction check
Setting up repositories
smeaddons                 100% |=========================|  951 B    00:00
contrib                   100% |=========================|  951 B    00:00
updates                   100% |=========================|  951 B    00:00
base                      100% |=========================| 1.1 kB    00:00
centosplus                100% |=========================|  951 B    00:00
addons                    100% |=========================|  951 B    00:00
smeos                     100% |=========================|  951 B    00:00
smeupdates                100% |=========================|  951 B    00:00
Reading repository metadata in from local files
--> Processing Dependency: libstdc++.so.5(CXXABI_1.2) for package: rar
--> Processing Dependency: libstdc++.so.5 for package: rar
--> Processing Dependency: libstdc++.so.5(GLIBCPP_3.2) for package: rar
--> Restarting Dependency Resolution with new changes.
--> Populating transaction set with selected packages. Please wait.
---> Package compat-libstdc++-33.i386 0:3.2.3-47.3 set to be updated
--> Running transaction check

Dependencies Resolved

=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 rar                     i386       3.5.1-7dmay      rar-3.5.1-7dmay.i386.rpm  432 k
 smeserver-backup2       noarch     0.0.1-b1dmay     smeserver-backup2-0.0.1-b1dmay.noarch.rpm   70 k
Installing for dependencies:
 compat-libstdc++-33     i386       3.2.3-47.3       smeos             226 k

Transaction Summary
=============================================================================
Install      3 Package(s)
Update       0 Package(s)
Remove       0 Package(s)
Total download size: 729 k
Is this ok [y/N]: y
Downloading Packages:
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: compat-libstdc++-33          ######################### [1/3]
  Installing: rar                          ######################### [2/3]
  Installing: smeserver-backup2            ######################### [3/3]

Installed: rar.i386 0:3.5.1-7dmay smeserver-backup2.noarch 0:0.0.1-b1dmay
Dependency Installed: compat-libstdc++-33.i386 0:3.2.3-47.3
Complete!
==============================================================
WARNING: You now need to run BOTH of the following commands
to ensure consistent system state:

signal-event post-upgrade; signal-event reboot

You should run these commands unless you are certain that
yum made no changes to your system.
==============================================================

[orcusomega]

LOL a LOT easier than my posting, and I had confused the Asterisk and the Backup2 posts.

Thanks for the help!

Bob

[stocky]

Thanks..you learn something new everyday around here!  
dmay, i followed your method above (i didnt know the bit about it being recommended to use YUM to install rpm's so thanks!) but it didnt work, but then i realised your yum must be configured to look a lot more places than mine!

Mine only looks in:

smeaddons
smeos
smeupdates

how do I add the rest (eg contribs?)

thanks in advance

PS i am noting all this down for future refereance so I can keep teaching myself!

[dmay]

A quick search would find this:

http://forums.contribs.org/index.php?topic=30043.0

[stocky]

sorry...my bad!

wget the files first!  

[stocky]

hmmm...i searched...didnt find that, so i asked.

Thank you dmay

[stocky]

Hi all,

Can anyone assist with this?

My current setup is.......

ROUTER(10.0.0.1)
|
|
Remote LAN (10.0.0.2)
SME BOX
Local LAN (192.169.101.1)
|
|
Remote Lan (192.168.101.2)
SBS 2003 Svr
Local LAN (192.168.201.2)
|
|
Rest of SBS network

I am trying to figure out a way to backup the SME box to the Windows server nightly so that evrything is backed up. I have created a share on the SBS server (SME_BU) \\servername\sme_bu

The problem I guess I am having it that the folder is not visible from the SME side of the SBS server. Am I right in thinking that I can set up the SBS box to have an FTP server and backup to that? Do I use Backup2 or rsync (that looks scary for a n00b to linux)

any help or suggestions gratefully accepted!

[p-jones]

Unless you have some strange security settings on the SBS I would expect that to work so long as you provided a correct username/password with appropiate privilleges for the SBS. It is almost identical to what I am doing very happily.

Have you checked the log files on the SME box ? What do the error messages say ? Are you certain the error is on the SBS box and not on the SME side - Incorrect syntax, invalid paths to the files you want to back up or something else which is relatively simple that you have overlooked.

[stocky]

so you have ur SME box on the internet side NIC of ur SBS box?

so I am right in sharing \\servername\sme_bu for example and should be able to save files to it with a suitable username & pw setup on the SBS box?

I will have to look at logs over the weekend and see what it is doing....

[p-jones]

Absolutely not. the server is on the network side of SMEand internet services from the server, which are few) are accessed by port forwarding from sme

[stocky]

Sorry, I think you may have misunderstood...is you setup the same as the diagram i posted? eg SBS has dual NICs, and the SME box is on the internet side of the SBS box passing required traffic thru to the SBS box...

I cant seem to see my share from the internet side NIC of the SBS box, so I must have to change a setting somewhere.

[p-jones]

SBS Server is not multihomed. Only single NIC 0nly. SME ETH1 is the border entry point

[stocky]

ok, different setup....might enable ftp with IIS and try rsync and see if I can work that out.

Backing up will be nice, but not crucial, as the mail is all passed to exchange anyway and that is what is important.

thank you for your input      

[p-jones]

Why do you want to multihome your SBS ? turn that around - why have SME running Gateway - Server in this config. Consider server only on the internal network.   All my systems have everything sitting on eth0. Eth1 is STRICTLY perimeter security. Maybe you need to consider something like IPCOP which has a DMZ. I fear when you achieve your goal, you will also have compromised your security ...

[stocky]

The reason is the app provider for the SBS box requested the unit be configured that way.....I didnt like like the idea, but we had to work with them.

Might back up to a USB device and forget the sbs tape drive

[dmay]

any help or suggestions gratefully accepted!

A forum search would find this:

http://forums.contribs.org/index.php?topic=30224.0

Darrell

[stocky]

Thanks Darrell,

I hadnt got as far as identifying if it was a permissions issue yet to know what to search for!

Thanks for the tip, I will try those command lines and see what results I get and will post my results on that other thread for others to see as well.

     

[electroman00]

Maybe you need to consider something like IPCOP which has a DMZ.

Just wondering if you have a IPcop setup with SME...?

[p-jones]

I have an SME sitting behind and IPCop

[electroman00]

p-jones

Nice to here.

Since we started here with Smoothwall we have kinda stuck with it.
We have a few IPcops but most are Smoothwall.

We have a number of them running 24/7 for 2 1/2 + years.

Everyone I have seen with these setups had some sort of
compromise event which lead to the deployment of this type of firewall.

I suppose you also had similar such an event?

I guess I should ask what made you set one up?

[NickCritten]

Darrell,

I'd also like to add my thanks for an extremely well put together package.  backup2 is getting installed on all future servers

I'd tried backup2ws in the past but could never get it going for some reason!

I didn't think backup2's implementation of backup2ws was working for a while either, but its started working now.... For some reason I'd configured it to do a daily backup at midnight but it didn't start doing it for 2 days!  Oh well, its working now

I've yet to try out the backup2USB, but it'll be great once I've got it working... No more remote logins to perform manual backups...  YAY!

Great work Darrell, if you're ever in sunny Wales, I'll take you out for a Pint or 6  

[p-jones]

Electroman,

I needed a firewall. I read about ipcop and it fitted the bill more than perfectly. I wasnt going to get embroiled in the ipcop-smoothwall flame. SME is perfect for SOHO envirement  but does break a few rules by having server and firewall combined and it lacks a dmz. Thats not a criticism, it is a fact and it becomes horses for courses and each has a worthwhile role within the bigger picture. eg I have a group of users who use a db based on MS SQL. That cannot be changed. It requires a terminal server or an mssql server and it must live within a dmz. SME handles all the other server requirements (and better). SME has enough flexibility to slot into either enviroment...are you getting the general idea of where I am coming from ?

[NickCritten]

Darrell,

Once again, great contrib!

Two minor requests though...

1) Is is possible to make the name of the backup folders customisable i.e. instead of a random 6 digit number we could have 'daily_backup', or 'ibays_backup' something like that?

2) Any chance of changing (Or allowing an option to change) the filename format from American (MM-DD-YY) to the ISO8601 standard of (YYYY-MM-DD)

The former is confusing to anyone who isn't American, and doesn't name-sort correctly if you have backups over more than one year.

If either of these can be changed with a template then please give me the ritual kick in the head and I'll do it myself!  

Thanks Darrell!

[dmay]

1) Is is possible to make the name of the backup folders customisable i.e. instead of a random 6 digit number we could have 'daily_backup', or 'ibays_backup' something like that?

The answer to every 'Is it possible' is almost always 'yes'. However this would not be a minor rewrite. I do like your examples. To make this work easier spaces would best be prohibited. Currently the job number is very important (ie 911). I'd have to add some new fields to make this work. Drop job number and replace with job name. Add a 911 enable/disable field.

2) Any chance of changing (Or allowing an option to change) the filename format from American (MM-DD-YY) to the ISO8601 standard of (YYYY-MM-DD)

This is much easier to accomplish. Requires a line item edit to the backup2-backup action script replacing the current $Date entry %m.%d.%y with %Y.%m.%d. Note the capitalization. You would also need to manually edit the panel text.

I'm slowly... working on rewriting this contrib in FormMagick. I'll consider both of these for inclusion.

Darrell

[NickCritten]

Excellent Stuff,

I'll have a go at 'hacking' the second one myself, using your instructions and I eagerly await your rewrite!

[Daniel B.]

I'm looking for a backup solution on SME7rc1 and backup2 seems the best i tried, the scripts are very clear and quite easy to personalize (I say quite 'cause I'm a newbe). thanks darell for this contrib.
The reason I want to personalize it is that there's a problem if we want to backup files to an ibay of an other SME server: the share name is the name of the ibay (for example an ibay called backup) but we must write in the "backup/files" directory. So when I start the backup, it return an error.
I also heard there was an rsync support in older version of the contrib, and I'd like to have a look on it but I don't know where to find it. If someone could help me.
just a last question: darell, why did you use rar witch is non-free instead tar.gz?

[dmay]

backup files to an ibay

Read this thread. Don't use ibays for sme2sme.

http://forums.contribs.org/index.php?topic=31309.0

I also heard there was an rsync support in older version of the contrib?

I dropped this a long time ago. This is on the drawing board now:

http://bugs.contribs.org/show_bug.cgi?id=1093

why did you use rar witch is non-free instead tar.gz?

Search the BugTracker, forums & mailing list as this has been discussed many times before. Nothing currently matches RAR's features for workstation backup. If anyone finds anything GPL that will (and can prove it) I'll happily switch. This has been discussed in the BugTracker 7.x Future.

Darrell

[Daniel B.]

I had a look at the "backup2-backup" script , and I tried to had some features like:
 -backing up from or to an ibay
 -to a local directory
 -from or to an ssh location (only with RSA authentication)
 -from a smb share (to save some client files)
 With this, we could not only save files from the server but configure an SME server to save everything we want, even other server's config over the net. As I'm working in a office wich install and manage SME servers for SOHO, I think this could be usefull.
The script seems to work (I just made a few test, I'm still working on it and there're probably some bug) but I must modify the pannel. I've started to modify it but I heard that the e-smith::cgi module will be replaced by e-smith::Formmagik on futur versions, so I don't think spending time on that is interesting. I saw that you are rewriting it in FormMagik Darell. Don't know if you're interested in this features but if you're not, I'm interested in having a look at your new panel even if it's not finished 'cause I don't quite understand formmagik for now.

(sorry for the mistakes)

[kangkc]

Thanks to dmay for the wonder script. I have added in a "backup2dvd" options which basically mkisofs the rar backup file into iso image and burn that onto CD.

It works pretty well (with some hard coding of device) on CD-RW. I can backup and restore using CD-RW.

When I start testing with DVD,  it failed when writing. The problem is not with the script but rather the cdrecord package with CentOS.

Anyone has succesfully implemented using cdrecord to burn DVD on SME7RC1?  It works fine with CD but not with DVD. Tested with DVD-R and DVD-RW.

[p-jones]

kangkc, can you please share with us EXACTLY what you did to get the CD to work.
Thanks
Peter

[Tib]

Hi

I have been playing arround with backup2 and find it's very nice.

One question though ... is it possible to restore one file only ... but without the whole directory structure included.

Every time I restore to a different location it restores the whole diredtory structure as well.

Regards,

Tib

[NickCritten]

Hi

I have been playing arround with backup2 and find it's very nice.

One question though ... is it possible to restore one file only ... but without the whole directory structure included.

Every time I restore to a different location it restores the whole diredtory structure as well.

Regards,

Tib

Tib,

Just use WinRAR to open the rar file, then drag out the file you want, and manually copy back onto your SME using SCP / SFTP or drop into an Ibay using SMB.

I'm assuming of course that you use Windows on one of your client machines.. If not you are on your own!

Personally I use WinRAR on all my machines.. Its Waaaay better in my opinion than WinZip (And can handle just about every compression format I've ever heard of)  Plus it doesn't have that annoying habit of WinZips where it doesn't show you the directory tree when opening up an archive.

[Tib]

NickCritten

LOL ... thats a good point ... must be getting late ... I should have thought of that myself.

I backup to a linux machine not an XP machine ... I hate XP ... only use it when I really have to ... and that is probably about 5% of the time.

Linux of course has rar as well so there are no problems in that department.

Thanks,

Tib

[dmay]

Darell. Don't know if you're interested in this features but if you're not, I'm interested in having a look at your new panel even if it's not finished 'cause I don't quite understand formmagik for now.

(sorry for the mistakes)

I'd be happy to look at your additions. At the moment I'm 75% complete on my FM rewrite. Open a backup2 bug report and post your changes here:

http://bugs.contribs.org/enter_bug.cgi?product=SME%20Contribs

Darrell

[Daniel B.]

thx for your attention dmay. hope I filled the bug repport correctly, it's the first time for me.
If someone else want to try the modifications i've done, you can download http://firewall-services.com/sme/files/backup2ws.tar.gz
this archive contain two script one called backup2-backup and the other called backup2.
You can install the dmay contrib normally and replace the scripts /etc/e-smith/events/actions/backup2-backup and /etc/e-smith/web/functions/backup2 by those in the archive. there's no restor support for now, it's just for testing the backup featurs.
There's still some bugs, you're prevented. Any comment is wellcome.