Koozali.org: home of the SME Server

BackupPC

Offline MMorro@HP

  • *
  • 9
  • +0/-0
BackupPC
« on: August 12, 2015, 05:51:29 AM »
Thanks to all who had a hand in creating a fairly simple product for SME's.  I've used various Linux's over the years, but have never delved "too deeply" - not a noob, but no expert either. I have just setup SME 9 for home use - mainly to control Internet access for the family, but also to give us (the family) some options.

One such option was BackupPC.  I was able to get manual backups working fairly easily using SMB, but auto-backups and rsync were not so simple...

"Gotcha's" I found were (in no particular order):

1) Use full path for all commands

2) put any commands you want the BackupPC admin to run in sudoers
   - but use /etc/e-smith/templates/etc/sudoers templates as /etc/sudoers will be overwritten by the system

3) I could not get /usr/bin/wol (from the SME 9 Server) talking to or waking Windows boxes - but /usr/bin/ether-wake does work
   - /usr/bin/ether-wake requires either root or sudo rights

4) I put my "wake and ping" script in the Server config section (see /etc/BackupPC/config.pl) in place of '$PingPath -c 1 -w 3 $host'

5) never having used "rsync" before, I did not know it needed to be running on both Server and host to be backed up
   - I now have it installed as an auto-starting service on the Windows boxes.


I am happy to provide an example copy of my working script if anyone would like to see it - it's not necessarily the prettiest script you'll ever see, but it works.

Michael

Offline stephdl

  • *
  • 1,519
  • +0/-0
    • Linux et Geekeries
Re: BackupPC
« Reply #1 on: August 12, 2015, 07:30:10 AM »
thanks for your feedback, our community is made by all the knowledge that individually we bring in the project, so you are welcome to share it in the forum. Maybe we can enhance it, add some new values.
At the end, the final place for what you describe is the wiki page of BackupPC -> http://wiki.contribs.org/BackupPC

for example I added the way on how backup a linux server trough rsync, obviously we lack some examples for windows host and WOL :)

« Last Edit: August 12, 2015, 07:33:17 AM by stephdl »
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline MMorro@HP

  • *
  • 9
  • +0/-0
Re: BackupPC
« Reply #2 on: August 12, 2015, 08:24:35 AM »
Thanks Stephdl - and if you haven't received one for a while, thanks for your repo - quite useful

I have read that wiki - I used the example script under "then simply add a script to your prebackup command or your ping command like" to create my working script to wake and Backup 2 x Windows 7 desktops and 2 x Windows 8.1 Laptops at home.  That script did not work for me - the full path to "sudo", "ether-wake", "ping" and even "sleep" was required, as was placing the ping at the end - although that may work - I wasn't using the full path before I moved that...  Something to test...

I assume the wiki is controlled by people such as yourself - adding detail as necessary and proven.  I hope the detail I've provided will make it into the wiki and help others (maybe even save a sleepless night or two).

As an FYI, just having rsync running on both server and backup client was the start to solving the rsync issues I had (point 5) - the standard settings within the rsync portion of BackupPC together with creating an identical user & password on the Windows' side was enough to have communication - then using the info provided by Michael Stowe (http://www.michaelstowe.com/backuppc/) to make sure the rsync share could be read and understood from the Server side was enough to have a working backup.

I found placing the script in prebackup was too late; the backup seemed to have already failed - substituting a "wake-and-ping" script for the built-in $PingPath worked better for me (point 4).

I put those points up as I found little information myself about them - the usual "solve one problem, find another" - those 5 points were the main stumbling blocks, as was the need to "read between the lines" a bit.

Happy to edit out "the personal" and post my rsync config as well as my wake-and-ping script if you think it would prove useful.

Michael

Offline stephdl

  • *
  • 1,519
  • +0/-0
    • Linux et Geekeries
Re: BackupPC
« Reply #3 on: August 12, 2015, 10:49:55 AM »
Thanks Stephdl - and if you haven't received one for a while, thanks for your repo - quite useful
merci :)

Quote
I have read that wiki - I used the example script under "then simply add a script to your prebackup command or your ping command like" to create my working script to wake and Backup 2 x Windows 7 desktops and 2 x Windows 8.1 Laptops at home.  That script did not work for me - the full path to "sudo", "ether-wake", "ping" and even "sleep" was required, as was placing the ping at the end - although that may work - I wasn't using the full path before I moved that...  Something to test...
probably because the backuppc user doesn't get login and bash access (=> no path to /usr/bin)
Code: [Select]
backuppc:x:393:393::/var/lib/BackupPC:/sbin/nologin
Quote
I assume the wiki is controlled by people such as yourself - adding detail as necessary and proven.  I hope the detail I've provided will make it into the wiki and help others (maybe even save a sleepless night or two).

you should ask a wiki edit -> http://wiki.contribs.org/Help:Contents#How_to_get_a_wiki_account.3F and if you are not sure you can start to work on the talk page and ask to other for the reviewing of your work. We are not legion, each work of us is important for the community.
Please take time to read http://wiki.contribs.org/SME_Server:Volunteering and if you have questions we have an irc chat for helping and provide guidance (see my signature).

Code: [Select]
As an FYI, just having rsync running on both server and backup client was the start to solving the rsync issues I had (point 5) - the standard settings within the rsync portion of BackupPC together with creating an identical user & password on the Windows' side was enough to have communication - then using the info provided by Michael Stowe (http://www.michaelstowe.com/backuppc/) to make sure the rsync share could be read and understood from the Server side was enough to have a working backup.

I found placing the script in prebackup was too late; the backup seemed to have already failed - substituting a "wake-and-ping" script for the built-in $PingPath worked better for me (point 4).

I put those points up as I found little information myself about them - the usual "solve one problem, find another" - those 5 points were the main stumbling blocks, as was the need to "read between the lines" a bit.

Happy to edit out "the personal" and post my rsync config as well as my wake-and-ping script if you think it would prove useful.
As I said some chapters should be written on how to use rsynd on window hosts, even we should provide a link, or binaries for installing it quickly on the local network (@daniel)

Michael, please don't be shy :p
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline MMorro@HP

  • *
  • 9
  • +0/-0
Re: BackupPC
« Reply #4 on: August 12, 2015, 01:20:47 PM »
Well, until I read the Volunteering and get it done properly, this is what worked for me...  And, before I start, I know you knew you were right, but confirming, backuppc has no login...  As I said, not a noob, but not an expert  :-?

Some information came from several sources including Michael Stowe and the SME:Contribs, others just had to be worked out.

_________________________________________________________________________________________________________

#!/bin/bash

# this is the wake-ping script
# “HOST” is the supplied parameter
HOST=$1

# Set the mac address for each host
MAC1=34:64:a9:aa:bb:11          # laptop1
MAC2=34:64:a9:aa:bb:22          # laptop2
MAC3=50:e5:49:cc:dd:11          # desktop1
MAC4=50:e5:49:cc:dd:22          # desktop2
MAC5=50:e5:49:cc:dd:33          # desktop3

# Set the variables for each IP
PC1=laptop1
PC2=laptop2
PC3=desktop1
PC4=desktop2
PC5=desktop3

# How long does the host take to wake up
TIME=20

# interface to send the WoL packets from
INTERFACE=eth0

# was for testing - echo $HOST $INTERFACE
#                 - /bin/sleep 2

case $1 in
       $PC1)
               /usr/bin/sudo /sbin/ether-wake -i $INTERFACE $MAC1
               /bin/sleep $TIME
       ;;
       $PC2)
               /usr/bin/sudo /sbin/ether-wake -i $INTERFACE $MAC2
               /bin/sleep $TIME
       ;;
       $PC3)
               /usr/bin/sudo /sbin/ether-wake -i $INTERFACE $MAC3
               /bin/sleep $TIME
       ;;
       $PC4)
               /usr/bin/sudo /sbin/ether-wake -i $INTERFACE $MAC4
               /bin/sleep $TIME
       ;;
       $PC5)
               /usr/bin/sudo /sbin/ether-wake -i $INTERFACE $MAC5
               /bin/sleep $TIME
       ;;
       *)
               echo "Unknown machine $HOST"
               exit 1
       ;;
esac

# Function to check if the host pings
/bin/ping -w 5 -c 2 $HOST
export PINGSTATUS=$?
echo $PINGSTATUS

____________________________________________________________________________________________________


From the Main Configuration Editor->Backup Settings:



Client Lookup
ClientNameAlias  <BLANK>

NmbLookupCmd  $nmbLookupPath -A $host

NmbLookupFindHostCmd   $nmbLookupPath -B 192.168.1.255 $host

FixedIPNetBiosNameCheck   <BLANK>

PingCmd   /etc/BackupPC/wake-ping.sh $host

PingMaxMsec   20


Assumes a local LAN of 192.168.1.x

_______________________________________________________________________________________________________________________


Used cygwin-rsyncd-2.6.8_0.zip – configs required are:

Detail from rsyncd.conf - the heading in the square brackets becomes the rsync share.

[cdocs]
    path = c:/User
    comment = c: Drive documents
    auth users = validbackupuser
    secrets file = c:/rsyncd/rsyncd.secrets
    hosts allow = 192.168.1.1
    strict modes = false
    read only = true
    list = false


Detail from rsyncd.secrets has a valid local Userid:Password (for the Windows box being backed up) – in this instance the UserID would be validbackupuser

Command to install as rsync as a daemon and start:

cygrunsrv.exe -I rsyncd -e CYGWIN=nontsec -p c:/rsyncd/rsync.exe -a "--config=c:/rsyncd/rsyncd.conf --daemon --no-detach"

cygrunsrv.exe --start rsyncd


NB – different rsync version, different settings!

EG:

[cdocs]
  Path = /cygdrive/c/Users/<USERNAME>
  comment = c: User Date
  strict modes = false
  auth users = validbackupuser
  secrets file = c:/rsyncd/rsyncd.secrets
  hosts allow = 192.168.1.1
  read only = false
  list = false

____________________________________________________________________________________________________________________

All other settings are “fairly simple” by comparison.

Offline stephdl

  • *
  • 1,519
  • +0/-0
    • Linux et Geekeries
Re: BackupPC
« Reply #5 on: August 12, 2015, 02:23:47 PM »
As I said, not a noob, but not an expert  :-?

No problem for that, I will look your code next after my coding pensum -> smeserver-dhcpmanagerV2
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline stephdl

  • *
  • 1,519
  • +0/-0
    • Linux et Geekeries
Re: BackupPC
« Reply #6 on: August 13, 2015, 02:19:56 PM »
hi michael

just an interrogation relevant to

Code: [Select]
NmbLookupFindHostCmd   $nmbLookupPath -B 192.168.1.255 $host
is it possible to avoid to change that line, or if not, to add something that can find the brodcast address automatically

Code: [Select]
NmbLookupFindHostCmd   $nmbLookupPath -B $(/usr/bin/sudo /sbin/e-smith/config getprop InternalInterface Broadcast) $host
« Last Edit: August 13, 2015, 02:24:50 PM by stephdl »
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline MMorro@HP

  • *
  • 9
  • +0/-0
Re: BackupPC
« Reply #7 on: August 14, 2015, 01:20:05 AM »
Hi Stephdl,

I changed it because the "standard" (within the normal settings of config.pl) suggested there can be issues, and specifying the Broadcast will - a: work, no question; b: save Network traffic.  Searching for it will work too, but I suggest most of us using something like this Platform should know the Broadcast Address. 

I seem to recall reading BackupPC somewhere won't work over Wireless - some may have wireless networks too perhaps?  Maybe it's me - I always prefer to specify "known detail".

Offline stephdl

  • *
  • 1,519
  • +0/-0
    • Linux et Geekeries
Re: BackupPC
« Reply #8 on: August 14, 2015, 07:34:27 AM »
No problem I tought about if you change settings of your network.....any suggestions @others...i know that backupPC is used by many people
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline MMorro@HP

  • *
  • 9
  • +0/-0
Re: BackupPC
« Reply #9 on: August 17, 2015, 08:30:12 AM »
Hi Stephdl,

How are you?

Another thing I recalled from the reams of info I read - I don't believe you'd be able to run the command as you've written it - I seem to recall a shell script would need to be created and called, rather than trying to run "on the fly" as you have written - and of course read the "$host" parameter and then respond with whatever the perl script of BackupPC is expecting...

Offline MMorro@HP

  • *
  • 9
  • +0/-0
Re: BackupPC
« Reply #10 on: August 17, 2015, 08:34:22 AM »
Just found the info I was referring to in /etc/BackupPC/config.pl :

"NOTE: all Cmds are executed directly without a shell, so the prog name needs to be full pate and you can't include shell syntax like redirection and pipes; put that in a script if you need it"

Offline MMorro@HP

  • *
  • 9
  • +0/-0
Re: BackupPC
« Reply #11 on: August 17, 2015, 08:35:57 AM »
Sorry, should have read .... prog name needs to be full path...

Offline stephdl

  • *
  • 1,519
  • +0/-0
    • Linux et Geekeries
Re: BackupPC
« Reply #12 on: August 17, 2015, 09:45:32 AM »
thank for your input and clarification  michael, indeed I recall that I write in the backuppc configuration  the fixed IP of each client I have, therefore it is the same case of you, if I change my network, I will have something to do.

But changing a network is not something that we do each day, I hope so :)

can you write your work to the wiki please, if you need a hand for the wiki syntax, I will do

just an idea more, you should give the full url to download the cygwin-rsyncd

tips : if you want to generate a strong password you have randpw as a contrib in the wiki -> http://wiki.contribs.org/Random_Strong_Password_Generator
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline MMorro@HP

  • *
  • 9
  • +0/-0
Re: BackupPC
« Reply #13 on: September 11, 2015, 01:21:45 AM »
Hi Stephdl, All

I've added a "howto" to the BackupPC page - "Backup a Windows Client with rsync"

I'd appreciate some feedback on what I've explained and whether readable/worthwhile please.  I hope you find the information contained therein worthwhile.

Thanks

Michael

Offline stephdl

  • *
  • 1,519
  • +0/-0
    • Linux et Geekeries
Re: BackupPC
« Reply #14 on: September 11, 2015, 08:03:28 AM »
You should detail how is written the rsynd.secret file. You missed it
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!