Topic: affa vs backup pc vs dungog rsync backup

[crazybob]

I'm finally getting around to updating my backup server to SME 9.2. The issue is I have a number of client servers that I backup and use the dungog rsync program.
Looking at affa to replace dungog, Im not seeing if it does incremental backups. It appears backup pc does.
I am also concerned if either of these options will recognize the backup files I already have and just update them if changed or replaced 
Looking over the documentation I'm just not sure which way to jump.

[Jean-Philippe Pialasse]

You will have to decide

None will be able to accept your previous backups to the best of my knowledge.
You can keep the old aside untill they are not usefull anyymore.
The issue might be the time to sync the first tim if you are off site, or the available space to backup.

Honestly I would go with backup pc if you havemultiple server backing up to one :
- it can reduce disk size when multiple backup use similar file
- the restore option for one file or multiple file with web browser are fantastic.
- the report on the web interface are clear.
- it is highly maintained
- it just works !


Affa could be great with the rise option, but only if you use a 1:1 server /backup. It is only maintained here.

[crazybob]

Thank you for the info.

I do have an additional question about using affa to upgrade to 9.2. Can I safely bring the opt and var folders from the 8.2 server to the 9.2? I figured it would be easier to ask rather than have ti do it twice.

Thanks

[Jean-Philippe Pialasse]

you can, but if your issue is just to bring those two folders, just do a regular backup restore and rsync the two folder after. The advantage is to restore your sme fast, and then move the big data here that is not mandatory to get the server up and running.

Affa could be used to migrate between versions, but this is not the initial purpose and it can lead to few issues, including :
- wrong /changed uid for user or group leading to incorrect read write access for some deamons
- custom template and templates restored while the rpm of the contribs has not been installed back.

On my own I am using either backup pc and create via the cli a smeserver.tgz with all the needed folders, see the wiki page for some informaiton. Alternatively I have started using a script initially created by FWS here : https://wiki.fws.fr/tuto/ipasserelle/divers/mise_a_jour_sme8_vers_sme9

I have started updating this script with a few more contribs and plan to put it in a rpm. I could release an alpha version if you want. The advantage is to create a tgz similar to console backup with all in one. You just have then to reinstall repo for sme9, then reinstall contrib. My update create a list of installed rpm to yum install this list after migration. The disadvantage is it could take some hours to tar all the files, move the tgz and restore it. But if you have a night or a week end this is simple and effective.

[crazybob]

I would be interested in giving the rpm you mentioned a try.
What I am doing is not at a critical point at this time.
I'm trying to upgrade and hoping to not have to re download all the data from my clients servers. Trying to make this as seamless as possible. I'm playing with about 1 tb of data.
Thanks for the info.

[Bud]

guys please can you help

i have a client with sme 9.2 ( server mode ) including a centos 7 server.

i need to copy 4 x folders, including subfolders and two mysql databases from the centos 7 server to the sme 9.2 server as backups. i then need to send the zipped backups to google drive

i will also require a notice that the backup has taken place on the sme 9.2 server

what do you recommend i use?

affa vs backup pc vs dungog rsync backup vs dar vs rclone

thank you for your help 

[ReetP]

FYI I am chiselling away slowly at updating the rsync contrib. I'll update you when I have something working.

Another alternative that I have used for years is unison - basically 2 way rsync with some fancy options. You can set it to just do one way only if required.

I really ought to get round to writing a contrib to configure it, but it isn't hard...

I'm currently running unison version 2.40.102 which is in EPEL

http://www.cis.upenn.edu/~bcpierce/unison/download/releases/unison-2.40.102/unison-2.40.102-manual.html

[Jean-Philippe Pialasse]

I would be interested in giving the rpm you mentioned a try.
What I am doing is not at a critical point at this time.
I'm trying to upgrade and hoping to not have to re download all the data from my clients servers. Trying to make this as seamless as possible. I'm playing with about 1 tb of data.
Thanks for the info.

will do something by the end of the month, quite busy until the 27th.

then need to send the zipped backups to google drive

I might suggest https://wiki.contribs.org/Rclone for that part

for the other question it depends on your needs:
- multiple backups versions in case of disaster and or user doing wrong (backuppc, rsync) ,
- unique backup (rsync or unisson),
- not really a backup but rather a unique  synced copy (rsync or unisson),
- need to have an identical machine ready to replace the other in case of failure (Affa)

most tools would be able to be adapted for all the needs, but are more intended for a specific use, and the one I put in front of the situation, are most likely more adapted. For instance, backuppc will be able to keep a single version of the Centos7 box, but this would be overkill for that prupose, plus it would be stored in a way it is not directly accessible with cli, and thus might not be best option for sync after to a cloud .
Affa is more intended for SME <->SME or Centos<->Centos works, but it still could do, but again might be overkill for the need.

Rsync (not necessary the rsync contrib, but mostly a simple one liner script ) / unisson might be the more suitable from what I understand, but again you must clarify your needs in term of number of version, purpose of the copy (ies), and how you want to access it.

On top of that you might also need an ecryption procedure, at least for the cloud backup, rclone might be able to do the trick for you too

[Bud]

Jean-Philippe Pialasse and ReetP thank you for the valuable info, much appreciated

Unison is what i think i require

i have installed the Epel repository

please can you explain how do i install the Unison application in sme 9.2

thank you for your support 

[ReetP]

If you can wait til later I'll give you some instructions.

In the meantime a quick search on the interwebs shows lots of guides eg

https://www.masterkenneth.com/2016/06/07/unison-filefolder-sync-for-centosrhel-6/

Note I think you'll need a version number

Eg

yum --enablerepo=epel install unison240

Make sure you can do passwordless logins using ssh keys.

I'll try and paste some basic configs later... there are lots of options but most can be ignored/left as default.

If you are going to test then make sure you setup a dummy directory each end to play with before trying real data !!

May try to build a contrib to generate configs later this week... but trying to fix the rsync contrib currently !!

[ReetP]

OK, so here is some stuff from mine. This is on the 'central' server.

Some generic stuff applying to all profiles:

.unison/common

Code: [Select]

# Unison preferences file
#include name - include name profile
#auto - skip non conflicting entries
#auto = true
#batch No Questions. Non-conflicts propagated, conflicts are skipped
batch = true

#silent = true makes silent - sets batch to true automatically
#silent = true

#copythreshhold - size in kb above which it uses rsync
#copythreshold = 1000

# Individual log dir in profile
log = true

#times file mod times are propagated, but not directory times
times = true

# Careful here - make sure you have equivalent users/groups each end if you want to sync user & groups atributes

#group sync group attributes
#group = true

#owner sync owner attributes
owner = true

#numericsids When this flag is set to true, groups and users are synchronized numerically, rather than by name.
#numericids = true

rshargs = -C
sshargs = -C

# Some stuff to generically ignore
ignore = Name *.db
ignore = Name *~
ignore = Name .*~
ignore = Name .~*
ignore = Name *.~
ignore = Name .~*
ignore = Name *.tmp
ignore = Name ~*
ignore = Name Normal.dot
ignore = Name FIFO*
ignore = Name .Trash*
ignore = Name .git*
ignore = Name .komo*

Here's the individual profile:

.unison/oneway_sync.prf

Code: [Select]

# Include Unison preferences file
include common

# Log file for this profile
logfile = /var/log/unison/vps_sql.log

# We are going to pull all DB backups in the remote dir: /root/backup/db to our local dir: /root/backups/VPS
# (I do this from a number of server back to a central server)

# Root dir THIS end
root = //root/backups/VPS

# Root dir Far end
root = ssh://some.remote.server.com:22221//root/backup/db

# Which end do we want to be in control?
# See the differences between force and prefer here
force = ssh://some.remote.server.com:22221//root/backup/db

First run takes a while as it builds the sync data.

To run:

Code: [Select]

unison oneway_sync.prf
Using crontab I actually use a script to check that there are no other instances of unison running first - in the early days on occasion a sync would run and not be completed before the next run (depends on how often you run it and how much data there is). So this will check if the same script is already running.

/root/scripts/oneway_sycnc.sh

Code: [Select]

#!/bin/sh
# From here :
# http://www.franzone.com/2007/09/23/how-can-i-tell-if-my-bash-script-is-already-running/

if [ ! -z "`ps -C \`basename $0\` --no-headers -o "pid,ppid,sid,comm"|grep -v "$$ "|grep -v "<defunct>"`" ]; then
#script is already running - abort
exit 1
fi

# Quick check to make sure the remote end is vaguely up - nmap required
X=`/usr/bin/nmap -PN -p22221 some.remote.server.com |grep open`
       
if [ ! "$X" = "" ]; then
    /usr/bin/unison -silent oneway_sync.prf
fi


Simple crontab

Code: [Select]

#Crontab
02,22,42 * * * * root /root/scripts/oneway_sync.sh >>/dev/null 2>&1
Note you could also add some error reporting / mails etc based on the unison exit code.

HTH - please ask if you have questions, but DO read the unison documentation first !!

[Bud]

ReetP, thank you for your help

where will i find the unison application and folders that need to be edited on the sme 9.2 server?
i only require a one way copy/sync of folders and mysql databases from the centos 7 server to the sme 9.2 server

i then need to send the zipped backups to google drive

i will also require a notice that the backup has taken place on the sme 9.2 server

sorry for the noob question 

[ReetP]

unison should be /usr/bin/unison

Conf files (as previoulsy illustrated) go in /root/.unison

Scripts - wherever you choose.

Mail - you can add something to the cron script to check the unison output (see the documentation for the return statuses.

http://www.cis.upenn.edu/~bcpierce/unison/download/releases/unison-2.40.102/unison-2.40.102-manual.html#exit

I'm no guru on bash scripting but there is loads online on how to do it

HTH

[Stefano]

well, my2c:

install google's drive CLI agent on your centos7 machine (google will tell you more)
then you need a script which
- create a db dump (hint: mysqldump)
- create a compressed tar.gz file (hint1: tar, hint2: use YYYYMMDD_backup.tar.gz name)
- use google's client to push the file on remote gdrive and keep last N backup (optional: send an email when done; warning: gdrive doesn't erase deleted file, it's up to you)
- use scp to push files to your server and send an email when done.

believe me, there are thousands of examples out there on how to achieve such a thing..
HTH

[ReetP]

Stefano,

for whatever reason the OP wanted to bring the backups to the SME 9.2 box first (presumably so they can be backed up from that machine locally to say USB)

AS an alternative I guess he could backup the CentOS7 box direct to Googly place AND copy back to the SME box