Koozali.org: home of the SME Server

Solved! BackupPC- 4.4 Changing ssh port

Offline michelandre

  • *
  • 261
  • +0/-0
Solved! BackupPC- 4.4 Changing ssh port
« on: April 22, 2018, 06:20:46 PM »
Hi all

I am looking at BackupPC. I install according to https://bugs.contribs.org/show_bug.cgi?id=10514
- I didn't change "$Conf{PingPath} = "/bin/ping";" as it is OK in version 4.4.
- I didn't change "$Conf{RsyncSshArgs} = $sshPath -l root" as it's been corrected in version 4.4.

All my servers are using port 2222.
With BackupPC-3, I had no problem.
With BackupPC-4.4, I received this message:
Code: [Select]
...
Executing DumpPreUserCmd: /usr/bin/ssh -l root -p 2222 192.168.1.33 /sbin/e-smith/signal-event pre-backup
...
This is the rsync child about to exec /usr/bin/rsync_bpc
ssh: connect to host 192.168.1.33 port 22: Connection refused
rsync_bpc: connection unexpectedly closed (0 bytes received so far) [Receiver]
...
Executing DumpPostUserCmd: /usr/bin/ssh -l root -p 2222 192.168.1.33 /sbin/e-smith/signal-event post-backup
...

In Server-Manager, I see the pre and post-backup are executed. So, the ssh communication is working.
I thing the problem is with the rsync_bpc command as it is trying  port 22:.
I looked everywhere to change the port to no avail...
I tried with a old version 3 updated to version 4.4, and with a brand new installation of 4.4. Exact same problem.
I tried with a machine created with localserver-template and another one created with smeserver-template: same problem.

Any help appreciated,

Michel-André

« Last Edit: April 27, 2018, 03:53:12 AM by michelandre »

Offline michelandre

  • *
  • 261
  • +0/-0
Re: BackupPC- 4.4 Changing ssh port
« Reply #1 on: April 23, 2018, 04:01:44 AM »
Hi all,

Total victory!

After reading:
https://sourceforge.net/p/backuppc/mailman/message/14217521/
and:
https://www.digitalocean.com/community/tutorials/how-to-configure-custom-connection-options-for-your-ssh-client

The solution is to create the file: /var/lib/BackupPC/.ssh/config
Code: [Select]
Host 192.168.1.33
Port 2222

or:
Code: [Select]
Host 192.168.1.33/24
Port 2222

or:
Code: [Select]
Host *
Port 2222


It took me a long time/search but I found the solution.

Michel-André


Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: SOLVED! BackupPC- 4.4 Changing ssh port
« Reply #2 on: April 23, 2018, 04:13:42 AM »
this is not the solution but one of them.


As you see your pre and post backup command are running fine, because you specified the port where you overrides the default for this host

Code: [Select]
Executing DumpPreUserCmd: /usr/bin/ssh -l root -p 2222 192.168.1.33 /sbin/e-smith/signal-event pre-backup
Code: [Select]
Executing DumpPostUserCmd: /usr/bin/ssh -l root -p 2222 192.168.1.33 /sbin/e-smith/signal-event post-backup

we do not see the content of the log with the backup command but you indeed should override  the value of $Conf{RsyncSshArgs} for this host

Code: [Select]
$sshPath -l root
to
Code: [Select]
$sshPath -l root -p 2222

Offline michelandre

  • *
  • 261
  • +0/-0
Re: SOLVED! BackupPC- 4.4 Changing ssh port
« Reply #3 on: April 23, 2018, 04:33:23 AM »
Hi Jean-Philippe,

Thank you for your reply.

I tried that in the configuration page under Xfer tab and it gave me: "error in the path" when I SAVE.

Also, I read in a file about BackupPC parameters someting like: port=PORT # ...double-colon...

So, I tried in different places in that tab and it didn't do the job but it didn't complain about malformation.
I tried port=2222, port 2222, and even port:2222 in the inserted field to no avail.

Michel-André

Offline michelandre

  • *
  • 261
  • +0/-0
Re: SOLVED! BackupPC- 4.4 Changing ssh port
« Reply #4 on: April 23, 2018, 04:49:24 AM »
Hi again Jean-Philippe,

I found where I saw that port=PORT. It is with:

Code: [Select]
# rsync_bpc --help
...
--port=PORT             specify double-colon alternate port number
...

Michel-André

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: SOLVED! BackupPC- 4.4 Changing ssh port
« Reply #5 on: April 23, 2018, 06:20:07 AM »
those are options for the rsync command. The port should not be defined for the rsync command but for the ssh command.

the potential issue is that the ssh command might need to be encapsulated and it is not  in the final command
Code: [Select]
rsync -e "/usr/bin/ssh -l root -p 2222 " myserveradress:/folders /var/backuppc/pc/mybackups
without the quotes,-l root and   -p 2222 are interpreted as part of rsync command not the ssh one

but you need to check for the complete command in the log.

Offline michelandre

  • *
  • 261
  • +0/-0
Re: SOLVED! BackupPC- 4.4 Changing ssh port
« Reply #6 on: April 23, 2018, 04:22:36 PM »
Hi Jean-Philippe,

Thank you for your reply.

You are right about the "between_double_quote". It will only apply to the ssh part of the rsync command(s).
Modifier la configuration >  Xfer > RsyncSshArgs   $sshPath -l root "-p 2222"

The problem with it though, is that it applies only to rsync and not the other ssh commands.
So the pre-backup and post-backup are refused as they will be issuing with port 22:

Code: [Select]
2018-04-23 09:13:55 Output from DumpPreUserCmd: ssh: connect to host 192.168.1.33 port 22: Connection refused
2018-04-23 09:13:55 incr backup started for directory /
2018-04-23 09:24:18 Output from DumpPostUserCmd: ssh: connect to host 192.168.1.33 port 22: Connection refused
2018-04-23 09:24:18 incr backup 7 complete, 84726 files, 3326676343 bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)

It implies that I have to also add -p 2222:
Modifier la configuration >  Paramètres de sauvegarde > DumpPreUserCmd, DumpPostUserCmd, and RestorePreUserCmd
Code: [Select]
$sshPath -l root -p 2222 $host /sbin/e-smith/signal-event pre-backup
$sshPath -l root -p 2222 $host /sbin/e-smith/signal-event post-backup
$sshPath -l root -p 2222 $host /sbin/e-smith/signal-event pre-restore

I am a very lazy admin and prefer to create a 2 lines file instead of modifying 4 parameters... :lol:

Also, if each of your servers uses different ssh port than the others, the config file can take care of it.

Michel-André




Offline michelandre

  • *
  • 261
  • +0/-0
Re: BackupPC- 4.4 Changing ssh port
« Reply #7 on: April 26, 2018, 06:02:36 PM »
Hi all,

The victory is not so complete... :sad:

I have a problem with remote backup.

I use VirtualBox and recreate both BackupPC host and remote. Pure SME without contribs and I did the SME updates.

The host backups always work.

The first remote backup works fine.
The second one always fails for both full and incremental.
The ssh connection times out.
I reboot.
The next backup works. All the others have ssh port time out.

When the ssh times out, I cannot ssh from the BackupPC host to remote.
I cannot ssh from WinSCP.

It looks that if I wait a long time, all is OK so I think that it is a locked out from ssh???

I was thinking that it is related to VirtualBox (I have the latest) because sometime after a boot, I cannot ssh to the machine. A reboot always solves the problem. I notice this behaviour since a long time...

Any hellp appreciated.

Michel-André

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com

Offline michelandre

  • *
  • 261
  • +0/-0
Re: BackupPC- 4.4 Changing ssh port
« Reply #9 on: April 26, 2018, 09:41:44 PM »
Hi Jean-Philippe,

Thank you so much, your're the greatest!

I would like to disable it before the backup and enable it after.
How and where can I do that with a pre/post-backup command?

pre-backup

Code: [Select]
config setprop sshd AutoBlock disabled
signal-event remoteaccess-update
sleep 3

post-backup
Code: [Select]
config setprop sshd AutoBlock enabled
signal-event remoteaccess-update
sleep 3

Maybe combining the commands in a script: pre-backup-command.sh?

Code: [Select]
$sshPath -l root $host /sbin/e-smith/signal-event pre-backup
config setprop sshd AutoBlock disabled
signal-event remoteaccess-update
sleep 3

Then the DumpPreUserCmd can be:
Code: [Select]
[path_of_the_script]/pre-backup-command.sh
- Will the $sshPath still be valid inside the script?
- Where will be the best place to store this script?
- Is it necessary to include a sleep delay to give some time for remoteaccess-update


Again, thank you so much,

Michel-André


Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: BackupPC- 4.4 Changing ssh port
« Reply #10 on: April 26, 2018, 09:47:02 PM »
just disable  it and put either denyhosts or fail2ban using one of the contribs.

they are doing almost the same but allow whitelisting

Offline michelandre

  • *
  • 261
  • +0/-0
Re: BackupPC- 4.4 Changing ssh port
« Reply #11 on: April 27, 2018, 03:52:38 AM »
Hi again Jean-Philippe,

Hoist the victory banner! :D

- The variables from BackupPC are not available inside the script.
- Instead, I passed it as a parameter with the calling script.
- signal-event remoteaccess-update takes a little bit more than a second to execute.

$DumpPreUserCmd = /var/lib/BackupPC/script/pre-backup.sh $host

/var/lib/BackupPC/script/pre-backup.sh

Code: [Select]
#!/bin/sh
/usr/bin/ssh -l root $1 /sbin/e-smith/signal-event pre-backup
/usr/bin/ssh -l root $1                  \
"config setprop sshd AutoBlock disabled; \
signal-event remoteaccess-update;        \
sleep 3"
# Check if AutoBlock=disabled  ## to be done with egrep
exit 0

$DumpPostUserCmd = /var/lib/BackupPC/script/post-backup.sh $host

/var/lib/BackupPC/script/post-backup.sh

Code: [Select]
#!/bin/sh
/usr/bin/ssh -l root $1 /sbin/e-smith/signal-event post-backup
/usr/bin/ssh -l root $1                  \
"config setprop sshd AutoBlock enabled;  \
signal-event remoteaccess-update;        \
sleep 3"
# Check if AutoBlock=enabled  ## to be done with egrep
exit 0

Thanks to you Jean-Philippe. May the cloud of happiness float over your head!

Michel-André


Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Solved! BackupPC- 4.4 Changing ssh port
« Reply #12 on: April 30, 2018, 01:44:32 PM »
Honestly this won't for sure prevent you from being locked up. This might work most of the time, but if for any reason the first ssh fails, bbpc will give a second try very fast and trigger the ban before disabling it.

Also your backup will run for one hour or so without any ban system enforcing. So either you are happy without any ban for one hour and i see no reason to not disable it all the time.
Either you are not happy to be without this protection, and then as I said choose to install fail2ban or denyhost in place of the autoblock, and set their whitelist option.

So again, while the shell writting exercise was interesting, I strongly suggest you to review the alternatives.

Offline michelandre

  • *
  • 261
  • +0/-0
Re: Solved! BackupPC- 4.4 Changing ssh port
« Reply #13 on: April 30, 2018, 07:14:32 PM »
Hi Jean-Philippe,

Thanks for the advices,

I am currently writing a howto for BackupPC for my clients. I have another howto for Fail2ban but it is not up to date. Later, I will also have to do one for WBL.

As you say, it's impossible to connect a USB device to a virtual machine for a restore during an installation and I'm doing some tests for a complete restore without USB.
Testing is complete for a restore using SSH commands from the BackupPC host server. It is no longer necessary to transport the TAR file. Everything is functional but, the latest signal-events (post-upgrade and reboot) require access to the console of the remote server.
The reconfiguration of network cards requires also to be at the remote console ... I will look at manipulating /etc/udev/rules.d/70-persistent-net.rules to solve that. Maybe save the file before restauration and put it back after but, there are also files that have to be deleted also...

For now, I will use the script below.

Code: [Select]
#!/bin/sh
/usr/bin/ssh -l root $1 /sbin/e-smith/signal-event pre-backup
/usr/bin/ssh -l root $1                         \
"config setprop sshd AutoBlock disabled;        \
/sbin/e-smith/signal-event remoteaccess-update"

# If the command: "config setprop sshd AutoBlock disabled" is not working,
# this test will stop the current backup and disable them for this machine for 24 hours.
#
# Réf: https://www.mail-archive.com/backuppc-users@lists.sourceforge.net/msg10075.html
# Stop/delay backups
#     BackupPC_serverMesg stop HOST REQ_USER BACKOFF
# HOST is the host to cancel, REQ_USER is the requesting user (just for
# log purposes) and BACKOFF is the backoff time in hours.

# This test will also write the status of AutoBlock in the log for this BackupPC machine.
# "Output from DumpPreUserCmd:     AutoBlock=disabled"
# or
# "Output from DumpPreUserCmd:     AutoBlock=enabled"

if  ( ! (/usr/bin/ssh -l root $1 config show sshd | grep  'AutoBlock=disabled') )
    then
        /bin/mail -s "AutoBlock=disabled NOT FUNCTIONAL, stopping backups of the machine $1 for 24 hours" admin@domain.com < /dev/null
        /usr/share/BackupPC/bin/BackupPC_serverMesg stop $1 backuppc 24
fi

Thank you again, I will look at the alternatives,

Michel-André

Offline michelandre

  • *
  • 261
  • +0/-0
Re: Solved! BackupPC- 4.4 Changing ssh port
« Reply #14 on: May 01, 2018, 11:25:12 PM »
Hi all,

Finally good news. Now everything is working properly.

In the script just above, I had removed the "sleep 3" since a server have to finish the command before returning to the console. I did not see the need to include these "sleep 3". Subsequently, once in a while a backup will not work properly.

I followed Jean-Philippe's advice and I installed Fail2ban and found that my complete network was in the "ignoreip" as defined by Fail2ban defaults. So everything should work fine now.

Alas! it was worse than before. I could not do any backups at all. I could log in with WinSCP and PuTTY but BackupPC always had a "connection timed out". However, the good part of it was that the "if ... then" worked perfectly.

I checked and rechecked Fail2ban without success until I thought back to that "sleep 3". So I put back this "sleep" but with 5 seconds and after each command in the script. Miracle! Everything started working with absolutely every backups without any errors.

Changing the port is done by creating a file named: /var/lib/BackupPC/.ssh/config
Code: [Select]
# Spécification du port SSH à être utilisé par BackupPC
Host *
    Port 2222

pre-backup script called by:  /var/lib/BackupPC/script/pre-sauvegarde.sh $host
Code: [Select]
#!/bin/sh
/usr/bin/ssh -l root $1                  \
"/sbin/e-smith/signal-event pre-backup;  \
sleep 5"
/usr/bin/ssh -l root $1                  \
"config setprop sshd AutoBlock disabled; \
signal-event remoteaccess-update;        \
sleep 5"

#
# Le test ci-dessous va écrire le statut de la propriété AutoBlock dans le journal.
# Output from DumpPreUserCmd:     AutoBlock=disabled
# ou
# Output from DumpPreUserCmd:     AutoBlock=enabled
#

if  ( ! (/usr/bin/ssh -l root $1 config show sshd | grep  'AutoBlock=disabled') )
    then
        /bin/mail -s "AutoBlock=disabled NON FONCTIONNEL, arrêt des sauvegardes de la machine $1 pour 24 heures" admin@toto.com < /dev/null

#    Arrêt de la sauvegarde en cours et désactivation des sauvegardes de cette machine
#    pour 24 heures.
#
#    Référence:
#    https://www.mail-archive.com/backuppc-users@lists.sourceforge.net/msg10075.html
#    Stop/delay backups
#        BackupPC_serverMesg stop HOST REQ_USER BACKOFF
#    HOST is the host to cancel, REQ_USER is the requesting user (just for
#    log purposes) and BACKOFF is the backoff time in hours.
#

    /usr/share/BackupPC/bin/BackupPC_serverMesg stop $1 backuppc 24
fi

post-backup script called by: /var/lib/BackupPC/script/post-sauvegarde.sh $host
Code: [Select]
#!/bin/sh
/usr/bin/ssh -l root $1                  \
"/sbin/e-smith/signal-event post-backup; \
sleep 5"
/usr/bin/ssh -l root $1                  \
"config setprop sshd AutoBlock enabled;  \
signal-event remoteaccess-update;        \
sleep 5"

#
# Le test ci-desous va écrire le statut de AutoBlock dans le journal.
# Output from DumpPreUserCmd:     AutoBlock=disabled
# ou
# Output from DumpPreUserCmd:     AutoBlock=enabled
#

if  ( ! (/usr/bin/ssh -l root $1 config show sshd | grep  'AutoBlock=enabled') )
    then
        /bin/mail -s "AutoBlock=ensabled NON FONCTIONNEL. Vérifiez le paramètre s.v.p." admin@toto.com < /dev/null
fi

In Xfer, I excluded /etc/udev/rules.d/70-persistent-net.rules and /etc/fstab. So no more ajusting the network cards after the restauration.

The restoration is a marvel. It is not necessary to install the same Contribs as in the Source server. They are all contained in the restoration.

* At the console of the BackupPC server, just create the TAR file.

Code: [Select]
/usr/share/BackupPC/bin/BackupPC_tarCreate  -h 192.168.1.152 -n 3 -s /  / > serveur-152-3.tar
* At the console of the destination server, only 4 lines of commands for the restoration and it is very fast:
Code: [Select]
cd /
signal-event pre-restore
ssh -p 2222 root@192.168.1.33 "cat /var/lib/BackupPC/mes_restaurations/serveur-152-3.tar" | tar -xvv
signal-event post-upgrade ; signal-event reboot

I conclude that a bug has slipped into ssh and should be related to some lack of time delay somewhere; I do not know because I'm not a programmer but...

What do the experts think?

Michel-André
« Last Edit: May 01, 2018, 11:52:04 PM by michelandre »