Koozali.org formerly Contribs.org

Check if Qpsmtpd:check smtp forward is working on delegated email setup

Hi folks,
We have our SME server delegating email to an internal exchange server but are generating a lot of bounces and have been blacklisted by the 'backscatter' list.

Looking in the /var/log/qpsmtpd/ logs I can see incoming email being accepted to addresses that don't exist on our exchange server, I expect these are generating bounces. IIRC this is meant to be stopped when using a delegated mail server by the check_smtp_forward plugin?

Checking the configs in /var/service/sqpsmtpd/config/peers they all appear to have "check_smtp_forward" enabled.
(there are several files including '0', 'local' and several links with file names as IP addresses linked to '0' and 'local')

Any hints as to where to check next?

below is the contents of /var/service/sqpsmtpd/config/peers/0

Thanks for you help.
Regards,
Mark Leman


Code: [Select]
#------------------------------------------------------------
#              !!DO NOT MODIFY THIS FILE!!
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at http://www.contribs.org/development/
#
# Copyright (C) 1999-2006 Mitel Networks Corporation
#------------------------------------------------------------

logging/logterse
tls ssl/cert.pem ssl/cert.pem ssl/cert.pem ssl/dhparam.pem
auth/auth_cvm_unix_local cvm_socket /var/lib/cvm/cvm-unix-local.socket enable_smtp yes enable_ssmtp yes
# Karma plugin is disabled
earlytalker
bogus_bounce
count_unrecognized_commands 4
# bcc disabled
relay
helo policy lenient reject naughty
resolvable_fromhost
headers
loadcheck max_load 7
# rhsbl disabled
sender_permitted_from reject 1 no_dmarc_policy 1
dkim reject 0
dmarc reject 1 reporting 1
# dnsbl disabled
naughty reject mail
# uribl disabled
badmailfrom
# 34badrcptto disabled for external connections
badrcptto more_badrcptto badrcptto_ext
check_smtp_forward


rcpt_ok
# pattern_filter disabled

tnef2mime

# 65disclaimer disabled for external connections
spamassassin munge_subject_threshold 5
virus/clamdscan scan_all yes clamd_socket /var/clamav/clamd.socket defer_on_error yes max_size 25000
queue/qmail-queue


Offline ReetP

  • *
  • 2,534
Looking at the template/etc/e-smith/templates/var/service/qpsmtpd/config/plugins/37check_smtp_forward we can see that check_smtp_forward is enabled if DelegateMailServer is enabled, or {$domain}MailServer is set.

If you read the code in /usr/share/qpsmtpd/plugins/check_smtp_forward you can see this. (The $server value is obtained from smtproutes.)

Code: [Select]
my $smtp = Net::SMTP->new(
                            $server,
                            Port => $port,
                            Timeout => 60,
                            Hello => $self->qp->config("me"),
                           ) || return (DENYSOFT,
                                        "Unable to connect to $server: $!");

That sets up a connection to your delegated server (not SME)

It then tries to see if that user is accepted on the server and if it isn't it gets DENY. And that appears to be working as designed.

SME is blocking mail because Exchange won't accept it. SME can't check against any accounts internally as the mail is processed on Exchange.

On Exchange you probably ought to accept mail to unknown addresses and send it to an admin/junk account for later checking (this is what you would do normally on SME).

This prevents exactly this sort of issue, especially if you get hit by a lot of junk that also have duff return addresses.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Hi,
Thanks for the swift reply.

The SME server has been working fine for many months, I have the internal delegate mail servers IP address set. SME server is successfully delivering mail to it.

I just can't see any evidence that qpsmtpd is doing the check_smtp_forward test.
And this is allowing many emails to arrive at the exchange server for non-existent users, which are then bouncing. I could set exchange to drop them but I would prefer to not waste the bandwidth receiving them in the first place.

I expected that check_smtp_forward would reject the emails due the recipients address before the SME server had actually received it, and it was only today when I had a dig in the qpsmtpd logs that I could not find evidence to show it doing this.

Oddly I know my old 7.6 server did because I asked another forum question in 2013 about it :-) https://forums.contribs.org/index.php/topic,49595.msg247989.html#msg247989

........on further investigation........
I did a quick
Code: [Select]
grep "check_smtp_forward" /var/log/qpsmtpd/current | tai64nlocal

And it did show couple of entries all grouped within a few seconds today, here is one
Code: [Select]
2020-07-07 14:54:37.872340500 18093 (deny) logging::logterse: ` 80.12.70.35     relais-inet.orange.com  relais-inet.orange.com  <web.clients@orange.com>                check_smtp_forward      902     Unable to connect to 10.42.0.3:         msg denied before queued

So it looks to me like check_smtp_forward can reject emails when the internal exchange server is momentarily offline.

Which makes me think this could be an issue with the config of the exchange server, I wonder if it is configured to accept all email addresses and then generate bounces. I will go and visit the rather unpleasant exchange web config interface and report back.

Regards,
Mark Leman


Offline mmccarn

  • *
  • 2,518
You can test your overall config by sending an email to a local non-existent user from any external email address you control (gmail or hotmail, for example). You would then get to see the bounce message which should have enough info in the headers to tell you exactly where the bounce is being generated.

Do you have a backup MX record configured?  These frequently accept all email and end up generating bounce messages when the email is collected.

Offline ReetP

  • *
  • 2,534
Looking in the /var/log/qpsmtpd/ logs I can see incoming email being accepted to addresses that don't exist on our exchange server, I expect these are generating bounces. IIRC this is meant to be stopped when using a delegated mail server by the check_smtp_forward plugin?

In the cool calm light of day after a long night wrangling codey stuff.... :-)

Yes, recipients should be rejected if the check_smtp_forward plugin has checked with the Delegate server and the recipient does not exist.

Quote
Looking in the /var/log/qpsmtpd/ logs I can see incoming email being accepted to addresses that don't exist on our exchange server, I expect these are generating bounces. IIRC this is meant to be stopped when using a delegated mail server by the check_smtp_forward plugin?

Yes..... hmm - I wonder why it isn't?

Just hacked from the plugin this as a bit of test code you can try and see what is occurring.

Copy it to a file, set a dummy incoming server, set your delegate Exchange server, a From and To address and then manually run it with perl myTest.pl or whatever.

Code: [Select]
#!/usr/bin/perl
use Net::SMTP;

my $server = 'your.delegate.server';
my $port = "25";

my $incomingserver = 'incoming.remote.server';
my $senderaddress = 'someone@gmail.com';
my $recipientaddress = 'user@yourrecipientdomain.com';


my $smtp = Net::SMTP->new(
                    $server,
                    Port => $port,
                    Timeout => 60,
                    Hello => $incomingserver,
                    SSL => 1,
                    Debug => 1,
                   ) || die "Unable to connect to $server: $!\n";

# From
  $smtp->mail($senderaddress);
  my $rc = $smtp->code;
  my $message = $smtp->message;
  chomp($message);
 
 print "Message Sender: $message\n";
 
 print "RC1 $rc\n";
 
   if ($rc =~ m/^4\d{2}$/ ) {
print "1 Unable to queue $message\n";
  } elsif ($rc =~ m/^5\d{2}$/ ) {
print "2 Unable to queue $message\n";
  }
 
# To
  $smtp->to($recipientaddress);
  $rc = $smtp->code;
  $message = $smtp->message;
  chomp($message);

 print "Message Recipient: $message\n";
 print "RC2 $rc\n";
 
  if ($rc =~ m/^4\d{2}$/ ) {
        print "3 Unable to queue $message\n";
  } elsif ($rc =~ m/^5\d{2}$/ ) {
        print "4 Unable to queue $message\n";
  }

  $smtp->quit();
  $rc = $smtp->code;

print "RC end $rc\n";

That should tell you what SME is seeing from Exchange.

Good connection:
Code: [Select]
Net::SMTP=GLOB(0xeb0b78)>>> RCPT TO:<john@mydomain.com>
Net::SMTP=GLOB(0xeb0b78)<<< 250 <john@mydomain.com>, recipient ok
Message Recipient: <john@mydomain.com>, recipient ok

User not known and rejected:

Code: [Select]
Net::SMTP=GLOB(0x17bdb88)>>> RCPT TO:<nonexistentuser@mydomain.com>
Net::SMTP=GLOB(0x17bdb88)<<< 550 relaying denied nonexistentuser@mydomain.com
Message Recipient: relaying denied nonexistentuser@mydomain.com
RC2 550
4 Unable to queue relaying denied nonexistentuser@mydomain.com


...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

mmccarn & ReetP,
Thank you both for your responses, they have helped me to work out cause of the issue.

I did a manual SMTP check from my SME box to the internal exchange server port 25, using  RCPT TO: a non existent user.

Code: [Select]
[root@sme]# telnet 10.42.0.3 25
Trying 10.42.0.3...
Connected to 10.42.0.3.
Escape character is '^]'.
220 SVR02.mydomain.co.uk Microsoft ESMTP MAIL Service ready at Wed, 8 Jul 2020 11:22:04 +0100
EHLO mydomain.co.uk
250-SVR02.mydomain.co.uk Hello [10.42.0.10]
250-SIZE 52428800
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-STARTTLS
250-X-ANONYMOUSTLS
250-AUTH NTLM
250-X-EXPS GSSAPI NTLM
250-8BITMIME
250-BINARYMIME
250-CHUNKING
250 XRDST
MAIL FROM: <me@my-my-home-email-account.co.uk>
250 2.1.0 Sender OK
RCPT TO: <non-existent-user@mydomain.co.uk>
250 2.1.5 Recipient OK


And it accepted it! Which I was not expecting, it certainly did not last time I acctually checked, but that would probably have been many years/versions of exchange ago.

So I did a bit of googling and found https://manuals.gfi.com/en/oneconnect/content/admin/settings/domain/recipientverificationexchange.htm

Which says
Quote
Enabling Recipient Verification in Microsoft Exchange.....
In Exchange 2013 Microsoft recipient checking is done after DATA reception. This means even if the recipient validation is enabled on the mail server, any recipient check gets a "250 OK" response for invalid recipients.

To work around this problem you need to enable “Anonymous Users” on the Default Hub Transport connector and access the server on port 2525. In this way, invalid recipients are rejected after they are specified using the RCPT TO command.

And goes on to give instructions, which I followed for the 2013/2016 version of exchange we now use, and now if I do a manual SMTP check to echange on port 2525:

Code: [Select]
[root@sme]# telnet 10.42.0.3 2525
Trying 10.42.0.3...
Connected to 10.42.0.3.
Escape character is '^]'.
220 SVR02.mydomain.co.uk  Microsoft ESMTP MAIL Service ready at Wed, 8 Jul 2020 11:15:06 +0100
EHLO mydomain.co.uk
250-SVR02.mydomain.co.uk  Hello [10.42.0.10]
250-SIZE
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-STARTTLS
250-X-ANONYMOUSTLS
250-AUTH NTLM
250-X-EXPS GSSAPI NTLM
250-8BITMIME
250-BINARYMIME
250-CHUNKING
250-XEXCH50
250-XRDST
250 XSHADOWREQUEST
MAIL FROM: <me@my-my-home-email-account.co.uk>
250 2.1.0 Sender OK
RCPT TO: <non-existent-user@mydomain.co.uk>
550 5.1.1 User unknown

Which is much better for Qpsmtpd:check smtp forward to check against :-)

Which just leaves me with the question:

How do I tell SME server to use port 2525 on my delegated email server?

Regards,
Mark Leman

« Last Edit: July 08, 2020, 02:05:55 PM by markleman »

Just hacked from the plugin this as a bit of test code you can try and see what is occurring.

Copy it to a file, set a dummy incoming server, set your delegate Exchange server, a From and To address and then manually run it with perl myTest.pl or whatever.

Thanks for the test code, I will file that away for future use, certainly easier to repeat than my usual telnet/cut-n-paste from a pre-prepared list of smtp commands :-)

Regards,
Mark Leman

Offline ReetP

  • *
  • 2,534
Which just leaves me with the question:

How do I tell SME server to use port 2525 on my delegated email server?

Read the plugin code and see what it says....

Code: [Select]
  my $server = $self->{_smtp_host}{$host} or return (DECLINED);
  my $port;

  ($server, $port) = split(/:/, $server);
  $port ||= 25;

  $self->log(LOGDEBUG, "Checking $recipient on $server:$port");

This in particular then

Code: [Select]
($server, $port) = split(/:/, $server);
So I guess you can set the DelegateMailServer as:

my.delegateserver.com:2525

Give it a try and see.... you could add a bit to my code to test as well.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Read the plugin code and see what it says....

Code: [Select]
  my $server = $self->{_smtp_host}{$host} or return (DECLINED);
  my $port;

  ($server, $port) = split(/:/, $server);
  $port ||= 25;

  $self->log(LOGDEBUG, "Checking $recipient on $server:$port");

Good thinking, I was assuming I needed to changing the entire delegate email details, but it is actually only the port for the check_smtp_forward plugin that needs changing. Once the check_smtp_forward test has passed using port 2525 the email can still be delivered to port 25 to the exchange server.

Possibly a less 'general' solution but it will fix my immediate need.

Off to visit the code and templates :-)

Thanks again for the help.
Regards,
Mark Leman
« Last Edit: July 08, 2020, 03:21:29 PM by markleman »

Then I noticed I had not read your reply properly and the port can be set as part of the delegate mail server <blush>
Off of try that as you suggested :-)

Offline ReetP

  • *
  • 2,534
Then I noticed I had not read your reply properly and the port can be set as part of the delegate mail server <blush>
Off of try that as you suggested :-)

Easily done.

I misread your original post !!!!

Let us know if it works OK

If you want to see v10 released then please come and chat to us on our Rocket.Chat instance and help test.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Some progress....
I tried entering the server:port in the server-manager web interface but it complained "This field must contain a valid IP number and can not be left blank."

So I used commands:
Code: [Select]
config set DelegateMailServer 10.42.0.3:2525
signal-event email-update

Then I tried sending email:
  • to myself - that arrives correctly
  • to an invalid address -  that bounces from SME
  • to myself & an invalid address -  that is delivered to me and also generates a bounce from SME
The bounce is like this
Code: [Select]
Hi. This is the qmail-send program at mydomain.co.uk.
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.

<non-existent-user@mydomain.co.uk>:
10.42.0.3 does not like recipient.
Remote host said: 550 5.1.1 User unknown Giving up on 10.42.0.3.
STARTTLS proto=TLSv1.2; cipher=ECDHE-RSA-AES256-SHA384; subject=/CN=SVR02; issuer=/CN=SVR02;
....

However I had been expecting check_smtp_forward to block the email at the mail header stage.

I still don't think check_smtp_forward is working (or is failing to work correctly with the exchange server)

I have set the delegated mail server port back to 25 and then manually edited the /usr/share/qpsmtpd/plugins/check_smtp_forward file to use port 2525, that way I sould be able to see connection occuring to both ports on my exchange server.

Continuing to investigate......

Offline ReetP

  • *
  • 2,534
Ok... port... that's most likely a bug in server-manager.

Can you log one? "DELEGATEmail Server cannot use alternative port" or similar.

Note that may have knock on effects as Delegate is used in a few places and will need checking. Also I think it can be manually set per domain. Needs checking.

Code: [Select]
cd /etc/e-smith/templates
grep -rin delegate *

Your case is also odd as it needs one port for the check and another for the actual send!

As far as your testing. Can you add the server debug line to the plugin like my code? See what happens.

We can probably add a few other debug lines to check what is going on (yes it is a nightmare to debug!!). I need to check some of the stuff I have used.

See the self->log bits in whitelist

Code: [Select]
        if ($h eq $ip or $ip =~ /^\Q$h\E/) {
            $self->qp->connection->notes('whitelisthost', 1);
            $self->log(2, "pass, is a whitelisted host"); <<<<< here
            $self->adjust_karma(5);
            return OK;

Or in check_smtp_forward like this:

Code: [Select]
  $self->log(LOGDEBUG, "Checking $recipient on $server:$port");

Ask for a rocket login if you want to chat....
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Hi,
I'll get the bug report raised in the morning.

I don't think I actually need separate ports for 25/2525, I just did it that way to help me debug so I could see the traffic for smtp mail transfer on 25 and the check_smtp_forward attempts on 2525. I don't think I am getting any traffic on 2525 so it looks like the check_smtp_forward test is not happening.

I have used your bit of perl to test  SME -> exchange on both port 25 and 2525, with both valid and invalid email addresses and all combinations behave as expected. I will add the debug code you suggest to check_smtp_forward to see what is going on.

Quote
We can probably add a few other debug lines to check what is going on (yes it is a nightmare to debug!!).

I am a C programmer / hardware engineer and spend my life in low level micro-controller code, which is equally fun to debug :-) I sometime find Perl and regex a bit alien! :shock:

More tomorrow and as ever thanks for the support.

Offline ReetP

  • *
  • 2,534
Hi,
I'll get the bug report raised in the morning.

Col thanks.

Quote
I don't think I actually need separate ports for 25/2525, I just did it that way to help me debug so I could see the traffic for smtp mail transfer on 25 and the check_smtp_forward attempts on 2525. I don't think I am getting any traffic on 2525 so it looks like the check_smtp_forward test is not happening.

Yes, but if you need to use 2525 to check you still need 25 to send which is with qmail. My guess is that if you DO set the port manually to 2525 it will also set qmail to SEND on 2525. Which is not what you want.....

Quote
I am a C programmer / hardware engineer and spend my life in low level micro-controller code, which is equally fun to debug :-) I sometime find Perl and regex a bit alien! :shock:

C? Crikey, you should be able to nibble this off in an afternoon without breaking sweat then... :-)

Perl is quite nice. Regex is not. Period.

Quote
More tomorrow and as ever thanks for the support.

NP. And don't forget to come and talk to us on Rocket and test v10 !
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation