Koozali.org: home of the SME Server

accepting email from .eu domain

Offline julianop

  • *
  • 61
  • +0/-0
accepting email from .eu domain
« on: January 03, 2019, 11:36:09 PM »
My SME9.x server is marking incoming email from valid .eu domain addresses as spam. Can someone point me in the right direction to correct this?
The server is up to date.
I need help, and will post whatever logs might be necessary to help diagnose the issue.
Thanks in advance.

Offline TerryF

  • grumpy old man
  • *
  • 1,821
  • +6/-0
Re: accepting email from .eu domain
« Reply #1 on: January 04, 2019, 12:05:49 AM »
Do you have a bare install? or do you have any contribs installed particularly spam/mail etc contribs?

Do you have RBLs enabled?
--
qui scribit bis legit

Offline TerryF

  • grumpy old man
  • *
  • 1,821
  • +6/-0
Re: accepting email from .eu domain
« Reply #2 on: January 04, 2019, 12:08:54 AM »
Should have added, essential reading:  https://wiki.contribs.org/Email#Spam
--
qui scribit bis legit

Offline julianop

  • *
  • 61
  • +0/-0
Re: accepting email from .eu domain
« Reply #3 on: January 04, 2019, 01:59:14 AM »
Do you have a bare install? or do you have any contribs installed particularly spam/mail etc contribs?

Do you have RBLs enabled?

Essentially it's a bare install, meaning that I have not installed or modified anything that relates in any way to email handling.
I dunno about RBLs; I certainly haven't consciously enabled blacklisting, using only the virus scanning and spam filtering that is installed by default.

Note on competency: I'm not a total noob by any means, but I am certainly not up to date on email security techniques, and this particular issue is outside my current knowledge. I apologize for that.

Offline TerryF

  • grumpy old man
  • *
  • 1,821
  • +6/-0
Re: accepting email from .eu domain
« Reply #4 on: January 04, 2019, 02:24:58 AM »
Note on competency: I'm not a total noob by any means, but I am certainly not up to date on email security techniques, and this particular issue is outside my current knowledge. I apologize for that.

Probably been at this longer than I have and as a job, I am just a plodder :-), no apology warranted..

The reason I added the link to that section of the wiki is due to recent updates to qpsmtpd and associated plugins..best to make sure what is enabled and included
--
qui scribit bis legit

Offline julianop

  • *
  • 61
  • +0/-0
Re: accepting email from .eu domain
« Reply #5 on: January 04, 2019, 02:26:17 AM »
Should have added, essential reading:  https://wiki.contribs.org/Email#Spam

Wading through that now :-)
Specifically, learning about RBLs... but I can't see anything that explains why .eu addresses are being summarily rejected.

Comically, and perhaps ironically, I've been subscribed to the Postfix mail list for some years, as my previous email server used it. The main person whose emails are being rejected is a senior and well respected contributor to that list !!!

I had asked on the Postfix list why his and only his emails were being rejected. I'd said that I'm using SME 9, not Postfix, to which his unhelpful answer was "ask on another maillist then".

Somebody else came to my "help" with...

Because the domain that he uses to send emails through this mailing
list has DMARC p=quarantine setting:
# dig +short _dmarc.junc.eu TXT
"v=DMARC1; p=quarantine; rua=mailto:report_rua@dmarc.junc.eu; fo=d;
adkim=r; aspf=r; sp=none"


Does that shed any light on the issue?

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: accepting email from .eu domain
« Reply #6 on: January 04, 2019, 03:54:58 AM »
It largely depends on your mailserver and what it thinks of his mail. It could be poor configuration their end, or aggressive settings at yours.

How do you receive mail ? Directly via SMTP or does the server fetch it via POP3?

First thing is in the mail headers - can you post the headers of a Spam mail please so we can have a look? That should give some clues.

You can have a look at the logs in /var/log/spamd

Depending on how you get mail the next log to search is:

/var/log/qpsmtpd/current

That has a series of archives but they are rotated pretty quickly.

You can have a look for his email

Code: [Select]
grep -ir "joe@postfix.eu" /var/log/qpsmtpd/current | tai64nlocal
or

Code: [Select]
grep -ir "joe@postfix.eu" /var/log/qpsmtpd/* | tai64nlocal
You hopefully you should find something like

Quote
2019-01-02 09:57:19.702845500 21326 dispatching RCPT TO:<joe@postfix.eu>

We can now look for the entire transaction which in this case is 21326

Code: [Select]
grep 21326 /var/log/qpsmtpd/current | tai64nlocal
Might tell us a bit more.

On top of that lets have a look at these:

Code: [Select]
config show spamassassin
Code: [Select]
config show qpsmtpd
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline julianop

  • *
  • 61
  • +0/-0
Re: accepting email from .eu domain
« Reply #7 on: January 04, 2019, 05:00:41 AM »
It largely depends on your mailserver and what it thinks of his mail. It could be poor configuration their end, or aggressive settings at yours.

How do you receive mail ? Directly via SMTP or does the server fetch it via POP3?

First thing is in the mail headers - can you post the headers of a Spam mail please so we can have a look? That should give some clues.

You can have a look at the logs in /var/log/spamd

Depending on how you get mail the next log to search is:

/var/log/qpsmtpd/current

That has a series of archives but they are rotated pretty quickly.

You can have a look for his email

Code: [Select]
grep -ir "joe@postfix.eu" /var/log/qpsmtpd/current | tai64nlocal
or

Code: [Select]
grep -ir "joe@postfix.eu" /var/log/qpsmtpd/* | tai64nlocal
You hopefully you should find something like

We can now look for the entire transaction which in this case is 21326

Code: [Select]
grep 21326 /var/log/qpsmtpd/current | tai64nlocal
Might tell us a bit more.

On top of that lets have a look at these:

Code: [Select]
config show spamassassin
Code: [Select]
config show qpsmtpd

OK, here we go:
First, I have a fixed IP address and am using smtp for mail.
Next, here's an extract from /var/log/spamd/current for the latest rejected mail from the sender:
@400000005c2e83d71a6b63bc Jan  3 15:51:09.443 [2081] info: spamd: connection from 127.0.0.1 [127.0.0.1]:33706 to port 783, fd 4
@400000005c2e83d71aadeafc Jan  3 15:51:09.447 [2081] info: spamd: checking message <82834dccf5f62ab898c6e9dfad03a412@junc.eu> for qpsmtpd:1005
@400000005c2e83d82af18d9c Jan  3 15:51:10.720 [2081] info: spamd: clean message (-5.3/7.0) for qpsmtpd:1005 in 1.3 seconds, 4434 bytes.
@400000005c2e83d82af4e8fc Jan  3 15:51:10.721 [2081] info: spamd: result: . -5 - BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED scantime=1.3,size=4434,user=qpsmtpd,uid=1005,required_score=7.0,rhost=127.0.0.1,raddr=127.0.0.1,rport=33706,mid=<82834dccf5f62ab898c6e9dfad03a412@junc.eu>,bayes=0.000000,autolearn=disabled
@400000005c2e83d82eedec74 Jan  3 15:51:10.787 [2325] info: prefork: child states: II


This message [2081] doesn't show up in /var/log/qpsmtpd, it jumps from [2069] to a [2183] "cleanup" for [2069] to [2082].

I use Thunderbird for Windows, which, while it allows me to see full headers, doesn't make it easy for me to copy. The message is filtered into a folder, so I'll go find it and post the header info next.

Offline julianop

  • *
  • 61
  • +0/-0
Re: accepting email from .eu domain
« Reply #8 on: January 04, 2019, 05:53:21 AM »
Here's the header. I hope to heck I'm not disclosing anything private, but this is a header from a piece of listmail, so I presume it's all OK. I've <xx>'d out the name parts of email addresses, just to add a thong of privacy to the innocent parties.

Subject:
Re: Limiting global number of outgoing connections
From:
Xxxxx Xxxxxxxx <xx>@junc.eu>
Date:
1/3/2019, 3:50 PM
To:
postfix-users@postfix.org
Return-Path:
<owner-postfix-users@postfix.org>
Delivered-To:
<myemailaddress>@spencer.barnlea.com
Received:
(qmail 6980 invoked by alias); 3 Jan 2019 21:51:11 -0000
Delivered-To:
alias-localdelivery-<myemailaddress>@barnlea.com
Received:
(qmail 6977 invoked by uid 453); 3 Jan 2019 21:51:10 -0000
X-Virus-Checked:
by ClamAV 0.100.2 on barnlea.com
X-Virus-Found:
No
X-Spam-Status:
No, score=-5.3 required=7.0 autolearn=disabled
X-Spam-Checker-Version:
SpamAssassin 3.4.1 (2015-04-28) on spencer.barnlea.com
X-Spam-Details:
* -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, * medium trust * [168.100.1.4 listed in list.dnswl.org] * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail * domains are different * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's * domain * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list * manager
X-HELO:
russian-caravan.cloud9.net
Authentication-Results:
barnlea.com; auth=none; spf=none smtp.mailfrom=postfix.org; dkim=pass header.i=@junc.eu; dmarc=pass (p=quarantine) d=junc.eu
Received:
from russian-caravan.cloud9.net (HELO russian-caravan.cloud9.net) (168.100.1.4) by barnlea.com (qpsmtpd/0.96) with ESMTPS (DHE-RSA-AES256-SHA encrypted); Thu, 03 Jan 2019 15:51:09 -0600
X-DKIM-Authentication:
domain: junc.eu, selector: default, result: pass, policy: o=~, name: sender, policy_result: neutral, policy: o=~, name: author, policy_result: accept, policy: dkim=all, name: ADSP, policy_location: junc.eu, policy_result: accept
Received-SPF:
none (postfix.org: No applicable sender policy available) receiver=spencer.barnlea.com; identity=mailfrom; envelope-from="owner-postfix-users@postfix.org"; helo=russian-caravan.cloud9.net; client-ip=168.100.1.4
Received:
by russian-caravan.cloud9.net (Postfix) id 3366C339B7F; Thu, 3 Jan 2019 16:51:00 -0500 (EST)
Delivered-To:
postfix-users-outgoing@cloud9.net
Received:
from localhost (localhost [127.0.0.1]) by russian-caravan.cloud9.net (Postfix) with ESMTP id 30CBA330B1A for <postfix-users-outgoing@cloud9.net>; Thu, 3 Jan 2019 16:51:00 -0500 (EST)
X-Virus-Scanned:
amavisd-new at cloud9.net
Received:
from russian-caravan.cloud9.net ([127.0.0.1]) by localhost (russian-caravan.cloud9.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2TYLRm7zUpLN for <postfix-users-outgoing@cloud9.net>; Thu, 3 Jan 2019 16:51:00 -0500 (EST)
Received:
by russian-caravan.cloud9.net (Postfix, from userid 54) id 1296E339F6C; Thu, 3 Jan 2019 16:51:00 -0500 (EST)
Delivered-To:
postfix-users@cloud9.net
Received:
from localhost (localhost [127.0.0.1]) by russian-caravan.cloud9.net (Postfix) with ESMTP id CD024339B7F for <postfix-users@cloud9.net>; Thu, 3 Jan 2019 16:50:59 -0500 (EST)
X-Virus-Scanned:
amavisd-new at cloud9.net
Received:
from russian-caravan.cloud9.net ([127.0.0.1]) by localhost (russian-caravan.cloud9.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tWWkXVYwnfN9 for <postfix-users@cloud9.net>; Thu, 3 Jan 2019 16:50:59 -0500 (EST)
Received:
from linode.junc.eu (linode.junc.eu [176.58.121.172]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by russian-caravan.cloud9.net (Postfix) with ESMTPS id 9BE40330B1A for <postfix-users@postfix.org>; Thu, 3 Jan 2019 16:50:59 -0500 (EST)
Received:
from localhost.junc.eu (localhost.junc.eu [127.0.0.1]) by localhost.junc.eu (Postfix) with ESMTP id 668D41BE163 for <postfix-users@postfix.org>; Thu, 3 Jan 2019 21:50:56 +0000 (GMT)
Received:
from localhost.junc.eu (localhost.junc.eu [IPv6:::1]) by linode.junc.eu (Postfix) with ESMTPSA id 375271BE110 for <postfix-users@postfix.org>; Thu, 3 Jan 2019 21:50:56 +0000 (GMT)
DKIM-Signature:
v=1; a=rsa-sha256; c=simple/simple; d=junc.eu; s=default; t=1546552256; x=1546984256; bh=wGHoMklRtAJ3K07aG9w6y3uuKu0bXAhM+XgSsIMe0xE=; h=Date:From:To:Subject:In-Reply-To:References; b=vBnKH0GuqmU9ZAINlW2t0NfYlAW8U+YqOEdABGDcIQrDzGitQjNM9MXcsEtSWO+Cr 4LFKI6m3HooU648IFo0ltOUlZKqYXUxbilAFvG5yBU46DMLOkDE/XMTjrV6zyFbQqT 99piwIjfbojIgfoAD+qRiK7ny5oVx+SRWOGABGwc=
MIME-Version:
1.0
Content-Type:
text/plain; charset=US-ASCII; format=flowed
Content-Transfer-Encoding:
7bit
Organization:
Jersore Underground Network Center
In-Reply-To:
<8CAA4826-40D7-4A3F-804F-C1B9545F29EC@dukhovni.org>
References:
<009C8269-3220-494D-9C4E-6431A3AFE88E@lorenzomilesi.it> <8CAA4826-40D7-4A3F-804F-C1B9545F29EC@dukhovni.org>
Message-ID:
<82834dccf5f62ab898c6e9dfad03a412@junc.eu>
X-Sender:
<xx>@junc.eu
User-Agent:
Roundcube Webmail/1.3.8
Sender:
owner-postfix-users@postfix.org
Precedence:
bulk
List-Id:
Postfix users <postfix-users@postfix.org>
List-Post:
<mailto:postfix-users@postfix.org>
List-Help:
<http://www.postfix.org/lists.html>
List-Unsubscribe:
<mailto:majordomo@postfix.org>
List-Subscribe:
<mailto:majordomo@postfix.org>

Offline sages

  • *
  • 182
  • +0/-0
    • http://www.sages.com.au
Re: accepting email from .eu domain
« Reply #9 on: January 04, 2019, 07:02:17 AM »
It's a wonder the email isn't seasick after passing through so many mail servers
...

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: accepting email from .eu domain
« Reply #10 on: January 04, 2019, 10:12:56 AM »
It's a wonder the email isn't seasick after passing through so many mail servers

Hehehehehe priceless !
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: accepting email from .eu domain
« Reply #11 on: January 04, 2019, 10:23:14 AM »
Good start @julianop

Quote
X-Spam-Status:
No, score=-5.3 required=7.0 autolearn=disabled

So looks like spamassassin isn't junking it then. So it isn't your server from the looks of things.

I presume it is ending up in your Thunderbird Junk?

My guess here & now is it is Thunderbird doing the Junking, not SME.

There are several different Junk settings in TB but on mobile at the minute so from memory....

You can 'train' it by clicking the Junk icon. You can also check the account junk settings. Don't junk if contact is in address book etc. Also in Preferences I think you can reset the Junk training. Might need to have play and tweak.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline julianop

  • *
  • 61
  • +0/-0
Re: accepting email from .eu domain
« Reply #12 on: January 04, 2019, 06:12:48 PM »
Good start @julianop

So looks like spamassassin isn't junking it then. So it isn't your server from the looks of things.

I presume it is ending up in your Thunderbird Junk?

My guess here & now is it is Thunderbird doing the Junking, not SME.

There are several different Junk settings in TB but on mobile at the minute so from memory....

You can 'train' it by clicking the Junk icon. You can also check the account junk settings. Don't junk if contact is in address book etc. Also in Preferences I think you can reset the Junk training. Might need to have play and tweak.

Thanks for your response, ReetP...
Yes, the emails are ending up in Junk, but there seems to be nothing I can do to prevent it, and it's only emails from that particular user using a .eu TLD that are summarily rejected for no apparent reason. And training seems to have no effect: I keep telling TB that the emails aren't junk, but it refuses to accept them as valid. Those behaviors are why I thought the issue was something embedded in the header on the server side.

I'll see if I can spin up another IMAP email client and see what happens...

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: accepting email from .eu domain
« Reply #13 on: January 05, 2019, 12:10:45 AM »
Ahhh ok.


From your email headers there was no sign that SME was junking it.

So it seems Thunderbird is the issue, and not SME.

You could try Thunderbird with a clean profile. I know it sometimes has a mind of its own.... and don't talk to me about their latest version breaking add ons and with a half baked new API..... Grrrrrr...
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline julianop

  • *
  • 61
  • +0/-0
Re: accepting email from .eu domain
« Reply #14 on: January 05, 2019, 01:20:47 AM »
Ahhh ok.


From your email headers there was no sign that SME was junking it.

So it seems Thunderbird is the issue, and not SME.

You could try Thunderbird with a clean profile. I know it sometimes has a mind of its own.... and don't talk to me about their latest version breaking add ons and with a half baked new API..... Grrrrrr...

I'll try the new profile, thanks, though that'll be a pain too, because I have several accounts set up.
I totally agree on the new version: as part of trying to solve this dilemma I had to juggle with filters, ended up with a pile of duplicated emails, only to find to my horror that the remove duplicates add-on was disabled. Grrrrr is right!
Can you suggest another client that works with SME and handles IMAP properly??