Koozali.org: home of the SME Server

Spam getting through spamassassin.

Offline brianr

  • *
  • 988
  • +2/-0
Spam getting through spamassassin.
« on: September 09, 2019, 12:34:46 PM »
I am trying to chase down why I am getting a lot more spam which is not marked as spam by spamassassin.

Here are the relevent headers:

X-Spam-Status: No, score=-2.0 required=4.0 autolearn=disabled
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on
      xxxxxxxxxxxxxxxxx
X-Spam-Details: *  0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.
      *       See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
      *      for more information.
      *      [URIs: fastly.com]
      * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
      *      [score: 0.0000]
      * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record
      * -0.0 SPF_PASS SPF: sender matches SPF record
      *  0.0 HTML_EMBEDS BODY: HTML with embedded plugin object
      *  0.0 HTML_MESSAGE BODY: HTML included in message
      * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
      *       domain
      * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
      *  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
      *      valid
      *  0.0 T_REMOTE_IMAGE Message contains an external image

In particular I am wondering what is the meaning of the first message. Particularily as the domain from which the email comes IS mentioned in URIBL.

The email has a very large hidden paragraph of gobbledegook which I presume is there to confuse the spam blocker.

Thunderbird seems to spot the spam though, but I'd prefer it to be eliminated earlier.

Thoughts?

Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #1 on: September 09, 2019, 12:52:04 PM »
Update.

Have now fully implemented:

https://bugs.contribs.org/show_bug.cgi?id=10127

I had all the settings, but not the "latest" version of dovecot-extras and dovecote-pidgeonhole.

Still like to understand what the story is with the URIBL though.
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #2 on: September 09, 2019, 02:36:41 PM »
Quote
The query to URIBL was blocked

For whatever reason when spamassassin looked up the URI for fastly.com with the uribl.com blocklist the query got refused.

No idea why.

You can give SpamAssassin version 3.4.2 a test run which is in smetest I think - I have been running it since we built it without any issues. See bug 10597
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #3 on: September 09, 2019, 04:23:22 PM »
you can give SpamAssassin version 3.4.2 a test run which is in smetest I think - I have been running it since we built it without any issues. See bug 10597

its in smeupdates-testing. Trying it...
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #4 on: September 09, 2019, 06:02:18 PM »
its in smeupdates-testing. Trying it...

Ahh damn I always forget the names !

Should be fine - I've had no adverse reactions but if you experience any then follow up on the bug.

This version is GeoIP2 capable - I have a few hacky bit to enable GeoIP in it, but was waiting til this was released.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #5 on: September 09, 2019, 10:19:48 PM »
Got another one with the rejection from  URIBL

X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
      xxxxxxxxxxxxxxxxxxx.co.uk
X-Spam-Details: *  0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was
      *      blocked.  See
      *      http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
      *      for more information.
      *      [URIs: yesbusinessfunding.co.uk]

When I look "yesbusinessfunding.co.uk" up on

https://admin.uribl.com/

It shows it as listed.
« Last Edit: September 09, 2019, 10:21:20 PM by brianr »
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #6 on: September 09, 2019, 11:42:57 PM »
You are not looking at the real problem.

This is your problem:

Quote
URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was
      *      blocked.  See
      *      http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block

You need to investigate that.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Online Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Spam getting through spamassassin.
« Reply #7 on: September 10, 2019, 12:43:00 AM »
The message states the query was refused / blocked. Do they have policy on max amount query per day ?
Do they have blocked your ip or you ip block ?

Misconfiguration of the db on your side ?

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #8 on: September 10, 2019, 07:53:06 AM »
You are not looking at the real problem.

This is your problem:

You need to investigate that.

I've looked at the page it referenced, it is a general page about SPA and the builtin external services it uses. No sign of any sort of Ip block etc.

I am running Pi-Hole (as a Container under Proxmox). and it is in use as the external DNS to SMEServer. There is nothing in the Pi-Hole logs that show anything untoward and I've added a whitelist entry for uribl, but perhaps it is interfering.  I'll remove it for a few days (and put up with the adverts) and see if that makes a difference.
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #9 on: September 10, 2019, 09:38:16 AM »
Pi Hole is unlikely because the query has hit their server but been blocked by them.

Read down that page and follow on from there.

Quote
Questions And Answers

Q: My queries to a DNS-blocklist were blocked. What does this mean?

.......
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Spam getting through spamassassin.
« Reply #10 on: September 10, 2019, 12:48:59 PM »
Our public mirror infastructure consists of donated hardware and bandwidth. If you abuse it, we will block your IP, or your nameserver IP that is producing the excessive queries.
...
...
 If you use your ISP Nameservers for resolution, and they are blocked, consider running your own caching nameserver. Otherwise, consider the commercial datafeed service to run local copies of the URIBL zones and keep your queries on your own network.

If you are using any upstream DNS - google, opendns, or your ISP's DNS servers - they may be "abusing" the uribl public dns servers.  Pi-hole, by default, uses either google or opendns servers for relayed queries.  I don't know what happens if you disable the upstream DNS in pi-hole.

In case it helps, my network looks like this:
- pi-hole points to SME for DNS
- SME DHCP customized to provide pi-hole IP for DHCP clients (here's an old forum post on how I did this)
- SME DNS has no "forwarder", so it's using the DNS root servers in order to avoid being blocked by RBL, RHSBL or URIBL services.

[edit]
The uribl.com page provides command line examples for testing your system (http://uribl.com/about.shtml#testing).  From my SME server, their test works OK:
Quote
[root@office ~]# host -tTXT test.uribl.com.multi.uribl.com
test.uribl.com.multi.uribl.com descriptive text "permanent testpoint"

 -- but if I tell it to use Google DNS it fails:
Quote
[root@office ~]# host -tTXT test.uribl.com.multi.uribl.com 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:

test.uribl.com.multi.uribl.com descriptive text "127.0.0.1 -> Query Refused. See http://uribl.com/refused.shtml for more information [Your DNS IP: 74.125.18.66]"
« Last Edit: September 10, 2019, 01:01:22 PM by mmccarn »

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #11 on: September 10, 2019, 01:06:21 PM »
For ref both at home and in our office I have PiHoles using OpenDNS with SME pointing to the PiHole for external lookups.

config show dnscache Forwarder

So SME does local DHCP & DNS and refers anything else to the PiHole.

Haven't seen this error, though neither have much volume.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #12 on: September 12, 2019, 01:11:26 PM »
For ref both at home and in our office I have PiHoles using OpenDNS with SME pointing to the PiHole for external lookups.

config show dnscache Forwarder

So SME does local DHCP & DNS and refers anything else to the PiHole.

Haven't seen this error, though neither have much volume.

This is the same as my setup, although I am using the cloudflared daemon running on the pihole to make the DNS lookup more secure.

If I revert to just SME for DNS, then the test:

host -tTXT test.uribl.com.multi.uribl.com

Works, but if I add in the pihole, then the test fails, even if I take out the cloudeflared and just use google or cloudflare servers direct.  setting OpenDNS as a choice does not work at all. Perhaps I need some sort of account?


Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Spam getting through spamassassin.
« Reply #13 on: September 12, 2019, 02:39:21 PM »
As I read the uribl.com website (http://uribl.com/datafeed.shtml) if you want to continue using the current config (using DNS for lookups), with your pihole pointing to Google or OpenDNS, you would end up needing to register all of the DNS provider's IPs with uribl.com, and you would end up paying for all the lookups by anyone using the same service.

It may be possible to configure your pihole to send uribl.com queries directly to uribl.com name servers.

I have not tested this in production, but I have confirmed that doing this causes the test query (host -tTXT test.uribl.com.multi.uribl.com) to be forwarded directly to the multi.uribl.com name servers by watching the 'live' pihole log while doing the test using the "tail pihole.log" menu option.

If this seems to work you would need to schedule a job to update /etc/dnsmasq.d/05-pihole.conf at regular intervals in case uribl.com changes their server addresses. 

Configure custom dns forwarding in pi-hole for 'multi.uribl.com'
(taken from https://blog.jstubberfield.net/pihole-conditional-forwarding/)



On your pi-hole system, run these commands to create /etc/dnsmasq.d/05-pihole.conf and restart dns:
Code: [Select]
nslookup -type=ns multi.uribl.com |grep '=' |sed 's/.*= //' |while read a; do host -i $a  |sed 's/.*address /server=\/multi.uribl.com\//'; done > /etc/dnsmasq.d/05-pihole.conf
pihole restartdns

breaking it down:
- get the name servers for multi.uribl.com
nslookup -type=ns multi.uribl.com

- pull out the lines containing "=", then dump everything in front of the IP address:
| grep '=' |sed 's/.*= //'

- step through the results, get the IP addresses, and massage the output into the format used by dnsmasq:
| while read a; do host -i $a  |sed 's/.*address /server=\/multi.uribl.com\//'; done

- send the output to /etc/dnsmasq.d/05-pihole.conf
> /etc/dnsmasq.d/05-pihole.conf

- restart the pihole dns services
pihole restartdns


In case of problems, remove the custom config file and restart dns again:
Code: [Select]
'rm' /etc/dnsmasq.d/05-pihole.conf
pihole restartdns

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #14 on: September 12, 2019, 05:28:13 PM »
https://support.opendns.com/hc/en-us/articles/227986727-Does-OpenDNS-Work-with-DNSBLS-and-URIBLS-

I've just been checking on my servers that have a local PiHole.

Both are low volume - my home one has more with a lot of mailing lists, but I haven't seen anything where they are blocked.

https://wiki.contribs.org/Qpsmtpd#URIBL

So it seems if you have high volume you either need to disable URIBL, or not use a piHole, or use DNS over HTTPS, or DNSSec?

Question.

If you use DNSSec will it bypass the ISP transparent proxy?

For some long while I have had issues with my UK ISP in my UK Office where certain sites are a real bind. Seems they are probably using a DNS transparent proxy and somehow trying to bypass it leaves some sites in a knot. (I have tested a lot over a period of time - if we route a browser via a VPN effectively completely bypassing the ISP the sites are perfect)
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation