Koozali.org: home of the SME Server

Spam getting through spamassassin.

Offline brianr

  • *
  • 988
  • +2/-0
Spam getting through spamassassin.
« on: September 09, 2019, 12:34:46 PM »
I am trying to chase down why I am getting a lot more spam which is not marked as spam by spamassassin.

Here are the relevent headers:

X-Spam-Status: No, score=-2.0 required=4.0 autolearn=disabled
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on
      xxxxxxxxxxxxxxxxx
X-Spam-Details: *  0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.
      *       See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
      *      for more information.
      *      [URIs: fastly.com]
      * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
      *      [score: 0.0000]
      * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record
      * -0.0 SPF_PASS SPF: sender matches SPF record
      *  0.0 HTML_EMBEDS BODY: HTML with embedded plugin object
      *  0.0 HTML_MESSAGE BODY: HTML included in message
      * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
      *       domain
      * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
      *  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
      *      valid
      *  0.0 T_REMOTE_IMAGE Message contains an external image

In particular I am wondering what is the meaning of the first message. Particularily as the domain from which the email comes IS mentioned in URIBL.

The email has a very large hidden paragraph of gobbledegook which I presume is there to confuse the spam blocker.

Thunderbird seems to spot the spam though, but I'd prefer it to be eliminated earlier.

Thoughts?

Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #1 on: September 09, 2019, 12:52:04 PM »
Update.

Have now fully implemented:

https://bugs.contribs.org/show_bug.cgi?id=10127

I had all the settings, but not the "latest" version of dovecot-extras and dovecote-pidgeonhole.

Still like to understand what the story is with the URIBL though.
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #2 on: September 09, 2019, 02:36:41 PM »
Quote
The query to URIBL was blocked

For whatever reason when spamassassin looked up the URI for fastly.com with the uribl.com blocklist the query got refused.

No idea why.

You can give SpamAssassin version 3.4.2 a test run which is in smetest I think - I have been running it since we built it without any issues. See bug 10597
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #3 on: September 09, 2019, 04:23:22 PM »
you can give SpamAssassin version 3.4.2 a test run which is in smetest I think - I have been running it since we built it without any issues. See bug 10597

its in smeupdates-testing. Trying it...
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #4 on: September 09, 2019, 06:02:18 PM »
its in smeupdates-testing. Trying it...

Ahh damn I always forget the names !

Should be fine - I've had no adverse reactions but if you experience any then follow up on the bug.

This version is GeoIP2 capable - I have a few hacky bit to enable GeoIP in it, but was waiting til this was released.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #5 on: September 09, 2019, 10:19:48 PM »
Got another one with the rejection from  URIBL

X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
      xxxxxxxxxxxxxxxxxxx.co.uk
X-Spam-Details: *  0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was
      *      blocked.  See
      *      http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
      *      for more information.
      *      [URIs: yesbusinessfunding.co.uk]

When I look "yesbusinessfunding.co.uk" up on

https://admin.uribl.com/

It shows it as listed.
« Last Edit: September 09, 2019, 10:21:20 PM by brianr »
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #6 on: September 09, 2019, 11:42:57 PM »
You are not looking at the real problem.

This is your problem:

Quote
URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was
      *      blocked.  See
      *      http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block

You need to investigate that.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Online Jean-Philippe Pialasse

  • *
  • 2,745
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Spam getting through spamassassin.
« Reply #7 on: September 10, 2019, 12:43:00 AM »
The message states the query was refused / blocked. Do they have policy on max amount query per day ?
Do they have blocked your ip or you ip block ?

Misconfiguration of the db on your side ?

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #8 on: September 10, 2019, 07:53:06 AM »
You are not looking at the real problem.

This is your problem:

You need to investigate that.

I've looked at the page it referenced, it is a general page about SPA and the builtin external services it uses. No sign of any sort of Ip block etc.

I am running Pi-Hole (as a Container under Proxmox). and it is in use as the external DNS to SMEServer. There is nothing in the Pi-Hole logs that show anything untoward and I've added a whitelist entry for uribl, but perhaps it is interfering.  I'll remove it for a few days (and put up with the adverts) and see if that makes a difference.
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #9 on: September 10, 2019, 09:38:16 AM »
Pi Hole is unlikely because the query has hit their server but been blocked by them.

Read down that page and follow on from there.

Quote
Questions And Answers

Q: My queries to a DNS-blocklist were blocked. What does this mean?

.......
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Spam getting through spamassassin.
« Reply #10 on: September 10, 2019, 12:48:59 PM »
Our public mirror infastructure consists of donated hardware and bandwidth. If you abuse it, we will block your IP, or your nameserver IP that is producing the excessive queries.
...
...
 If you use your ISP Nameservers for resolution, and they are blocked, consider running your own caching nameserver. Otherwise, consider the commercial datafeed service to run local copies of the URIBL zones and keep your queries on your own network.

If you are using any upstream DNS - google, opendns, or your ISP's DNS servers - they may be "abusing" the uribl public dns servers.  Pi-hole, by default, uses either google or opendns servers for relayed queries.  I don't know what happens if you disable the upstream DNS in pi-hole.

In case it helps, my network looks like this:
- pi-hole points to SME for DNS
- SME DHCP customized to provide pi-hole IP for DHCP clients (here's an old forum post on how I did this)
- SME DNS has no "forwarder", so it's using the DNS root servers in order to avoid being blocked by RBL, RHSBL or URIBL services.

[edit]
The uribl.com page provides command line examples for testing your system (http://uribl.com/about.shtml#testing).  From my SME server, their test works OK:
Quote
[root@office ~]# host -tTXT test.uribl.com.multi.uribl.com
test.uribl.com.multi.uribl.com descriptive text "permanent testpoint"

 -- but if I tell it to use Google DNS it fails:
Quote
[root@office ~]# host -tTXT test.uribl.com.multi.uribl.com 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:

test.uribl.com.multi.uribl.com descriptive text "127.0.0.1 -> Query Refused. See http://uribl.com/refused.shtml for more information [Your DNS IP: 74.125.18.66]"
« Last Edit: September 10, 2019, 01:01:22 PM by mmccarn »

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #11 on: September 10, 2019, 01:06:21 PM »
For ref both at home and in our office I have PiHoles using OpenDNS with SME pointing to the PiHole for external lookups.

config show dnscache Forwarder

So SME does local DHCP & DNS and refers anything else to the PiHole.

Haven't seen this error, though neither have much volume.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #12 on: September 12, 2019, 01:11:26 PM »
For ref both at home and in our office I have PiHoles using OpenDNS with SME pointing to the PiHole for external lookups.

config show dnscache Forwarder

So SME does local DHCP & DNS and refers anything else to the PiHole.

Haven't seen this error, though neither have much volume.

This is the same as my setup, although I am using the cloudflared daemon running on the pihole to make the DNS lookup more secure.

If I revert to just SME for DNS, then the test:

host -tTXT test.uribl.com.multi.uribl.com

Works, but if I add in the pihole, then the test fails, even if I take out the cloudeflared and just use google or cloudflare servers direct.  setting OpenDNS as a choice does not work at all. Perhaps I need some sort of account?


Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Spam getting through spamassassin.
« Reply #13 on: September 12, 2019, 02:39:21 PM »
As I read the uribl.com website (http://uribl.com/datafeed.shtml) if you want to continue using the current config (using DNS for lookups), with your pihole pointing to Google or OpenDNS, you would end up needing to register all of the DNS provider's IPs with uribl.com, and you would end up paying for all the lookups by anyone using the same service.

It may be possible to configure your pihole to send uribl.com queries directly to uribl.com name servers.

I have not tested this in production, but I have confirmed that doing this causes the test query (host -tTXT test.uribl.com.multi.uribl.com) to be forwarded directly to the multi.uribl.com name servers by watching the 'live' pihole log while doing the test using the "tail pihole.log" menu option.

If this seems to work you would need to schedule a job to update /etc/dnsmasq.d/05-pihole.conf at regular intervals in case uribl.com changes their server addresses. 

Configure custom dns forwarding in pi-hole for 'multi.uribl.com'
(taken from https://blog.jstubberfield.net/pihole-conditional-forwarding/)



On your pi-hole system, run these commands to create /etc/dnsmasq.d/05-pihole.conf and restart dns:
Code: [Select]
nslookup -type=ns multi.uribl.com |grep '=' |sed 's/.*= //' |while read a; do host -i $a  |sed 's/.*address /server=\/multi.uribl.com\//'; done > /etc/dnsmasq.d/05-pihole.conf
pihole restartdns

breaking it down:
- get the name servers for multi.uribl.com
nslookup -type=ns multi.uribl.com

- pull out the lines containing "=", then dump everything in front of the IP address:
| grep '=' |sed 's/.*= //'

- step through the results, get the IP addresses, and massage the output into the format used by dnsmasq:
| while read a; do host -i $a  |sed 's/.*address /server=\/multi.uribl.com\//'; done

- send the output to /etc/dnsmasq.d/05-pihole.conf
> /etc/dnsmasq.d/05-pihole.conf

- restart the pihole dns services
pihole restartdns


In case of problems, remove the custom config file and restart dns again:
Code: [Select]
'rm' /etc/dnsmasq.d/05-pihole.conf
pihole restartdns

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #14 on: September 12, 2019, 05:28:13 PM »
https://support.opendns.com/hc/en-us/articles/227986727-Does-OpenDNS-Work-with-DNSBLS-and-URIBLS-

I've just been checking on my servers that have a local PiHole.

Both are low volume - my home one has more with a lot of mailing lists, but I haven't seen anything where they are blocked.

https://wiki.contribs.org/Qpsmtpd#URIBL

So it seems if you have high volume you either need to disable URIBL, or not use a piHole, or use DNS over HTTPS, or DNSSec?

Question.

If you use DNSSec will it bypass the ISP transparent proxy?

For some long while I have had issues with my UK ISP in my UK Office where certain sites are a real bind. Seems they are probably using a DNS transparent proxy and somehow trying to bypass it leaves some sites in a knot. (I have tested a lot over a period of time - if we route a browser via a VPN effectively completely bypassing the ISP the sites are perfect)
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #15 on: September 13, 2019, 07:47:00 AM »
As I read the uribl.com website (http://uribl.com/datafeed.shtml) if you want to continue using the current config (using DNS for lookups),

----cut-------

On your pi-hole system, run these commands to create /etc/dnsmasq.d/05-pihole.conf and restart dns:
Code: [Select]
nslookup -type=ns multi.uribl.com |grep '=' |sed 's/.*= //' |while read a; do host -i $a  |sed 's/.*address /server=\/multi.uribl.com\//'; done > /etc/dnsmasq.d/05-pihole.conf
pihole restartdns

----cut-----

In case of problems, remove the custom config file and restart dns again:
Code: [Select]
'rm' /etc/dnsmasq.d/05-pihole.conf
pihole restartdns


Wow! Many thanks for this - have tried it on the pihole system and it seems to work a treat. Many thanks for this.

I'd also posted on the pihole discourse forum here:

https://discourse.pi-hole.net/t/spamassassin-uribl-access-not-working-through-pihiole/23477/4

So I've posted a link to this thread.
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Spam getting through spamassassin.
« Reply #16 on: September 13, 2019, 02:02:38 PM »
Here is a script that I think could be scripted to run regularly to update multiple DNS BL services in pihole:
Code: [Select]
#!/bin/bash
# list of lists for conditional forwarding. Surround with quotes, separate with spaces
lists="zen.spamhaus.org bl.spamcop.net truncate.gbudb.net ix.dnsbl.manitu.net b.barracudacentral.org dbl.spamhaus.org rhsbl.sorbs.net multi.surbl.org black.uribl.com multi.uribl.com"

# path to custom config file to be updated
customcfg=/etc/dnsmasq.d/05-pihole.conf

# create customcfg file if it does not exist to avoid errors with 'sed -i'
if [ ! -f $customcfg ]
then
  touch $customcfg
fi
 

for list in $lists
do
  # delete the old entries for this list
  sed -i "/$list/d" $customcfg

  # get the name servers for this list, then get their IPs
  host -t ns $list |sed 's/.*name server //' | \
  while read ns
  do
    host -i $ns |\
    grep 'has address' |\
    sed "s/.*has\ address\ /server=\/$list\//"
  done >> $customcfg
done

# restart the dns service to activate the changes
pihole restartdns

Changes from the original command:
- using 'host -t ns' instead of 'nslookup -type=ns' since the pihole examples use host
- loops through multiple blocklists
- removes old entries for each blocklist, then recreates them, instead of overwriting 05-pihole.conf to avoid conflicting with other customizations

The script above works for all of the DNSBL services listed except multi.surbl.org.  multi.surbl.org blocks lookups from large dns servers -- so host -t ns multi.surbl.org fails from my pihole, or if I redirect the query to Google, but works from my SME server using tinydns/dnscache and direct lookups through the root servers.

To handle multi.surbl.org (until they change their name server naming scheme):
Code: [Select]
#!/bin/bash
customcfg=/etc/dnsmasq.d/05-pihole.conf

if [ -f $customcfg ]
then
  sed -i '/surbl.org/d' $customcfg
fi

for h in {a..n}
do
  host -i $h.surbl.org |\
  grep 'has address' |\
  sed "s/.*has\ address\ /server=\/$list\//"
done >> $customcfg

pihole restartdns

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #17 on: September 14, 2019, 08:25:05 AM »
Here is a script that I think could be scripted to run regularly to update multiple DNS BL services in pihole:

--cut---

So you think that the other blocklists also suffer from the same problem? I've not seem any sign of it, although I'm not discounting it.
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #18 on: September 14, 2019, 08:48:06 AM »
I've run the script on my pihole and it fails on:
bl.spamcop.net (not as expected?)
multi.surbl.org (as expected)



Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #19 on: September 14, 2019, 10:03:27 AM »
Shouldn't:

  sed "s/.*has\ address\ /server=\/$list\//"

be:

  sed "s/.*has\ address\ /server=\/$h.surbl.org\/$list\//"

In the "special" handling for surbl?
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Spam getting through spamassassin.
« Reply #20 on: September 14, 2019, 05:57:13 PM »
Shouldn't:
...

Worse than that - it should have been hard coded as 'multi.surbl.org'.

However, I think I have a new version of the script that works for multi.surbl.org without breaking it out.  I found that using "dig +trace ..." instead of "host -t ns ..." worked with my pihole configured to use OpenDNS for upstream.

I've posted the latest version to github here:
https://github.com/mmccarn/pihole/blob/master/bin/pihole-dnsbl.sh

and here is the code:
Code: [Select]
#!/bin/bash
# list of lists for conditional forwarding. Surround with quotes, separate with spaces
lists="zen.spamhaus.org"
lists="$lists bl.spamcop.net"

# truncate.gbudb.net does not have its own NS so we use gbudb.net
lists="$lists gbudb.net"
lists="$lists ix.dnsbl.manitu.net"
lists="$lists b.barracudacentral.org"
lists="$lists dbl.spamhaus.org"
lists="$lists rhsbl.sorbs.net"
lists="$lists multi.surbl.org"
lists="$lists black.uribl.com"
lists="$lists multi.uribl.com"


# path to custom config file to be updated
customcfg=/etc/dnsmasq.d/15-pihole.conf

# create customcfg file if it does not exist to avoid errors with 'sed -i'
if [ ! -f $customcfg ]
then
  touch $customcfg
fi
 

for list in $lists
do
  echo Processing $list
 
  # delete the old entries for this list
  sed -i "/$list/d" $customcfg

  # insert a divider in the config file
  printf "# $list\n" >> $customcfg

  # get the name servers for this list, then get their IPs
#  host -t ns $list |grep "name server " |sed 's/.*name server //' | \
  dig +trace $list |grep $list.*NS |sed 's/.*NS\s*//' | \
  while read ns
  do
    printf "  $list\t$ns\n"
    host -i $ns |\
    grep 'has address' |\
    sed "s/.*has\ address\ /server=\/$list\//" >> $customcfg
  done
done

# restart the dns service to activate the changes
pihole restartdns

Changes:
* I tried to make "lists" easier to read
* I switched from "host -t ns ..." to "dig +trace ..."
  ==> this change makes the separate script for multi.surbl.org unnecessary
* I am inserting a commented separator into the dnscache custom config file
* I have changed the custom config filename from 05-pihole.conf to 15-pihole.conf to avoid conflicting with other instructions I've seen online using 05-pihole.conf
* The program now lists each name server on the screen as it is processed

If you switch to this script you should remove the original custom conf at /etc/dnsmasq.d/05-pihole.conf

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Spam getting through spamassassin.
« Reply #21 on: September 15, 2019, 12:41:57 AM »
So you think that the other blocklists also suffer from the same problem? I've not seem any sign of it, although I'm not discounting it.

(I didn't answer this...)

Yes, I've seen the same problem with other DNSBL services - barracudacentral and at least one other (but I forget which)

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #22 on: September 16, 2019, 10:30:25 AM »
I'm getting this from that dig command:

[root@pihole ~]# dig +trace gbudb.net

; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> +trace gbudb.net
;; global options: +cmd
;; connection timed out; no servers could be reached
[root@pihole ~]#

It "works" on my Fedora 30 desktop, but not on the pihole Container.

However:

dig @8.8.8.8 +trace gbudb.net

seems to work!
« Last Edit: September 16, 2019, 10:43:10 AM by brianr »
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Spam getting through spamassassin.
« Reply #23 on: September 16, 2019, 12:14:05 PM »
I got a few timeouts using 'dig +trace...', but then they would go away.  I had assumed I was breaking pihole while working on the script...

I've updated the script on github to add '@8.8.8.8':
https://github.com/mmccarn/pihole/commit/2650dfbd9a4de3382102d0621fd4b1cc0c4a026c

I notice from pihole.log that dnsmasq is sending the RBL lookups to every server listed in the config file - 44 servers in the case of multi.surbl.com.  I wonder if the config file should be limited to 1, 2, or 3 entries per blocklist...

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #24 on: September 16, 2019, 12:48:38 PM »
I notice from pihole.log that dnsmasq is sending the RBL lookups to every server listed in the config file - 44 servers in the case of multi.surbl.com.  I wonder if the config file should be limited to 1, 2, or 3 entries per blocklist...

Mine has 45. Do we need more than 1?
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #25 on: September 16, 2019, 04:26:24 PM »
Been playing with this on mine.

However, when I run it via cron I get this error:

Code: [Select]
cat /etc/cron.d/dnsupdate

5 * * * *   root    PATH="$PATH:/usr/local/bin/" /root/scripts/pihole-dnsbl.sh

Quote
Restart pihole DNS  [✗] /usr/local/bin/pihole: line 121: service: command not found

Here are the lines but no idea why it fails :-(

Code: [Select]
114    svc="service ${resolver} ${svcOption}"
......
121    output=$( { ${svc}; } 2>&1 )
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #26 on: September 17, 2019, 10:19:16 AM »
Been playing with this on mine.

However, when I run it via cron I get this error:

Code: [Select]
cat /etc/cron.d/dnsupdate

5 * * * *   root    PATH="$PATH:/usr/local/bin/" /root/scripts/pihole-dnsbl.sh

Here are the lines but no idea why it fails :-(

Code: [Select]
114    svc="service ${resolver} ${svcOption}"
......
121    output=$( { ${svc}; } 2>&1 )

I dropped mine into /etc/cron.daily and it seems to have run ok last night:

Code: [Select]
root@pihole ~]# ls -l /etc/dnsmasq.d/
total 12
-rw-r--r-- 1 root root 1885 Sep 12 17:57 01-pihole.conf
-rw-r--r-- 1 root root 8310 Sep 17 04:13 15-pihole.conf
[root@pihole ~]#

and nothing in the logs that I can see.
« Last Edit: September 17, 2019, 10:24:21 AM by brianr »
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #27 on: September 17, 2019, 10:44:21 AM »
I dropped mine into /etc/cron.daily and it seems to have run ok last night:

Ahhh OK - I had mine as a separate script and tried to run it from cron eg

Script is here

Code: [Select]
/root/scripts/pihole-dnsbl.sh
Cron like this

cat /etc/cron.d/dnsupdate

Code: [Select]
5 * * * *   root    PATH="$PATH:/usr/local/bin/" /root/scripts/pihole-dnsbl.sh
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Spam getting through spamassassin.
« Reply #28 on: September 17, 2019, 01:17:07 PM »
On my pi-hole server (running on raspbian 9 / stretch) the default path inside a cron job is /usr/bin:/bin

The default path in a user shall is /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

"service" is in /usr/sbin:
Code: [Select]
which service
/usr/sbin/service

On my system both of these versions of '/etc/cron.d/dnsupdate' successfully regenerate /etc/dnsmasq.d/15-pihole.conf:

Quote from: /etc/cron.d/dnsupdate option 1 - global PATH
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
5 * * * *   root    /root/scripts/pihole-dnsbl.sh

Quote from: /etc/cron.d/dnsupdateoption 2 - local PATH[tt
5 * * * *   root PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" /root/scripts/pihole-dnsbl.sh

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #29 on: September 17, 2019, 03:27:52 PM »
On my pi-hole server (running on raspbian 9 / stretch) the default path inside a cron job is /usr/bin:/bin

I'm not running on a Raspberry Pi at all. I think the container was built on Centos 7.
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #30 on: September 17, 2019, 04:18:16 PM »
On my pi-hole server (running on raspbian 9 / stretch) the default path inside a cron job is /usr/bin:/bin

The default path in a user shall is /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Mine is a LXC instance on Prox. But I think it is a path issue. I'll test those options thanks
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #31 on: September 17, 2019, 04:21:25 PM »
Just had a note to say my URIBL had gone nuts and been rejecting everything!

Need to go back and take a look at what I have done :-(

uribl: www.centos.org in black.uribl.com: 127.0.0.1 -> Query Refused. See http://uribl.com/refused.shtml for more information [Your DNS IP: 1.2.3.4]
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #32 on: September 17, 2019, 10:33:30 PM »
Just had a note to say my URIBL had gone nuts and been rejecting everything!

Need to go back and take a look at what I have done :-(

uribl: www.centos.org in black.uribl.com: 127.0.0.1 -> Query Refused. See http://uribl.com/refused.shtml for more information [Your DNS IP: 1.2.3.4]

That is my original error message.  Seems to be ok here...(although that spam I was bothered wqith seems to have gone away)
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #33 on: September 18, 2019, 11:38:10 AM »
That is my original error message.  Seems to be ok here...(although that spam I was bothered with seems to have gone away)

Yes it is and I am a little baffled.

I'll have to go back and play as I am not sure why it has suddenly decided this. Not got much time right now which is annoying :-(

...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline compdoc

  • ****
  • 211
  • +0/-0
Re: Spam getting through spamassassin.
« Reply #34 on: September 20, 2019, 01:34:51 PM »
*****
X-Spam-Details: *  0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.
      *       See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
      *      for more information.
******************************


I have not read this entire thread, so Im sorry if this has been answered...

This error happens if you forward your network to a popular free DNS service like Google's 8.8.8.8 and 8.8.4.4. Too many queries from there upsets the URIBL servers. Use a less popular service. Cloudflare is working for me atm, and I think Verisign is good too.

Also, do yourself a favor and install smeserver-wbl, then use it to block entire domains like:

*.fr
*.fun
*.icu
*.id
*.pl
*.ru
*.xyz

If certain messages keep making it past, view the message source to know which server/domain to block.
« Last Edit: September 20, 2019, 02:01:53 PM by compdoc »

Offline brianr

  • *
  • 988
  • +2/-0
Re: Spam getting through spamassassin.
« Reply #35 on: September 20, 2019, 02:00:07 PM »
*****
This error happens if you forward your network to a popular free DNS service like Google's 8.8.8.8 and 8.8.4.4. Too many queries from there upsets the URIBL servers. Use a less popular service. Cloudflare is working for me atm, and I think Verisign is good too.


I was getting this error using the cloudflare servers, Although I use "cloudflared" locally so that the request is sent over https.
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Spam getting through spamassassin.
« Reply #36 on: September 20, 2019, 02:06:22 PM »
I have not read this entire thread, so Im sorry if this has been answered...

We understand the issue entirely and are looking at ways around it (and we are using PiHoles too..... https://pi-hole.net/ )

I know about wbl. I have done a number of patches to it....

Always take the time to read the threads please.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation