Koozali.org: home of the SME Server

Using SpamAssassin...

spook

Using SpamAssassin...
« on: March 21, 2006, 08:22:45 PM »
When I do recieve spam, in spite of SpamAssasin being active, is there an easy way for me (and my users) to report it as spam? Was wondering if you could just forward the mail to "spam@mydomain.com" and have it "sent" to spamassassin?

Offline idyll

  • ***
  • 113
  • +0/-0
yes
« Reply #1 on: March 21, 2006, 09:49:44 PM »
First the system needs to have seen anough SPAM for auto-learning to be turned on. Turning it on too soon makes it behave very poorly.

You can use fetchmail or a simple perl script to poll folders in each home directory where your users deposit email which should be learned as SPAM. This is the cleanest way to do it. It is also completly overlooked in most SA installations.

I used this on my previous 6.0.1 server (with Jesper's kind assistance) and it trained SA to a very high degree.

I am now using 7.0 RC1 and waiting for it to have seen enough SPAM to invoke auto-learn.

regards,

patrick
...

jsheets

Using SpamAssassin...
« Reply #2 on: March 23, 2006, 05:22:20 PM »
Does SA invoke autolearn automatically when it has seen enough spam?

Offline idyll

  • ***
  • 113
  • +0/-0
yes
« Reply #3 on: March 23, 2006, 05:43:44 PM »
This issue took me way further into the bowels of spamassassin than I ever imagined or hoped for.   Plus I can't seem to post anything about the issue without being pressed to use RBLs :-).

Note: I use RBLs, and always have. They are a given in my environment.

Auto-learn is indeed set to commence after 200 SPAM and 200 HAM and this can be changed - but there is no reason to fiddle with that parameter.

It is the Bayes filtering itself which is NOT "on" by default, and needs to be in my opinion. Others may not agree.

SA relies upon directories in each home, or otherwise known as ~/.spamassassin. Use this sommand logged in as the specific user, NOT root, to see what the system knows for THAT user...if you are logged in as root it will fail complaining about lack of a toks database.

sa-learn -D --dump magic

You'll see at the end how many "nspam" and "nham" it has learned. By deault it has zeros.

If you look at the spamd log files, you will see "autolearn:no" and "autolearn:failed" pretty much all of the time. This is perfectly normal as most of the SPAM does not need to be learned again. The "failed" can be ignored as the system uses a simple file locking mechanism and it just passes on the db lookup if it cannot get an answer. Only if it "fails" all of the time should you be concerned.

So my advice is to turn it "on" and let the full system work over time. But if you run an extremely slow system, it may add too much load. I mean like 386 or a 486 with say 256mb of RAM, etc. I know they exist and I am not dismissing them as servers.

regards,

patrick
...

meb

How to enable?
« Reply #4 on: March 25, 2006, 12:08:27 AM »
Is RBL blocking enabled by default?  If not, how can I enable it?

How do I enable learning?

Offline idyll

  • ***
  • 113
  • +0/-0
use search
« Reply #5 on: March 25, 2006, 12:18:42 AM »
The SEARCH function serves a very good purpose. did you try it?

To view the defaults for the RBL and DNSBL:

# config show qpsmtpd
qpsmtpd=service
Bcc=disabled
BccUser=maillog
DNSBL=disabled
LogLevel=6
MaxScannerSize=25000000
RBLList=sbl-xbl.spamhaus.org,whois.rfc-ignorant.org,dnsbl.njabl.org,relays.ordb.org
RHSBL=disabled
RequireResolvableFromHost=no
SBLList=dsn.rfc-ignorant.org
access=public
status=enabled

To enable both available lists:

# config setprop qpsmtpd DNSBL enabled RHSBL enabled
# signal-event email-update

To enable Bayes or auto-learn you need to educate yourself so you know what you are getting into. Go to the spamassassin wiki and read up a bit.

patrick
...

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: yes
« Reply #6 on: March 25, 2006, 01:40:30 AM »
Quote from: "idyll"

SA relies upon directories in each home, or otherwise known as ~/.spamassassin.


SA in SME server runs as the unprivileged user spamd, and doesn't have access to any user's home directory or any ~/.spamassassin files.

Offline idyll

  • ***
  • 113
  • +0/-0
OK
« Reply #7 on: March 25, 2006, 02:03:23 AM »
The output I depicted, after I was asked, is still valid for that user's home and the SA status for that same user.

Agree?

patrick
...

Offline chris burnat

  • ****
  • 1,135
  • +2/-0
    • http://www.burnat.com
Using SpamAssassin...
« Reply #8 on: March 27, 2006, 11:26:41 AM »
# config setprop qpsmtpd DNSBL enabled RHSBL enabled
# signal-event email-update

Should you also run:  svc -t /service/qpsmtpd ?
Thanks.
- chris
If it does not work out of the box, please fill in a Bug Report @ Bugzilla (http://bugs.contribs.org)  - check: http://wiki.contribs.org/Bugzilla_Help .  Thanks.

spook

Using SpamAssassin...
« Reply #9 on: March 31, 2006, 08:02:46 AM »
Quite impressive, 8 answers, none of which seems to answer my question:

How do I report spam as being spam to spamassassin?

Rather simple, I thougt, but I might be wrong...

Offline chris burnat

  • ****
  • 1,135
  • +2/-0
    • http://www.burnat.com
Using SpamAssassin...
« Reply #10 on: March 31, 2006, 08:40:58 AM »
Spook,  
"How do I report spam as being spam to spamassassin"
Lets try...It may be that your question is a little ambiguous.  There is no need to report anything to spamassassin.  Spamassassin is a program designed to identify spams using a number of mechanisms, then tag the offensive email to facilitate further processing.  In SME7, the default processing consists in the forwarding the offensive email to the junkmail folder of relevant users.  

You could modify the standard settings of SME7 to change the way in which spams are processed.  One way is to use procmail.  Another way would be to modify 15sortspam to send all spams to a common email folder associated with a user created just for this purpose.  Is this what you are after?
- chris
If it does not work out of the box, please fill in a Bug Report @ Bugzilla (http://bugs.contribs.org)  - check: http://wiki.contribs.org/Bugzilla_Help .  Thanks.

Offline raem

  • *
  • 3,972
  • +4/-0
Using SpamAssassin...
« Reply #11 on: March 31, 2006, 09:45:26 AM »
burnat

>...Should you also run:  svc -t /service/qpsmtpd ?

Yes
]
...

Offline chris burnat

  • ****
  • 1,135
  • +2/-0
    • http://www.burnat.com
Using SpamAssassin...
« Reply #12 on: March 31, 2006, 09:57:46 AM »
Thank you Ray.
- chris
If it does not work out of the box, please fill in a Bug Report @ Bugzilla (http://bugs.contribs.org)  - check: http://wiki.contribs.org/Bugzilla_Help .  Thanks.

Offline brianr

  • *
  • 988
  • +2/-0
Using SpamAssassin...
« Reply #13 on: March 31, 2006, 12:09:40 PM »
I suspect that spook is looking for a faciility to report false negatives to SA so that it is trained to spot them next time.

I have considered this on a number of occasions, but not followed through my thoughts.

My ideas are that each user has another folder "IsJunkMail" which they use to drag and drop spam which has not been spotted by SA.  Then every so often, the folder is interrogated and passed to SA for training and then the contents are deleted, this could  happen daily or weekly, depending on how much gets through.  This can be done by a script (Perl or Shell), which i have not yet had the time to develop.  If anyone else wants to have a go I will be glad to help..
Brian j Read
(retired, for a second time, still got 2 installations though)
The instrument I am playing is my favourite Melodeon.
.........

Offline idyll

  • ***
  • 113
  • +0/-0
the script for the sa-learn process
« Reply #14 on: March 31, 2006, 04:50:24 PM »
Jesper wrote this perl script a few months ago as we worked out such a "training" process for our 6.0 servers. He shared it with me, and I do the same with you. It is approproate to leave his name, etc. intact or you are subverting the notion.

SA needs to see 200 such SPAM before autolearn will initiate. This script will help get to that number and then continue to feed it.

Jesper is quite busy with his new company and at the same time, quite happy with his 6.0 servers.  I have moved to the 7.0 series for my own reasons. However, this script works perfectly with the 7.0 servers. You just need to create your own cron job, I run it every 8 hours. We named it LearnAsSpam.pl  (and the folders are named the same) but obviously you may do as you wish and modify the script to meet your naming needs. It works out of the box, BUT the users must be granted ssh access for it to run  without issue. If your users do NOT have ssh access you need to enter this line..

# chsh -s /bin/bash <user>


patrick

---- snip -----------

#!/usr/bin/perl

#############################################################################
#
# This script has been developed
# by Jesper Knudsen at http://sme.swerts-knudsen.dk
#
# Revision History:
#
# January 18, 2006:      Initial version
#############################################################################

use Sys::Hostname;

use esmith::AccountsDB;

my $hostname = hostname();
 
my $adb = esmith::AccountsDB->open_ro()
        or die "Couldnt' open AccountsDB\n";
 
my @users = $adb->users;

foreach my $user (@users)
{
  my $firstname = $user->prop('FirstName');
  my $lastname = $user->prop('LastName');
  my $key = $user->key;

  printf("Checking for user (%s): %s %s\n", $key,$firstname, $lastname);

  $MailDir = "/home/e-smith/files/users/" . $key . "/Maildir";
  opendir(LOGDIR, $MailDir);
  my $dirname = sprintf "LearnAsSpam";
  my @logdirs = sort grep { /$dirname/ } readdir(LOGDIR);
  closedir(LOGDIR);

  foreach my $logdir (@logdirs) {

    my $SpamDir = $MailDir . "/" . $logdir . "/cur/";

#    printf("Checking Dir: %s\n",$SpamDir);
    opendir(SPAMDIR, $SpamDir);
    my @spamfiles = sort grep { /$hostname/ } readdir(SPAMDIR);
    closedir(SPAMDIR);

    foreach $spamfile (@spamfiles) {
      my $filetolearn = $Maildir . $SpamDir . $spamfile;

      $filetolearn =~ s/;/\\;/g;
      $filetolearn =~ s/:/\\:/g;

      printf("Learning Spammail: %s\n",$filetolearn);
      my $result = su - $key -c "/usr/bin/sa-learn --spam $filetolearn";
      printf("Result of sa-learn: %s\n",$result);
      # Now delete the file after learning
      my $delete = su - $key -c "rm -f $filetolearn";
#      printf("Result of delete: %s\n",$delete);
    }
  }
}

--------------- snip --------------------
...