Topic: Some questions on spamassassin and junk

[itguy2012]

Hi, I'm looking to tighten up the spam settings, and had a few basic questions...

1. I assume from http://wiki.contribs.org/Email#Spamassassin that by just setting these option from the web interface that spamassassin is then installed enabled and running?

2. in mentions the junk folder - is this per user? Ideally I'd like to be able to have one overview of what's been junked/rejected etc and change their status if applicable - rather than having people have to do it themselves.

Is there a default/recommended web interface for spamassassin to enable me to do this?

many thanks!

[mmccarn]

SME server delivers spam to a folder named 'junkmail' for each user.  This setting is controlled by the template fragment found here:
/etc/e-smith/templates-user/.qmail-junkmail/10deliver_to_junkmail

I don't know of any easy way to create a common junk folder for the entire server, and I don't know of any easy way to create a 'quarantine' system for SME server such as some spam filters provide. 

I supposed you could create a custom template fragment to override the default in order to forward spam email to a common mailbox - but that wouldn't let you 'release' it to the user's inbox -- you'd need to forward it back to them instead. (I found this forum discussion describing exactly this scenario from 2007: http://forums.contribs.org/index.php/topic,35178.msg153410.html#msg153410)

From the server itself you can get stats on overall spamassassin performance from the logfiles in /var/log/qpsmtpd

There is a wiki page showing various ways to get stats on mail server performance:
http://wiki.contribs.org/Email_Statistics

I highly recommend configuring Bayesian Autolearning and enabling DNSBL on your server, in addition to spamassassin:
http://wiki.contribs.org/Email#Bayesian_Autolearning
http://wiki.contribs.org/Email#Real-time_Blackhole_List_.28RBL.29

At present, I am using the following RBLs:
# config getprop qpsmtpd RBLList
zen.zpamhaus.org:bl.spamcop.net:truncate.gbudb.net:bl.nosolicitado.org:ix.dnsbl.manitu.net

Additionally, I have created a custom template fragment to allow me to use the Barracuda blacklist as described here:
http://bugs.contribs.org/show_bug.cgi?id=8484#c8

[devtay]

I have a follow up question on this. I currently use the Learn contrib (http://wiki.contribs.org/Learn) and I have baysean autolearning (http://wiki.contribs.org/Email#Bayesian_Autolearning) and custom settings in my email panel in server manager. In particular, I was interested in the "create a common junk folder for the entire server" statement.

Our setup has all of the clients connecting to the server via POP3 to get email in Outlook. I know this is old and I know IMAP is better. I use it myself but my users are not receptive to change no matter the benefits. That means they can't sort email into the learnasspam and learnasham email folders for individual accounts. When I setup the email server, I enabled BCC (http://wiki.contribs.org/Email#Keep_a_copy_of_all_emails). So, I recently started to sort spam email received in my maillog account into the learnasspam folder for the maillog account. This represents all of the email for the entire server. I have verified the server is learning tokens for the spam I'm manually sorting and the email generated by Learn is listing the scanning.

This all comes down to am I wasting my time by sorting in the maillog account? It seems like I have been getting a lot more spam lately. I think my setup could be considered as a common place to sort junkmail close to what itguy2012 was looking for aside from the quarantine system.

This could be a thread hijack. It's pertaining to the same subject so I thought what the heck.

[mmccarn]

I'm seeing more spam lately, too.

In addition to the dnsbl settings mentioned in my earlier post I have created a "private" dnsbl setup on my Windows Active Directory controller, to which I add IPs any time I receive obvious, off-the-wall spam.  Usually, the IP in question gets listed by one of my other dnsbl services within 12 - 24 hours.

Having said that, the only point in having a server-wide "junkmail" folder would be to let you over-tighten your spamassassin settings, knowing that you could 'release' mis-identified HAM back to the users.

You may want to reset your bayes database as shown here:
http://wiki.contribs.org/Email#Reset_the_Bayes_Database

[itguy2012]

mmccarn - thanks for your replies there's some useful info there. I'd looked at the http://wiki.contribs.org/Sme-unjunkmgr plugin which I like the idea of, but would rather not trouble users with having to do this, we don't have an enormous amount of spam but it's still preferable to get the admin of it done behind the scenes.

Another reason I'd looked at doing this is because each account is forwarded on to another server - so the users themselves don't actually connect to the sme server normally so don't want to complicate usage.

devtay - I think we're looking for similar ideas there, no worries for the hijack

[devtay]

Thanks for the advice. It's a little scary to drop all of that data from the bayes database but I went ahead and did it. I have about 3500 known spam emails that I fed into my learnasspam folder on the server so next time Learn runs, I'll see how it goes. From the initial testing, I can see the tokens cranking up when I dump magic and I can see the emails coming into my maillog account getting tagged as spam by my server. All seems to be functioning now. I know it's been more than 2 years since the database was empty.

My server processes about 6000 emails a day so I should have a pretty good indication by tomorrow morning if this helped or not. Again, thanks. I don't read through the howto's often and I missed that one.

I'm seeing more spam lately, too.

In addition to the dnsbl settings mentioned in my earlier post I have created a "private" dnsbl setup on my Windows Active Directory controller, to which I add IPs any time I receive obvious, off-the-wall spam.  Usually, the IP in question gets listed by one of my other dnsbl services within 12 - 24 hours.

Having said that, the only point in having a server-wide "junkmail" folder would be to let you over-tighten your spamassassin settings, knowing that you could 'release' mis-identified HAM back to the users.

You may want to reset your bayes database as shown here:
http://wiki.contribs.org/Email#Reset_the_Bayes_Database

[devtay]

I thought about a new topic on this one but I'd rather have all the info in one place. Let me know if I mess something up by posting this here and I'll make a new thread. No problem.

Anyways, I tried all the steps above and it looks like spamassassin is working for the most part. The problem is, I still get a ton of junk emails through the server no matter how often I sort to LearnAsSpam.

Seems like the same types of email are getting through. When I do a blacklist lookup on MXTOOLBOX.COM, the address or domain isn't on any blacklist. I started checking and was able to trace a big bulk of the email to certain ip addresses. I started using the firewall rules to drop the connection to the server based on IP address (custom masq template). What I found is the source IP address for the spam emails just changed on the same subnet. After adding about 60 /32 entries, I changed the entry and did a /24 to cover the whole thing. It's helping but what I'm worried about is stopping legitimate email from that IP range. Surely every single host on 173.233.129.1/24 isn't a spam source.

Maybe the email isn't spam but UCE? Am I thinking about this correctly? Should I be looking at trying something else first? I'm almost to the point where I'm just going to buy a solution. Frustrated. Any help would be greatly appreciated. It's very time consuming for me to look at email headers and get IP Addresses out of them. Is there an easier way?

[CharlieBrady]

Maybe the email isn't spam but UCE?

AFAIK, spam and UCE are one and the same thing.

[devtay]

AFAIK, spam and UCE are one and the same thing.

You are correct sir. I'm just trying to come up with solutions to why these IP Addresses haven't been flagged and spamassassin won't learn the messages as spam.

[janet]

devtay

Chasing & blocking IP addresses is a temporal solution only. Spammers etc just change servers & get a new IP, so it will be a never ending effort on your part, & time consuming.

Have you enabled executable content filtering/blocking in server manager Email panel ? It takes care of a lot of virus infected spam messages.

[devtay]

Janet,

Thanks for the reply. I do have executable content blocking enabled. It helps sometimes. I still get some versions of zip files through both clamav and the content blocking filter. Most of the time our antivirus catches the files when they get downloaded into Outlook so that helps too. I still have the oblivious user that can't understand why they screwed up by downloading and opening that zip file from the nice prince that was going to give them a million bucks.

Here is an example of what I've been doing up to this point. I use all the DNSBL settings as the wiki suggests. I have custom filtering enabled and my numbers are 4 (tag) and 11 (reject). I've installed the Learn contrib and I have baysean filtering enabled. I have been feeding spam emails into the LearnAsSpam folders for literally months and the same types of messages get through. No change at all in the level of spam reduction. If anything, the SPAM volume has gone up. I even reset the bayes database to see if that would help. Basically I've followed everything I can follow and the SPAM level is crazy high. When I examine the X-headers to see what's going on, the SPAM coming in has like a -4 to 3.5 score. I understand that Spamassassin only scores and categorizes email so I'm not looking for a "block" from it. I just can't figure out why obvious SPAM (UCE in this case) is sneaking through the filter.

Anyways, I appreciate the reply. I think you are right about the IP Blocking. I'll start to negatively affect things before long with this policy. I think I need to find another program that can better classify SPAM or find a tweak to spamassassin to clean things up for me.

[janet]

devtay

There are other additional things admins here are doing with clamav & spamassassin (eg using other lists etc), so search & read.
There are examples mentioned in this thread
http://forums.contribs.org/index.php/topic,51501.0.html

Re blocking zip files, there were two new patterns added to the database in December 2014 but the updated packages are still in one of the test repos.
It was mentioned recently in Forums and/or bugzilla, so if you installed that update manually (using yum & pointing at the additional repo with --enablerepo = ....) then you will block those zips, as long as you enable them in server manager Email panel.

That may give you the improvement you are looking for.

[devtay]

Thank you for the pointers, Janet. I'll do some more digging. I appreciate the help.

[janet]

devtay

Found it, take a look at
bug 8717 & 8718 & 8835

[devtay]

Perfect. That's exactly what I need for my zip file problem. I'm going to install tomorrow am. Thanks. I owe you.