Topic: Email Issue: Whitelist does not work?

[mmccarn]

1) even if DATE_IN_PAST is over 1, they will be very close to limit value (5). Shouldn't a white list work and be more efective ?

Yes

2) where I can see what value has each check of SpamAssassin ?

I started with a google search for the exact rule name as shown in the message header, then poked around a bit.  I ended up on the spamassassin 'tests' page, and searched within the page for the rule name:  http://spamassassin.apache.org/tests_3_3_x.html

3) do you know why to add e-mail address in whitelist of SA do not release it from being checked ?

No - this is puzzling. 

I have only one SME that uses any spamassassin whitelisting -- I usually use bayes filtering 'learnasspam' and 'learnasham' to avoid managing a whitelist.

On my SME that does use whitelists, the whitelists are kept in the the 'spamassassin' db, and the results are templated into /etc/mail/spamassassin/local.cf by the template fragment /etc/e-smith/templates/etc/mail/spamassassin/local.cf/60globalWBL.

Take a look at /etc/mail/spamassassin/local.cf - it should contain a line that says:
whitelist_from mtarbox@metalplate.com

If you see that entry already, you might want to add an upper case version (although if this fixes your issue, it's worth raising a bug with apache against spamassassin):
whitelist_from MTARBOX@METALPLATE.COM

If you see a rule that you think uses a wildcard, be aware that spamassassin email address whitelists require a wildcard character, so this would match any sender from "@metalplate.com":
whitelist_from *@metalplate.com

But this not match any valid sender email address:
whitelist_from @metalplate.com


To generate the whitelist entry I think you want in /etc/mail/spamassassin/local.cf, you would use these commands (upper case "W" in "White" is required):

Code: [Select]

db spamassassin setprop wbl.global mtarbox@metalplate.com White
signal-event email-update


[ghorst352]

Do you have any idea why the Pseudonym fails but the actual email address works?  I have never seen an email issue like this ever. 

[janet]

bhay3s

Do you have any idea why the Pseudonym fails but the actual email address works?  I have never seen an email issue like this ever.

If you want answers like that, then it would help to answer my earlier questions.
ie
Please confirm exactly what/where/which whitelists you are adding entries to, is it a contrib you have installed, if so which one ? Please describe fully.

Also your dedicated sme mail server, is it a gateway server configuration ? Please describe fully the network layout and where this machine is connected etc and what other servers are in the "mix" ? Thanks

[ghorst352]

bhay3s

If you want answers like that, then it would help to answer my earlier questions.
ie
Please confirm exactly what/where/which whitelists you are adding entries to, is it a contrib you have installed, if so which one ? Please describe fully.

Also your dedicated sme mail server, is it a gateway server configuration ? Please describe fully the network layout and where this machine is connected etc and what other servers are in the "mix" ? Thanks

Refer below:

qpsmtpd whitelisthosts:
67.220.101.207 <- the ip of the smtp relay that metalplate uses

qpsmtpd whitelisthelo:
metalplate.com
mtarbox@metalplate.com

qpsmtpd whitelistsenders:
@metalplate.com

spamassassin whitelist_from:
@metalplate.com
mtarbox@metalplate.com

There is redundancy as you will notice throughout the whitelist.  Before somebody makes an obvious comment that you didn't need to do both the @domain and the actual email address, yes of course I realize that.  I tried both just because.

bhay3s

If you want answers like that, then it would help to answer my earlier questions.
ie
Please confirm exactly what/where/which whitelists you are adding entries to, is it a contrib you have installed, if so which one ? Please describe fully.

Also your dedicated sme mail server, is it a gateway server configuration ? Please describe fully the network layout and where this machine is connected etc and what other servers are in the "mix" ? Thanks

The server is not a gateway server.  Just a standalone email server. 

ip:
192.168.1.7
255.255.255.0
192.168.1.254

the router config the server sits behind:
WAN IP: 69.69.69.x (for ex.)  The public ip is routed via the Wan->Lan policy as you can see below.
Firewall settings:
WAN -> LAN POLICY:
pop3s    Allow    Any    69.69.69.x    TCP 0 -> 995
smtpin    Allow    Any    69.69.69.x    smtp-in
SMTPS    Allow    Any    69.69.69.x   TCP 0 -> 465

I have 5 other servers that are standalone servers, 2 more SME boxes, 1 proxy box, 2 win term servers.   How does a network layout affect spamassassin btw? Just curious.



Yesterday I called the client to discuss this issue further and this is what I found out.  They basically scan the invoices in batch during the day and at night around 7:00PM its automated to send the invoices via email.  This would be the reason for the DATE_IN_PAST_03_06 failure I would believe.  Plus this is the body of the email:


THIS EMAIL IS AUTOMATICALLY GENERATED AND IS UNABLE TO ACCOMODATE REPLIES.
PLEASE CONTACT METALPLATE AT YOUR NEAREST LOCATION IF YOU HAVE ANY
QUESTIONS.  AS ALWAYS, THANK YOU FOR YOUR BUSINESS!

TERMS AND CONDITIONS: WWW.METALPLATE.COM/PAPERLESS.PHP

* Notice the email is all uppercase which is responsible for another failure in regards to spamassassin.   

[janet]

bhay3s

the router config the server sits behind:
WAN IP: 69.69.69.x (for ex.)  The public ip is routed via the Wan->Lan policy as you can see below.
Firewall settings:
WAN -> LAN POLICY:
pop3s    Allow    Any    69.69.69.x    TCP 0 -> 995
smtpin    Allow    Any    69.69.69.x    smtp-in
SMTPS    Allow    Any    69.69.69.x   TCP 0 -> 465

I have 5 other servers that are standalone servers, 2 more SME boxes, 1 proxy box, 2 win term servers.   How does a network layout affect spamassassin btw? Just curious.

I'm not sure what your router is doing there re incoming smtp.
There are many external lookups & smtp transactions done by the gateway smtp server that assist mail robustness & filtering. Search the forums, as much has been said over the years.
Refer http://wiki.contribs.org/SME_Server:Documentation:FAQ#Server_Only

Your gateway (server) should ideally handle ALL incoming mail, which implies the gateway (router) should be a sme server in server gateway mode.

If you wish to have mail handled by another internal (or external) dedicated mail server, then you should setup mail delegation on the gateway server (either all domains or on a per domain basis if required). The gateway smtp server will receive the mail transactions and then pass the mail onto the dedicated specified mail server for further processing. Spam filtering and such like will be carried out on the gateway server.
http://wiki.contribs.org/SME_Server:Documentation:FAQ#Internal_Mail_Servers

This may also be of interest (although not strictly tro do with your issue AFAICT)
http://wiki.contribs.org/SME_Server:Documentation:FAQ#Secondary.2FBackup_Mail_Server_Considerations

[janet]

bhay3s

Also is there a way to restart the process or service responsible for the whitelist?

I think this would be a better choice, which will restart everything associated with mail
signal-event email-update

[ghorst352]

Mary,

I appreciate your response.  Let me explain, I was grandfathered into this network as I am the new IT Director for my company about 6 months in.  I have a whole grocery list of items that I am always working on whether it's tightening up security, eliminating unneeded servers, etc etc.  Your right that would be a better way in regards to the config,  but this has absolutely nothing to do with the reason for this post.  I am trying to understand the mechanics of spamassasin and now why a Pseudonym fails but the actual address does not?  I am considering this case closed because I have rectified or worked around the issue by having the mail routed to the non-Pseudonym address and I also now have 3 legitimate reasons for the failures please refer below to the email I sent my client.



*************************************************************
After further research I have at least 3 legitimate reasons for some of the failures.

1.  The body of the email is all uppercase.  Most spam filters look for this.

2.  The fact that the invoices are scanned during the day and then scheduled to send out at 7PM is what I believe is responsible for the DATE_IN_PAST_03_06 failure.  This is another trick by spammers as they will schedule batch emails to be sent out so spam filters such as ours will trip a fail in regards to that particular test.

3.  If you notice all of the email address (names) listed by the scanner are all uppercase.  This is another fail.
From: MARIE TARBOX - METALPLATE
Sent: Tuesday, March 27, 2012 3:01 PM
To: CAROLYN WHITEHEAD
Cc: MARIE TARBOX ; GRANT HAUTH
Subject: Metalplate Galvanizing
 
If you have no issues as of right now in regards to clients receiving the emails then no worries, however if you do run into an issue in regards to spam filtration on the receiving end, then you have at least 3 things listed here that you can forward to your IT dpt in regards to modifying your scanner config.


Sincerely,

[janet]

jader

3) do you know why to add e-mail address in whitelist of SA do not release it from being checked ?

It will still check but not reject
From the WBL panel:
"Any envelope sender of a mail (*@host or user@host) matching an entry in whitelist_from will be exempted from spamassassin rejection."

[janet]

bhay3s

....  but this has absolutely nothing to do with the reason for this post.  I am trying to understand the mechanics of spamassasin and now why a Pseudonym fails but the actual address does not? 

Not so. As I understand it, your various issues (whitelisting not working correctly etc) may be due to running a mail server in server only mode.
The smtp transactions on a sme server in server only mode could be behaving differently than on a sme mail server configured in server gateway mode that is directly connected to the outside world (via bridged modem etc). To some degree it may depend on what your existing router is doing in smtp-in mode.

Over the years there have been many advices in these forums from the experts to put a sme server in server gateway mode to handle incoming mail to get the full benefits of sme server filtering techniques etc (smtp & qpsmtpd & spamassassin).

[mmccarn]

Do you have any idea why the Pseudonym fails but the actual email address works?  I have never seen an email issue like this ever. 

I feel like I have seen some bugs over the years about differing mail filter behavior for either groups or pseudonyms (I forget which, and I don't see anything obvious in the bug tracker).

[complete guesswork]
A possibility that occurs to me is that your SME server has "autowhitelist" enabled, which affects the spamassassin score for emails back and forth to *real* users (because they send email out to the vendor), but does not have the same effect on email sent to the pseudonym because the pseudonym is never used as the "from" address for emails that go out to this vendor.
[/complete guesswork]

You might figure out what's up by increasing the logging levels for qpsmtpd, spamd (and spamassassin, if there is a different log detail setting for that), then examining the logs to see what's going on.

I would be curious to see what happens if you used another sme user account (forwarding to the same recipient) instead of a pseudonym (that is - delete the pseudonym, create a new user using the pseudonym name, then configure the new user to forward all email to the person who is supposed to get it).  I think if my guess about "auto whitelisting" is correct, this would still fail.  If this works, then there's a problem buried in spamd or qpsmtpd related to how pseudonyms are handled, or there's something in the configuration of your SME that is causing this (custom templates? custom contribs?)

[ghorst352]

I just realized that probably my entire WBL is not working.  I just blacklisted my personal comcast email address and I was able to send email to my company account.  Something is awry with the my WBL???? 

[ghorst352]

I will be uninstalling the WBL and Learn contribs and then reinstalling WBL and then get back with my results.

[piran]

I will be uninstalling the WBL and Learn contribs and then reinstalling WBL and then get back with my results.

You might be able to whitelist, or blacklist but some systems
ONLY do one or the other and give erratic results attempting
both. Keep it in mind when next reading up...