Topic: greylisting comments

[meanlocha]

I started to play with greylisting to understand the issues.
SMESERVER 7.5

Here is what I did:
mkdir -p /var/lib/qpsmtpd/greylisting
chown -R qpsmtpd.qpsmtpd /var/lib/qpsmtpd
cd /etc/e-smith/templates/var/service/qpsmtpd/config/peers/0/
echo "greylisting" >09greylisting
signal-event email-update

The sme server gets about 1500 attempted mail connections a day, at least 1400 are spam.

The upside:
a) Some excellent points in its favor:
It works extremely efficiently by issuing DENYSOFT for all connections.
It then waits 50 minutes for mail servers to try again and if they do, adds them to the whitelist.
b) SPAM went to almost zero.

The downside:
a) many friends complained about mail being refused.
b) There seem to be lots of non-compliant mail servers out there, if you dont accept the first
attempt they dont try again. This is true, for example, for some maillist servers.
c) the default setting for the "grey" timeout, the period it waits after 50 minutes
for the second connection, is too short at 3 hours and 20 minutes. For example,
contribs.org mail did not make it through!
d) many major mail servers such as gmail and yahoo get it right and keep trying to send
the email message and it does go through. This is because gmail and yahoo used
the same mail-server each time to make the connection.
e) among the big losers was aol which changes the mail-server, and therefore the ip, from which
the email was sent. It would be unlikely that an aol message would ever get through.

Conclusion:
Greylisting is a great idea, but the downsides are too messy.

Anthony

Anthony

[chris burnat]

Moving to General Discussions section where this topic is more appropriate.

[piran]

The sme server gets about 1500 attempted mail connections a day, at least 1400 are spam.
...
Greylisting is a great idea, but the downsides are too messy.

Install the earlytalker qpsmtpd plugin.
http://wiki.contribs.org/Qpsmtpd_check_earlytalker
Anything >20sec pretty much gets the job done.
Mine is set for 120sec as  I'd like to have a life.
Spam effectively drops to zero.
I don't even bother installing spamassassin.
No friends get ticked off.
Occasionally I have to drop it to 15sec to encompass
surprisingly incompetent ISPs from whom I am expecting
subscriber mail. Overall it's highly effective - simple too.

[meanlocha]

Thanks
sounds like a good idea. I will give it a go.

Anthony

[CharlieBrady]

Install the earlytalker qpsmtpd plugin.
http://wiki.contribs.org/Qpsmtpd_check_earlytalker

The earlytalker is installed and enabled by default. The wiki page is long out of date, and no custom templates are required, unless you wish to tweak the timings as you have done. Adjusting the timing is unlikely to be required - if a virus is going to connect and try to send immediately, you will know that very quickly - no need to wait for two minutes.

[meanlocha]

I adjusted the wait time to 15 seconds with no noticeable
retarding effect on spambots. The early_talker's desire to talk
exceeds belief. They always seem to talk before even the standard 1 second is up.

I need to restrain myself now from making non-PC comments....
I have been forced to bite my tongue
by over-exposure to Californian Universities...

Anthony

[CharlieBrady]

Google for "site:contribs.org greylisting". You'll find that greylisting has been evaluated in the past, and found to be much more trouble than it is worth.

[piran]

The earlytalker is installed and enabled by default. The wiki page is long out of date, and no custom templates are required, unless you wish to tweak the timings as you have done. Adjusting the timing is unlikely to be required - if a virus is going to connect and try to send immediately, you will know that very quickly - no need to wait for two minutes.

Nothing to do with virus connections, never said it was.
Not using spamassassin was a throw away comment to
reinforce the beneficial use of the early_talker plugin,
which stops those that 'send before SMTP greeting'.
http://git.develooper.com/?p=qpsmtpd.git;a=blob;f=plugins/check_earlytalker;hb=HEAD
Typically those that do do so for commercial gain
from their spamming sponsors (or incompetence).
120secs gets the spamming/ISP hosts every time
or they drop their own connection meanwhile.
Either way no spam and nothing to do with viruses.

[piran]

Google for "site:contribs.org greylisting". You'll find that greylisting has been evaluated in the past, and found to be much more trouble than it is worth.

completely agree

[piran]

I adjusted the wait time to 15 seconds with no noticeable
retarding effect on spambots. The early_talker's desire to talk
exceeds belief. They always seem to talk before even the standard 1 second is up.

I need to restrain myself now from making non-PC comments....
I have been forced to bite my tongue
by over-exposure to Californian Universities...

Anthony

To significantly address spamming hosts my own
heuristic findings (fancy term for eyeballing logs)
indicate 65+ seconds is about right. A significant
number of known spamming hosts/ISPs use ~58sec
'timers'. So 65secs pretty much gets most of them.
I favour having a life and so use 120sec;~)
YMMV

[meanlocha]

My real beef is with multiple simultaneous spambot connections (SSC)

I dont like giving a spambot the benefit of my computing resources for all of 65 seconds.
Since quite a few spambots spray the server with 3 simultaneous connections (the default SME limit),
that ties up the server for one minute. If you are the only one behind the server you may
be able to tolerate that, but when there are several users sending emails, their send
may stall based on waiting for one of these timeouts. So this option does not work
as an option for my situation.

Since spamhaus is a terrific resource for denying the SSCs, it only
takes a few seconds to deny the connection, I am finding the SSCs less bothersome.

Anthony

[meanlocha]

I dont see how to easily avoid these simultaneous identical spambot connections.
The same spambot ips often repeat the exercise multiple times on a given day.
It would be nice to have a local cache of denied connections to minimize this.

Anthony

[CharlieBrady]

Nothing to do with virus connections, never said it was.

spambot/virus - does it matter what we call it? let's just say "non-legitimate mail transfer agent".

Typically those that do do so for commercial gain
from their spamming sponsors (or incompetence).
120secs gets the spamming/ISP hosts every time
or they drop their own connection meanwhile.

Do you have any evidence that these spammers have MTAs which send without waiting for a banner message? Or are you just saying that they have a shorter-than-average connection timeout?

I can set my MTA to not show a login banner for, say, ten minutes. I won't get much spam. But I won't get much legitimate email either.

[piran]

I dont see how to easily avoid these simultaneous identical spambot connections.
The same spambot ips often repeat the exercise multiple times on a given day.
It would be nice to have a local cache of denied connections to minimize this.

If they use the same IPs then dealiing with them permanently
would take seconds (depending on your typing skills)...
http://wiki.contribs.org/Firewall#Custom_templates
...I keep a local database of just such for that very reason.
Your SMEserver, your IP, would become effectively
invisible or unworkable to these people - no www and no emails.
Their spamming procedures are rendered completely ineffective.

[piran]

Do you have any evidence that these spammers have MTAs which send without waiting for a banner message? Or are you just saying that they have a shorter-than-average connection timeout?

I can set my MTA to not show a login banner for, say, ten minutes. I won't get much spam. But I won't get much legitimate email either.

Evidence? Observation of the qpsmtpd log, maintenance
of a local database and experimentation with the delay.
Observation of the logs shows how long the spamming
hosts wait before 'blurting'. I don't have data on
'shorter-than-average' connection timeout,
I didn't think averaging would be helpful.
 
I haven't observed legitimate stuff (potentially) losing out
with earlytalker set to less than 15secs. Currently I have
two legitimate areas that fall foul with short timeouts.

Adaptec dot com and the getsatisfaction dot com organisation
that runs the emails for the ClamAV for Windows cloud-based
A/V forum. If I need to talk to Adaptec I have to reduce the
120secs. The getsatisfaction hosts seem unwilling to raise
their existing timeout setting so none of the associated forum
stuff gets delivered unless I reduce my earlytalker setting.

Your forum, Charlie, and that of the associated bugzilla
properly waits for my own SMTP greeting before sending.
Legitimate MTAs properly wait. Spamming hosts don't.
Spamming hosts have very low timeouts to maximise
their income from their funding spammers.

It's up to your own local criteria to establish the extent.
If ten minutes loses you legitimate emails then don't set it for 
ten minutes. I have found two minutes to be more than ample,
65+ seconds pretty much loses the vast majority of (spamming)
transaction attempts.

I have established to my satisfaction that all traffic accurately
assessed to be crud attempts to be transacted from MTAs
that are set to tolerate little or no delay with SMTP greeting.
Locally... 15sec stops most spam, 65secs stops almost all
spam and 120sec stops all of it. Be aware that legitimate
emails from low timeout MTAs may be affected. However
as I keep an eye on the logs I am certain to see this
quite rare occurrence and take appropriate action.
YMMV

PostEdit: typo: insert (spamming)
...vast majority of (spamming) transaction attempts.