Koozali.org formerly Contribs.org

Using SME for 1600-2600 mailboxes?

Using SME for 1600-2600 mailboxes?
« on: April 24, 2020, 11:48:34 AM »
Hello,

there's a government organization considering ways to move away from their existing mail server setups. Effectively they need something that can have the following:

*1600 mailboxes and possibly scale up to 2600
* Most users are via POP-3 (ie no mail stays on the server, so that eases disk space requirements)
* Should run in a ESXi VM
* Active directory integration would be a huge plus

Functionally-wise (ie AD integration) could SME fit the bill? And hardware-wise, what would it take to host these many mailboxes with the embedded antispam and antivirus?

Offline mmccarn

  • *
  • 2,557
Re: Using SME for 1600-2600 mailboxes?
« Reply #1 on: April 24, 2020, 01:33:40 PM »
Googling 'dovecot high volume server' leads to this discussion from 2015:
https://dovecot.org/pipermail/dovecot/2015-September/102017.html

In that thread there is discussion of dovecot installs with 28K total users and up to 4K mailboxes per server, with some server hardware listed.

I couldn't find anything else talking about the specific requirements for sizing a high volume server.

* You would likely need to adjust some of the SME default settings.
* You might want to have high performance storage (ssd-based or with an ssd cache)
* If possible, you may want to gather some specs on the current system if there is one
- new emails received per day
- emails sent per day
- average email size
- average mailbox size
- pop3, pop3s, smtp, and smtps traffic by hour/day/week


...and a warning:
I had an office many years ago where all users used POP3 but set their client to never delete mail from the server.  Everything worked fine until a user's mailbox reached a specific size, then bad things would happen. When the time required for the client to scan the mailbox exceeded the time between pop3 mailbox checks it started taking 2 to 4 hours for new mail to be seen by the pop3 client.  The "solution" for these users was to switch them to IMAP...

Offline ReetP

  • *
  • 2,795
Re: Using SME for 1600-2600 mailboxes?
« Reply #2 on: April 24, 2020, 03:52:27 PM »
Can't imagine anyone really using POP today, especially when you have to record stuff etc etc.

It also means they are limited to a single device which is inflexible (all those new home/remote workers who now have their email stuck on their desktop at work???). And storage is cheap......

Not sure on your time scale, but note the EOL date with CentOS6 & SME v9.

We are trying hard to get SME v10 ready, but as always, time, and hands. We have a few more volunteers working on it but there is a helluva lot to do. But focus should probably be on that.

If we do then you probably want to look at something like this for AD integration (depending on exactly what you mean but integration)

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/windows_integration_guide/introduction

Essentially SME can theoretically do anything that CentOS/Redhat can do. It is just a case of configuring it.

Daniel may have some ideas on this. He is usually around on our RocketChat instance. If you want a login to see what is going on then please ask me.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Re: Using SME for 1600-2600 mailboxes?
« Reply #3 on: April 25, 2020, 10:15:52 AM »
Thank you both for your answers. I am using pop3  because we have significant storage issues.

SME has performed adorably in smaller installations I've done, that is why it was my first thought.

I know that 9 is approaching EOL, but I'll keep using it until 10 is ready enough.

The dovecot thread game me some info, thank you for that as well!

Offline ReetP

  • *
  • 2,795
Re: Using SME for 1600-2600 mailboxes?
« Reply #4 on: April 25, 2020, 11:25:05 AM »
If storage is an issue (not sure why) can you not run a periodic scrub of mailboxes or something?

There are plenty of solutions where you can archive or delete older messages.

Is there any reason why you are limited on storage?

Just thinking if you said 2000 boxes with say 1Gb each that's a couple of Tb which is not large by todays standards?

Just curious!!

Bearing in mind also that this is for a government organisation I would also imagine security (so using pop3s at least) and keeping proper records are probably extremely important.

We have also been discussing PCI-DSS compliance and think we have some updated defaults that might be of interest.

We'll try & post some stuff soon.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Re: Using SME for 1600-2600 mailboxes?
« Reply #5 on: April 25, 2020, 06:55:29 PM »
When referring to POP-3 (or IMAP for that matter) I'm always referring to their encrypted counterparts.

As for the storage part, that's a painful story: the specific organization has gone some significant restructuring, meaning that equipment upgrades should be done for the entire organization, and not for parts of it. That it turn has turned an easy server upgrade task into a 5-year-going nightmare.

From the looks of it, things are going to be resolved finally and, if this thread is any proper indication, having 1Gb web/IMAP storage is doable! :)

EDIT: This is off-topic, but would you consider Horde, as implemented on SME 9, to be more secure or less secure compared to Secure POP3/IMAP/SMTP (ie using Thunderbird for example from the WAN, to access SME-hosted mailboxes)?
« Last Edit: April 25, 2020, 06:57:20 PM by Michail Pappas »

Offline ReetP

  • *
  • 2,795
Re: Using SME for 1600-2600 mailboxes?
« Reply #6 on: April 25, 2020, 10:41:58 PM »
KK - POP3s is really POP3 with a stunnel, as I have discovered.

We'll try and post how to update some defaults on that shortly.

Upgrade - SNAFU then !!!

IMAP - fab- that's the way to go...

Horde vs TB ?

Hmm - good question, IMAP I think should at least be considered safe to to manage....

As far as connections I think it is probably similar as they both use TLSv1.1 (if you have it set up correctly with the new defaults)

Someone else may give you more accurate guidance here though.

...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Jean-Philippe Pialasse

  • *
  • 1,598
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Using SME for 1600-2600 mailboxes?
« Reply #7 on: May 03, 2020, 06:18:27 AM »
Sme 9 is not using dovecot for pop3 but sme10 is now.

For the Out of the topic : the security will be on how the users handle their credential and how you check bruteforce attacks.

we juste secured the TLS protocols and ciphers on SME  10 to TLS1.2 and you could select the 4 stronger cipher you want on each services.
so again the game is just detecting bruteforce and avoid social engineering and phishing.

Offline Knuddi

  • *
  • 540
    • http://www.scanmailx.com
Re: Using SME for 1600-2600 mailboxes?
« Reply #8 on: May 14, 2020, 08:44:02 PM »
When having such a bunch of mailboxes, I would also strongly consider having a separate spam/AVfilter in front of the SME server. This will help with Bounce Storms and the recent Subscription Bombs I have experienced.

The SME server filter is OK but absolutely not (in my opinion) good enough for that many users and their (most likely) loose online behavior with their e-mail addresses.

Re: Using SME for 1600-2600 mailboxes?
« Reply #9 on: May 15, 2020, 12:25:29 PM »
When having such a bunch of mailboxes, I would also strongly consider having a separate spam/AVfilter in front of the SME server. This will help with Bounce Storms and the recent Subscription Bombs I have experienced.

Recommendations? Free / ClamAV alternative? Thanks.

Offline Knuddi

  • *
  • 540
    • http://www.scanmailx.com
Re: Using SME for 1600-2600 mailboxes?
« Reply #10 on: May 15, 2020, 12:33:20 PM »
Two possibilities - either a hard customized SME box in front only handling spam/AV. Pros: Cheap, Cons: Only one AV engine and limited filter and configuration capabilities. The alternative is either a cloud-based paid solution (I can obviously advocate for ScanMailX) or another Open Source solution. I have never tried https://www.mailcleaner.org/, but it could be a possibility.

Offline Knuddi

  • *
  • 540
    • http://www.scanmailx.com
Re: Using SME for 1600-2600 mailboxes?
« Reply #11 on: June 19, 2020, 08:20:27 AM »
Curious to hear whether you concluded on this one - did you decide to go with a SME server?

Re: Using SME for 1600-2600 mailboxes?
« Reply #12 on: June 19, 2020, 08:51:52 AM »
Curious to hear whether you concluded on this one - did you decide to go with a SME server?

Not really. Seems the mail admin is not pushing things forward for some reason. Some paid suggestions have been presented like from Kerio (too expensive) and from ESET (front-end antispam/AV only). Don't know where this will end...

Offline Knuddi

  • *
  • 540
    • http://www.scanmailx.com
Re: Using SME for 1600-2600 mailboxes?
« Reply #13 on: June 19, 2020, 09:28:19 AM »
What are they running on today? I mean if it's an old Exchange server with an old school AV/Spam plugin, then SME will be much better already. SME 9 has EOL in 2024, so you have a long time before SME 10 needs to be ready.

SME 9 can run with DKIM, SPF, DMARC and supports TLS 1.2. You can even get MTA-TLS and DANE to work as well if you really want to bring up security for your mail.

As an AV Clam does OK but is not anywhere bulletproof. With SpamHaus' new attachment HASH offering you could add an extra layer there. The SME spam filter can be customized to do well, and again likely better than an old server. But its an endless fight and the bad guys are getting better and better - SpamAssassin which is the core of SME does absolutely not do the job.

Consider testing with Hardenize to see how the old server performs. As a reference see ScanMailX - https://www.hardenize.com/report/scanmailx.com/1592551511

/Knuddi

Offline ReetP

  • *
  • 2,795
Re: Using SME for 1600-2600 mailboxes?
« Reply #14 on: June 19, 2020, 09:35:00 AM »
SME 9 has EOL in 2024, so you have a long time before SME 10 needs to be ready.

Not sure where you got that from?

EOL CentOS 6 is November 30, 2020

CentOS 7 is 2024

https://wiki.centos.org/About/Product

It's why we have been asking for ages for help to prepare v10....

Suggest you get testing if you want to see it released.

https://forums.contribs.org/index.php/topic,54246.0.html

...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Knuddi

  • *
  • 540
    • http://www.scanmailx.com
Re: Using SME for 1600-2600 mailboxes?
« Reply #15 on: June 19, 2020, 09:38:40 AM »
You are right ReetP - I looked wrong, my bad.

Offline ReetP

  • *
  • 2,795
Re: Using SME for 1600-2600 mailboxes?
« Reply #16 on: June 19, 2020, 11:58:20 AM »
:-)

Get testing then!!!
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation