Topic: sme 9.2 smtp problem (qpsmtpd)

[tomeratch]

Hi everyone 2 days ago my sme 9.2 server started to act strange after a regular reboot.
all mail client connected get  timeout on sending mail (mail is stuck in outbox in outlook)
it seems smtp service has a problem it all starts randomly .. mail takes more than 20 seconds to leave outbox to a point that all mail clients stop delivering mail (stuck in outbox as mentioned) .
at the beginning i rebooted the server and it woks fine until the problem just comes back randomly sometimes after 24 hours and sometimes after 30 minutes .I found that restarting the qpsmtpd service also fixes the issue until the problem returns randomly.
i used  tail -f /var/log/qmail/current when the problem occurs but found nothing strange ..or I'm not skilled enough to see something out of order.
I even used affa to raise my backup server to find that the same happens on that server as well.
any help would be appreciated 

Thanx in advance Tom

[ReetP]

Hmmmm. Sounds more like a network or Outlook issue than the server, especially if it is both the normal and affa one (did you ensure the affa one was updated the same as the main one?)

First thing is to probably try using webmail.

That will isolate the server. Do mails go normally from a webmail account?

If so then run some network tests, and check if Outlook is throwing any errors.

[tomeratch]

Thank you for your reply .
I did check with webmail(Horde) and its the same problem, cannot send mail trough webmail as well.
I did make sure that both the production and the backup server are with the same rpms .
is it possible that some config file get damaged or changed and that is why even my backup has the same problem for it is replicated to the backup server?.
the problem occurred again today i was thinking of making a script restarting the qpsmtp service every hour or so.
would it help if i post the qpsmtp log(or any other log you suggest) while the problem happens ? .

Again thank you for your help in advance

[ReetP]

Remember that config files are regenerated from templates. So unless you have a hacky template, or template override, a signal-event post-upgrade/reboot should sort out your configs.

One thing to check.... what is the server load like? Just wondering if you have had a hack of some description that is putting a high load on the server?? Just a thought.

If we suspect it might be a config issue the two things to look at are what other contribs have you installed that may affect things, and what do your logs say?

You can generate a report of new rpms etc from server-manager (can't remember where exactly)

Or use newrpms from this:
https://wiki.contribs.org/Audit_Tools

If you have a lot of info I'd suggest posting on say pastebin after removing anything sensitive.

Troubleshooting email here
https://wiki.contribs.org/Email

Check your messages/qpsmtpd/qmail/sqpsmtpd logs for errors:
https://wiki.contribs.org/Log_Files

Don't just post swathes of stuff. Grep for errors and see what you can find.

https://wiki.contribs.org/Useful_Commands

Somewhere on the wiki are some grep commands that will look for errors or warnings in logs. Can't find it on mobile, and I'm about to take off.... !

[tomeratch]

Thank you ReetP
 I also think it might be a load issue and that is why server gets slow and smtp delivers mail but slowly at the beginning to a point it stops delivering .
and weird that restarting the service fixes the issue .
any tips how to monitor smtp load?or how to find the session overloading (if any) ?
i did monitor the network load to the server and it looks ok so its not a network issue.
any help is appreciated i have minimal knowledge.

[Daniel B.]

You need to look at the qpsmtpd and/or sqpsmtpd logs during a transaction

[brianr]

Is is possible you have a compromised PC on your network which is trying to send lots of emails?

[tomeratch]

Thanks brianr .. might be a compromised pc and it would explain the randomness.. how can I check from the server side?
I have more than 200 workstations so I would like to monitor such traffic from the server side .

Thanks in advance

[ReetP]

Have a look at contribs. Sysmon & think there is another.

[brianr]

Have a look at contribs. Sysmon & think there is another.

The mailstats contrib might help. although you'd have to leave it a day before the first report is delivered.

https://wiki.contribs.org/Mailstats

[tomeratch]

Thanks ReetP seems that mailstats it the tool for the job..I'll set it up and try to find in the daily log any suspicious traffic.
I did set up sysmon as well but it is not monitoring any smtp sessions or loads so I'll try mailstats for now.
thanks again

[ReetP]

Might be worth a combination (it won't hurt) so you monitor mail traffic, and server load as well.

[brianr]

Thanks ReetP seems that mailstats it the tool for the job..I'll set it up and try to find in the daily log any suspicious traffic.
I did set up sysmon as well but it is not monitoring any smtp sessions or loads so I'll try mailstats for now.
thanks again

If you need help interpreting the mailstats report then post here.  Can do it by pm if necessary.

[Catton]

I too am getting this same random slow or stuck Windows 10 - Outlook 2010 - outbox. Past 2 days as of 10-18-2018
what has worked so far was to set Outlook outbound from SSL-465 to auto-25.
A send and receive all - cleared the outbox.
I haven't yet checked the SME logs.

# tail -f sqpsmtpd/current
@400000005bc7825b02084574 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc7825c020c9ebc 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc7825d02112acc 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc7825e0215226c 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc7825f0218facc 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc78260021d0dc4 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc78261022118ec 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc782620224fd04 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc782630228d94c 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc78264022d0b84 3743 Too many connections: 10 >= 10.  Waiting one second.
@400000005bc782650230fb54 3743 Too many connections: 10 >= 10.  Waiting one second.

I think this is caused by a rough PC, But it cleared before I could narrow it down. Now sending is working again and the log looks normal.

If I understand the log correctly sqpsmtpd is SSL port 465.
If this happens again - How can I tell what PC/ip address is pounding sqpsmtpd port 465 ?

[ReetP]

Rule 1. Do no harm

Always check your logs before you leap in and randomly change things.

So, what do your logs tell you?

For instance I have been getting timeouts on non Windows:

https://forums.contribs.org/index.php/topic,53769.0.html