Koozali.org: home of the SME Server

SME 9 stopped passing email to internal mail server

Offline mmccarn

  • *
  • 2,626
  • +10/-0
SME 9 stopped passing email to internal mail server
« on: October 01, 2014, 01:12:19 AM »
My virtual SME 9 spam filter stopped passing email to my internal mail server at around 3:30pm today.

* Nothing was updated at that time
* Mail flows fine if I redirect port 25 in my firewall to another system
* If I direct port 25 to my SME server, in-bound connections on that port hang, but connections on other ports (ssh) work.
* If I reboot the SME server, 3 - 4 emails will be passed in from off-site before mail stops flowing again.

I will be setting up a new server to see if it behaves differently, but I would appreciate any ideas on where to look for the cause of my problem.

Offline Stefano

  • *
  • 10,839
  • +2/-0
Re: SME 9 stopped passing email to internal mail server
« Reply #1 on: October 01, 2014, 09:07:49 AM »
antything stange in the logs? (qmail/messages)

guest22

Re: SME 9 stopped passing email to internal mail server
« Reply #2 on: October 01, 2014, 11:13:34 AM »
Anything scheduled for cron at that specific time?

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: SME 9 stopped passing email to internal mail server
« Reply #3 on: October 01, 2014, 12:58:12 PM »
I don't see anything odd in the logs and I don't have anything unusual in cron (or anything specifically scheduled for 3:30pm, that I can find).

I should add that I restored my system from a veeam backup to a point at 11pm the night before, and the behavior didn't change.

My primary suspect at the moment is a problem in my firewall or local networking.

I wanted to mention it here in case I'm wrong and other users are seeing anything similar.

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: SME 9 stopped passing email to internal mail server
« Reply #4 on: October 01, 2014, 02:11:09 PM »
Update:

I have set up a new SME 9 server with a different LAN IP and it is processing mail as expected.

Setup notes:
- install from SME9 iso using 'noraid nolvm'
- yum update
- restore a 'backup to desktop'
 (note: I ran 'su admin' before rebooting in order to change the restored server IP and hostname)
- install vmware tools
- install qplogtail and qploggrep

Next steps:

I have a new firewall scheduled for installation in the coming days - I'll see if the new firewall solves the problem with the original SME spam filter.

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: SME 9 stopped passing email to internal mail server
« Reply #5 on: October 01, 2014, 11:18:52 PM »
* If I reboot the SME server, 3 - 4 emails will be passed in from off-site before mail stops flowing again.

That's a clue that there is a problem with logging. The logger cannot write to disk and once the pipe on its standard input fills up, whatever is writing to the pipe stalls.

Is the file system full? Are there permission problems?

Check 'runsvdir' in the output of 'ps fax'; any error messages from the loggers will show up there. You can make the error messages more readable via:

tr \\0 \\n < /proc/$(pidof runsvdir)/cmdline

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: SME 9 stopped passing email to internal mail server
« Reply #6 on: October 03, 2014, 01:28:29 PM »
Quote from:  # df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       45G  3.1G   40G   8% /
tmpfs                 947M     0  947M   0% /dev/shm
/dev/sda1             485M   70M  390M  16% /boot

Quote from:  # tr \\0 \\n < /proc/$(pidof runsvdir)/cmdline

runsvdir
-P
/service
log: ...
(There are 395 dots in the results but the forum only displays 3)

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: SME 9 stopped passing email to internal mail server
« Reply #7 on: October 03, 2014, 04:14:53 PM »
You'll have to identify which process isn't doing its job, and use strace to find out what it is doing instead of doing its job.

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: SME 9 stopped passing email to internal mail server
« Reply #8 on: October 03, 2014, 04:49:09 PM »
You'll have to identify which process isn't doing its job, and use strace to find out what it is doing instead of doing its job.
Thanks.

I'm not convinced the problem is with the SME server -- I am much more suspicious of my firewall, virtual infrastructure, or network switches.

I'll be coming back to this after my new firewall is installed and I have cycled power on my core switches.