Koozali.org: home of the SME Server

command line to clean out bloated admin inox

Offline ber

  • ****
  • 239
  • +0/-0
command line to clean out bloated admin inox
« on: February 08, 2019, 04:17:38 AM »
Hi guys. My admin inbox has been bloated because of a monitoring software wrongly configured. Trying to cleanout the inbox and the deleted folder is almost inaccessible using webmail or a email client software- 64K plus emails.
Is there a quick script i can run to clean out the admin inbox and the deleted.

Thanks in advance.

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: command line to clean out bloated admin inox
« Reply #1 on: February 08, 2019, 01:09:34 PM »
Thanks for asking -- I discovered looking into this that I had over 500K emails in my admin Inbox...

You can delete the extra emails from the command line.

These commands will completely empty the Inbox for admin:
Code: [Select]
find /home/e-smith/Maildir/cur -name "*$(config get SystemName)*" -exec 'rm' "{}" \;
find /home/e-smith/Maildir/new -name "*$(config get SystemName)*" -exec 'rm' "{}" \;

Notes:
* /home/e-smith/Maildir - admin's mail folder
* /home/e-smith/Maildir/new - new messages - never touched by an email client
* /home/e-smith/maildir/cur - messages that have been accessed by an email client (whether "read" or "unread")
* $(config get SystemName) - get the config variable "SystemName".  SME includes this value in the name of each message filename by default.  You could probably replace -name "*$(config get SystemName)*" with -type f, but I did my testing using the commands shown.

If you want to delete only the emails related to the mail loop (or whatever), find some text specific to the unwanted messages, modify the "find" command, and feed the output to "xargs rm":

In this example, I want to remove only the emails containing the string "MaxHwTemp":
Code: [Select]
find /home/e-smith/Maildir/cur -name "*$(config get SystemName)*" -exec grep -l "MaxHwTemp" "{}" \; |xargs rm
find /home/e-smith/Maildir/cur -name "*$(config get SystemName)*" -exec grep -l "MaxHwTemp" "{}" \; |xargs rm

If you want to avoid slowing your server down while the cleanup is running, put "nice" at the start of each command line:
Code: [Select]
nice find /home/e-smith/Maildir/cur -name "*$(config get SystemName)*" -exec 'rm' "{}" \;
nice find /home/e-smith/Maildir/new -name "*$(config get SystemName)*" -exec 'rm' "{}" \;

Be patient.

My server (ARM processor, spinning HDD) was deleting messages at a rate of 25 - 55 per second without "nice" and (unexpectedly) closer to 80 per second with "nice". 500K messages will take over 5 hours to clean up at 25/second, or 1.5 hrs at 80/sec

There is no output from the running command.  Consequently, the workstation you're using could go to sleep and disconnect the session unexpectedly.  To avoid this, run the commands from the console, or from a desktop that does not go to sleep, or use "screen" so you can start it up and disconnect until it's done.
https://www.rackaid.com/blog/linux-screen-tutorial-and-how-to/

[edit]fix time estimate for 25 messages per sec
« Last Edit: February 08, 2019, 01:20:54 PM by mmccarn »

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: command line to clean out bloated admin inox
« Reply #2 on: February 09, 2019, 02:07:48 PM »
[Update]

The commands I posted yesterday reduced the message count in /home/e-smith/Maildir/cur from 500,000+ to 947, but afterwards I could not access the Admin mailbox using sogo, and my system was pegged at a load average of 40+ with the CPU at 95%+, with lots of busy "dovecot-lda" processes.

Since I didn't need anything from the Admin mailbox, I tried doing this:
Code: [Select]
cd /home/e-smith
sv d dovecot
mv Maildir Maildir.xxx
mkdir Maildir
chown admin:admin Maildir
sv u dovecot

At this point, I could get to the Admin mailbox from sogo (no messages), and the various dovecot-specific index, acl, and uid files appeared in the new /home/e-smith/Maildir folder.  However, my system's load average was still 40+ with 95%+ CPU.

On the assumption that the busy processes were related to dovecot attempting to re-index the Admin mailbox I did this:
Code: [Select]
kill -9 $(pgrep dovecot-lda)
signal-event email-update

CPU was better, but load average was still around 20 (not 0.20).  lsof showed that there were still files in use by "imap" from the renamed "Maildir.xxx" folder, so I did this to kill those processes:
Code: [Select]
lsof |grep Maildir.xxx |awk '{print $2}' |sort |uniq |xargs kill -9

I think I could have avoided the various kill commands if I had started out by renaming and recreating "Maildir" yesterday instead of deleting the individual messages and leaving dovecot to clean up the indexes on its own, or possibly if I had included "sv d imap" along with "sv d dovecot" before renaming Maildir, plus "sv u imap" after the rename.

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: command line to clean out bloated admin inox
« Reply #3 on: February 09, 2019, 03:23:32 PM »
I think there is something you can do to delete out the old dovecot index files.

That lets it just reindex the few left rather than having to contrast & compare what has changed.

Have done it in the past but can't remember which files to dump. Sure there is stuff online.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: command line to clean out bloated admin inox
« Reply #4 on: February 10, 2019, 03:28:09 PM »
Simply delete the old index file they will be created from scratch. Which is faster than erroring on all deleted mail still in the index.

Offline ber

  • ****
  • 239
  • +0/-0
Re: command line to clean out bloated admin inox
« Reply #5 on: February 11, 2019, 01:36:36 PM »
Hi mmccarn
After running the script you recommended the server is now constantly running very high CPU usage. Can you advise how to fix.

[root@server e-smith]# top
top - 09:28:04 up 17:40,  1 user,  load average: 10.65, 6.47, 5.82
Tasks: 239 total,  10 running, 229 sleeping,   0 stopped,   0 zombie
Cpu(s): 98.2%us,  1.7%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.2%hi,  0.0%si,  0.0%st
Mem:   4045920k total,  3164504k used,   881416k free,   303560k buffers
Swap: 16506876k total,    62204k used, 16444672k free,   585516k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7748 qpsmtpd   20   0  248m 139m 3588 R 24.4  3.5   0:20.09 qpsmtpd-forkser
 7740 qpsmtpd   20   0  252m 142m 3588 R 24.1  3.6   0:21.66 qpsmtpd-forkser
 7749 qpsmtpd   20   0  249m 140m 3588 R 24.1  3.6   0:20.29 qpsmtpd-forkser
 7753 qpsmtpd   20   0  249m 140m 3588 R 24.1  3.5   0:18.67 qpsmtpd-forkser
 7743 qpsmtpd   20   0  250m 140m 3588 R 23.4  3.6   0:21.04 qpsmtpd-forkser
 7744 qpsmtpd   20   0  251m 142m 3588 R 23.4  3.6   0:21.87 qpsmtpd-forkser
 7752 qpsmtpd   20   0  248m 139m 3588 R 23.4  3.5   0:18.95 qpsmtpd-forkser
 7758 qpsmtpd   20   0  248m 139m 3588 R 23.4  3.5   0:18.46 qpsmtpd-forkser
 7836 root      20   0 35008 4300 1028 S  1.0  0.1   0:00.07 plymouthd
 7976 root      20   0  106m 1872 1232 S  0.7  0.0   0:00.02 ifdown-eth
  258 root      20   0     0    0    0 S  0.3  0.0   1:11.37 md1_raid1
  964 root      20   0     0    0    0 S  0.3  0.0   0:42.39 kondemand/0
  965 root      20   0     0    0    0 S  0.3  0.0   0:41.16 kondemand/1
 2081 qpsmtpd   20   0  108m  22m 3644 S  0.3  0.6   0:03.19 qpsmtpd-forkser
 7736 root      20   0 15160 1380  920 R  0.3  0.0   0:00.54 top
 7840 root      20   0  106m 1788 1256 S  0.3  0.0   0:00.02 rc
 7949 root      20   0  106m 1972 1288 S  0.3  0.0   0:00.01 K90network
 ...

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: command line to clean out bloated admin inox
« Reply #6 on: February 11, 2019, 03:04:36 PM »
I'm not sure you told us the whole story here. I presume you are still receiving lots of messages?

Check the forums for:

qpsmtpd-forkser high load

First thing to do is check your logs.

/var/log/qpsmptd/current
/var/log/messages

Somthing is keeping qpsmtpd busy, so what is it?
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation