Topic: [Announce] New version of the mailstats script (0.6.15)

[brianr]

I have re-written some of it to parametrise the columns that it shows and also added column and row totals and percentages.  The default is that some columns are not shown until a non zero count is found.  It uses DB entries to control this. 
See the header on the script for details.

I also added league tables for the qpsmtpd reason codes, and also for
Spamassassin rule hits.  Db entries control if you want to see these as well (the default is yes).

Download from my contribs area ("bread"),

http://distro.ibiblio.org/pub/linux/distributions/smeserver/contribs/bread/mailstats/

or from the bugzilla entry

http://bugs.contribs.org/show_bug.cgi?id=3068

Enjoy! all bugs to bugzilla please.

Still got a couple of outstanding bugs:

1. XferErr column not populated.
2. It ignores emails to multiple recipients now (it didn't used to)

I Also have some more ideas about where the script might go, ideas are:

1. Sort the junkmail counts in order of emails left
2. Output report in html to an ibay
3. Output stats to a mysql database, and develop some php to display the results on a web page.

Any Thoughts, or other ideas?

[jfarschman]

Hey Brian,

  This is cool stuff 

  But I have one questions about the parameters.  Where can I find the 'column header' parameters?  I've poked around in the code and got a little confused around lines 307-332.  If I understand correctly these categories are all dynamically generated from the log files?

Code: [Select]

config setprop mailstats <column header> ("yes"|"no"|"auto")
Anyway it's a nice tool.  Thanks.

[brianr]

Where can I find the 'column header' parameters? 

the parameters are in the database under the "mailstats" heading, and are exactly the same characters as the column headings.
See here from my main server:

# config show mailstats
mailstats=report
    Del.Spam=yes
    Email=smeserver
    Fetchmail=yes
    Ham=yes
    Local=yes
    Mailman=yes
    Non.Conf.=yes
    Qued.Spam?=yes
    RBL/DNS=yes
    Status=enabled
    Virus=yes
    WebMail=yes

the column entries are created  and set to "yes" if a non zero count is detected (they may not exist before). "Yes" means that that column is then always shown. If you delete the entry or set it to "auto" then it will only show if a non zero occurs again (and then get set to "yes").  I suppose a 4th option would be to only show if non zero, and supress otherwise.

Anyway it's a nice tool.  Thanks

no problem.

[jfarschman]

Thanks Brian,

  That's a great way to control the output.

[jfarschman]

Brian,

  I ran into a little trouble.  My column heading do not write to the configuration DB

Code: [Select]

db config setprop mailstats Del.Spam yes
db config setprop mailstats Email smeserver
db config setprop mailstats Fetchmail yes
db config setprop mailstats Ham yes
db config setprop mailstats Local yes
db config setprop mailstats Mailman yes
db config setprop mailstats Non.Conf. yes
db config setprop mailstats Qued.Spam? yes
db config setprop mailstats RBL/DNS yes
db config setprop mailstats Status enabled
db config setprop mailstats Virus yes
db config setprop mailstats WebMail yes

[root@mail ~]# db configuration show mailstats
mailstats=service
    Email=jay@nowhere.com
    JunkMailList=disabled
    QpsmtpdCodes=enabled
    SARulePercentThreshold=0.5
    SARules=enabled
    Status=enabled

[brianr]

Before we get bounced on please open a bugzilla report, and show the results of "config show mailstats". amd put a reference to it here.

thanks

[brianr]

actually Jay I think I can see the problem:

You can either do

db configuration setprop <whatever>

or

config setprop <whatever>

NOT db config <whatever>

B.

[jfarschman]

Your ability to see the obvious far outshines my own.  Thanks.

[brianr]

 

[shell]

Hi Brianr,

Love your mailstats script and use it as one of my "must haves".  Thanks heaps for your time and excellent work.

I have a query on the stats that are produced.  I put the second script in conjunction with the 1st and have the scripts running an hour apart:

this info is uniform on both emails:

Period Beginning : Mon Oct 15 00:00:00 2007
Period Ending    : Tue Oct 16 00:00:00 2007

Reporting Period : 24.00 hrs
----------------------------

Clam Version : ClamAV 0.91.2/4540/Sun Oct 14 14:43:55 2007
SpamAssassin Version : SpamAssassin version 3.2.3
  running on Perl version 5.8.5
Tag level:   3; Reject level:  10 

However the stats appear to be different and am confused now as my totals are out of whack - for example see the summary below for yesterdays stats:

New script:

Code: [Select]

/usr/bin/spamfilter-stats2-7.pl Version : 0.6.15

Reporting Period : 24.00 hrs
----------------------------

All SMTP connections accepted:      17         
Emails per hour              :      0.7/hr

Average spam score (accepted):        0.00
Average spam score (rejected):        0.00
Average ham score            :      -52.05

17 mails were processed for 17 Recipients

Old script:

Code: [Select]

/usr/bin/spamfilter-stats-7.pl Version : 0.5.1

All SMTP connections accepted    :       18         
       
Total emails processed           :       15 (    0.62/hr)

Average spam score (accepted):        0.00
Average spam score (rejected):        0.00
Average ham score            :      -78.08

16 mails were processed for 18 Recipients

I am guessing this is (as usual  !) my lack of understanding as to what is being summarised, or more importantly the differences in the scripts.  not my strong point... can you shed some light?

Cheers, shell

[brianr]

Shell

I can't give you chapter and verse, as I don't have the logs, but I am sure that the "latest" one is the most accurate.  The earlier one used the transaction based log entries, rather than the summarised one (produced by logterse), and one of the reasons I moved it was to get closer to reality!

Thank you for your kind words, much appreciated.