This program was written to help me analyze how i might want to handle bad email by determining the country origin using the geoip lookup software that is easily installed following a wiki page from contribs.org.
This program help me see where email might be coming from. It will not be perfect for a variety of reason but it is a tool just the same.
Also this program can be use to test if email might be coming into the server from an unwanted country.
I am sure it can use it for other purposes too.
There is also a function that i added to see where emails marked with a certain spam level using astericks placed the email headers, presumably by spammassain, may be coming from.
In the future, i will not be allowing any email from countries outside of the usa to accounts on our servers as a general rule. But i will allow certain mail to the maillog account.
This program was more than i needed by i tried to make it for other users. My learning curve of bash and regular expressions has been slow down.
This script does not delete emails as written.
From what I have seen with some other addon programs for sme server, you have to exclude countries. That could be ok for somebody wanting that. But I need to filter in and not out emails within countries I want.
This script makes use of the GeoIp program found here
http://wiki.contribs.org/GeoIPYou can find most of the countries listed there also.
The program has to be loaded on your sever for this script to run as is.
The script was written with as few functions as possible to keep it fast but also trying to keep it somewhat modular for changing.
There are some areas where i have commented out lines while testing. I have limited time to do this kind of stuff and it earns me zero dollars except for trouble the script may save me in down time in the future.
I will share another script in about a month that will be derived from this and it will be much leaner to process received emails full time as they arrive.
This script was set up to view emails headers that are received from outside of the United States. Any outside ip address for the countries wanted in the email header flags the email I the script as rejected.
I had a thought about placing countries into the email header of all received emails. That might prove useful somewhere down the road. But I am learning to crawl before I run.
#!/bin/bash
### This program was written to help evaluate received emails to determine
### if emails need to be blocked by ip addresses in the email headers or
### to see if emails are being received for countries not wanted.
### There are other uses for this program one might want to use it for.
### This program makes use of a lot of functions to make it somewhat easier
### to alter to get other results
### This program does not alter emails as written and the users takes full use for its actions.
### This script will search emails that have emails where a ip address does not
### come from a safe list placed into a string.
### There is another option to search only on the spam asterick marks
### In the "X-Spam-Level:" email header.
### Those are compared to how may astericks are in the field.
### You set how many minuium astericks to have in the fiel.
### Any number or higer number of astericks will display the email
### spamastericks must be set to 0(zero) for script to work on ip addresses alon
###---------------------------------------------------------------------start of variables
### All countries need a space before and after the country and
### a comma immediately after the country abbreviation.
### Between each country needs to be a backslash and a bar together
safelistofcountries=" US, \| UM, \| GOV, \| MIL, "
### edit the wanipaddressofthisemailserver
### place you ip address in so that it will not be searched
### this is not necessary but may prove helpful.
### the program will not report on the ipaddresses in this string
### place a minus sign n front of each ip 4 address with no spaces in the string
### an example is "-78.152.2.1-645.1.284.291"
wanipaddressofthisemailserver=""
### Set nosafecountryduplicate to 1 to not record a second ip lookup of an address
### that was already found in a previous email that was geoip looked up if that ip
### address fell into one of the safe countries list.
### All addresses outside of the safe list will be shown and reocrded in the log files
### The setting of 0(zero) will record all ip addresses. This feature was designed to
### speed up the program when set to 1, but you could make good use of the log
### when you record all ip addresses.
### Any ip address will be listed only once for each email header scanned.
nosafecountryduplicate=1
### set readmaillogemails to 0 to exclude the emails to the maillog account
### set readmaillogemails to 1 to exclude the emails that are not in the maillog account
### set readmaillogemails to 2 to read all emails, both maillog and non maillog accounts
### if you are reading a directory full of email flles not under the MailDir directory set this value to 2
#readmaillogemails=0 # exclude emails under maillog account
#readmaillogemails=1 # read only the emails under the maillog account
#readmaillogemails=2 # read all emails
readmaillogemails=2 # read all emails
### set spamastericks to the number of how many spam astericks need to to be found in the spam header line
spamastericks=0
### assign the program to lookup geo ip address based on countries
geoiplookupcommand="/usr/bin/geoiplookup"
###---------------------------------------------------------------------end of variables
###---------------------------------------------------------------- start of functions
function whereisipfrom {
local stemp=""
iplocationaccept=0
local tempcount=0
if [ $nosafecountryduplicate -eq 1 ];then
tempcount=$(echo $sipacceptlist | grep -c "\-$1\-")
if [ $tempcount -gt 0 ]
then
iplocationaccept=1
return
fi
fi
stemp=$($geoiplookupcommand "$1" | sed -e 's/.*://')
# stemp=$(echo $stemp | tr '[:upper:]' '[:lower:]')
sgroupedcountries="$sgroupedcountries$1 $stemp\n"
echo -e "$1\t$stemp" >> /tmp/geoemailall
tempcount=$(echo $stemp | grep -i -c "$safelistofcountries")
if [ $tempcount -gt 0 ]
then
iplocationaccept=1
sipacceptlist="$sipacceptlist$1-"
echo -e "$1\t$stemp" >> /tmp/geoemailaccepted
return
fi
if [ $iplocationrejected -eq 0 ];then clear;fi
tempcount=$(echo $stemp | grep -i -c " not found")
if [ $tempcount -gt 0 ];then iplocationaccept=-1;fi
echo "$1 $stemp"
echo -e "$1\t$stemp" >> /tmp/geoemailrejected
if [ "$iplocationaccept" -eq 0 ] || [ "$iplocationaccept" -eq -1 ]
then
iplocationrejected=2
ipthisreceivedlinerejected=1
fi
return
}
function checkipaddresses {
local tempcount=0
local stemp=""
local kk=""
ipthisreceivedlinerejected=0
set $1
for item
do
tempcount=$(echo "-$item" | grep -c "\-127.0.0.1\|\-192.168.\|\-10.\|\-169.254.")
if [ "$tempcount" -gt 0 ];then continue;fi
tempcount=$(echo "-$item" | grep -c "\-172.")
if [ "$tempcount" -gt 0 ];then
tempcount=$(echo "-$item" | grep -c "\-172.16.\|\-172.17.\|\-172.18.\|\-172.19.\|\-172.20.\|\-172.21.\|\-172.22.\|\-172.23.\|\-172.24.\|\-172.25.\|\-172.26.\|\-172.27.\|\-172.28.\|\-172.29.\|\-172.30.\|\-172.31.")
if [ "$tempcount" -gt 0 ];then continue;fi
fi
tempcount=$(echo $ipaddresseschecked | grep -c "\-$item\-")
if [ "$tempcount" -gt 0 ];then continue;fi
ipaddresseschecked="$ipaddresseschecked$item-"
whereisipfrom "$item"
done
return
}
function locateipaddresses {
local stemp=""
local ipaddresses=""
iplocationrejected=0
ipaddresseschecked="-127.0.0.1-$wanipaddressofthisemailserver-"
for item in $semailheader
do
stemp=$(echo "$item" | grep -i "^Received:" )
if [ ! -z "$stemp" ]
then
ipaddresses=$(echo "$stemp" | grep -E -o '((25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]|[0-9])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]|[0-9])')
#echo $ipaddresses
if [ ! -z "$ipaddresses" ]
then
checkipaddresses "$ipaddresses"
#the next line is testing, where you can view the recevied line in the email header where the rejected ip address is found
if [ $ipthisreceivedlinerejected -gt 0 ];then echo "$stemp";echo " ";fi
fi
fi
done
return
}
function isemailspamflagged {
local tempcount=0
local stemp=""
flaggedwithspam=0
for item in $semailheader
do
stemp=$(echo "$item" | grep -i "^X-Spam-Level:" )
if [ ! -z "$stemp" ];then
tempcount=$(echo "$stemp" | grep -c "$sspamstring" )
if [ $tempcount -gt 0 ];then flaggedwithspam=1;fi
fi
done
return
}
function echoemailheaderline {
for item in $semailheader
do
echo "$item" | grep -i "$1"
done
return
}
function getemailheaderfromemailfile {
semailheader=$(grep -B10000 -m1 ^$ $filename | tr -s '\t' '\040'|sed -e ':a;N;$!ba;s/\n / /g')
return
}
###----------------------------------------------------------------------end of functions
### -----------------------------------------------------------------------------------------------
### main program starts hear
###
if [ ! -f "$geoiplookupcommand" ]
then
echo "the program below does not exist and is needed"
echo "$geoiplookupcommand"
fi
rm -f /tmp/geoemailall
rm -f /tmp/geoemailaccepted
rm -f /tmp/geoemailrejected
echo "all email geoipaddress " > /tmp/geoemailall
echo "" >> /tmp/geoemailall
echo "all email geoipaddress accepted" > /tmp/geoemailaccepted
echo "" >> /tmp/geoemailaccepted
echo "all email geoipaddress rejected " > /tmp/geoemailrejected
echo "" > /tmp/geoemailrejected
counter=0
while [ $counter -lt $spamastericks ]
do sspamstring="$sspamstring\*"
let counter=counter+1
done
clear
echo "working ! readng emails ......................"
echo ""
echo ""
countfile=0
sipacceptlist="-"
IFS=$(echo -en "\n\b")
for filename in $(find /home/e-smith/files/users/*/Maildir/ -name "1*.*.$HOSTNAME*" -type f )
do
if [ "$readmaillogemails" -eq 0 ];then
if [[ $filename == */maillog/Maildir/* ]];then continue; fi
fi
if [ "$readmaillogemails" -eq 1 ];then
if [[ $filename != */maillog/Maildir/* ]];then continue; fi
fi
let countfile+=1
#echo $countfile
#echo $filename
getemailheaderfromemailfile "$filename"
#echo -e "$semailheader"
sgroupedcountries=""
iplocationrejected=0
locateipaddresses
flaggedwithspam=0
if [ $spamastericks -gt 0 ]
then
iplocationrejected=0
isemailspamflagged
fi
showemail=0
if [ $flaggedwithspam -gt 0 ];then showemail=1; fi
if [ "$iplocationrejected" -gt 0 ] && [ "$spamastericks" -eq 0 ];then showemail=1;fi
if [ $showemail -gt 0 ]
then
echoemailheaderline ":"
echo ""
echo "-------------------------------------------"
echo -e "$sgroupedcountries"
echoemailheaderline "^Date: "
echoemailheaderline "^TO: "
echoemailheaderline "^From: "
echoemailheaderline "^Return"
echoemailheaderline "^Subject: "
echoemailheaderline "^X-"
echoemailheaderline "^RECEIVE"
echo "filename $filename"
echo "--------------------------------------------------------------------------------------------------"
read -p "Press [Enter] key to continue ......."
echo "working ! readng emails ......................"
echo ""
echo ""
fi
done
echo ""
echo ""
echo "total emails read $countfile"
echo ""
echo "next! view 3 files with only geoiplookup returned iinformation"
echo "view geoip counties in the file /tmp/geoemailall"
echo "view geoip counties in the file /tmp/geoemailaccepted"
echo "view geoip counties in the file /tmp/geoemailrejected"
echo ""
echo " To quit viewing each list, press the q key to quit"
echo " You will have to delete the 3 files above manually"
echo " when are complete and done using this program."
echo " They are left undeleted so you can use them in other ways."
read -p "Press [Enter] key to continue ......."
cat /tmp/geoemailall | less
cat /tmp/geoemailaccepted | less
cat /tmp/geoemailrejected | less
exit 0