687
edits
Changes
Jump to navigation
Jump to search
Line 174:
Line 174: − + Line 190:
Line 190: − + − + + Line 205:
Line 206: + + + + + + + + + + + + + + + + + + + + + + + + + − + + + + + + + Line 234:
Line 266: − + − + − + − + − + − + − + + + + + + + + + Line 261:
Line 301: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
→Display messages that would have been blocked via DNSBL: fix typo
print count[j] "\t" j; \
print count[j] "\t" j; \
}' \
}' \
$(find /var/log/qpsmtpd /var/log/sqpsmtpd -ctime -$DAYS -type f)</nowiki>
$(find /var/log/qpsmtpd /var/log/sqpsmtpd -ctime -$DAYS -type f -name "@*" -o -name current)</nowiki>
Sample Output:
Sample Output:
echo -n "Days of logfiles to scan [$DAYS]: "; read NEWDAYS; \
echo -n "Days of logfiles to scan [$DAYS]: "; read NEWDAYS; \
if [ $NEWDAYS ]; then DAYS=$NEWDAYS; fi; \
if [ $NEWDAYS ]; then DAYS=$NEWDAYS; fi; \
awk -F"[\t]" ' /logterse plugin/ { svc=$6; count[svc]++; count["Total"]++; } END { for (j in count) print count[j] "\t" j; }' \
awk -F"[\t]" ' /logterse/ { svc=$6; count[svc]++; count["Total"]++; } END \
$(find /var/log/qpsmtpd /var/log/sqpsmtpd -ctime -$DAYS -type f) \
{ for (j in count) print count[j] "\t" j; }' \
$(find /var/log/qpsmtpd /var/log/sqpsmtpd -ctime -$DAYS -type f -name "@*" -o -name current) \
|sort -nr</nowiki>
|sort -nr</nowiki>
6 check_spamhelo
6 check_spamhelo
6 auth::auth_cvm_unix_local</nowiki>
6 auth::auth_cvm_unix_local</nowiki>
I extended the above script to show % of each one:
<nowiki>
if [ -z $DAYS ]; then DAYS=1; fi; \
echo -n "Days of logfiles to scan [$DAYS]: "; read NEWDAYS; \
if [ $NEWDAYS ]; then DAYS=$NEWDAYS; fi; \
awk -F"[\t]" ' /logterse/ { svc=$6; count[svc]++; count["Total"]++; } END \
{ for (j in count) print count[j] "\t" j "\t" expr count[j]/count["Total"]*100"%" ; }' \
$(find /var/log/qpsmtpd /var/log/sqpsmtpd -ctime -$DAYS -type f) \
|sort -n
</nowiki>
Sample output of extended version:
<nowiki>
1 headers 0.00769468%
2 auth::auth_cvm_unix_local 0.0153894%
33 earlytalker 0.253924%
355 spamassassin 2.73161%
401 naughty 3.08556%
698 tls 5.37088%
774 rhsbl 5.95568%
1127 check_goodrcptto 8.6719%
1359 queued 10.4571%
8246 resolvable_fromhost 63.4503%
12996 Total 100%
</nowiki>
===Display messages that would have been blocked via DNSBL===
===Display messages that would have been blocked via DNSBL===
From time to time I try out new DNSBL services. Some of these generate instant complaints from my users about correspondents who can no longer send us email.
This command has two objectives -
# Testing a new dnsbl service<br><nowiki>
Show you what emails would have been blocked by a new dnsbl service.</nowiki><br><nowiki>
From time to time I try out new DNSBL services. Some of these generate instant complaints from my users about correspondents who can no longer send us email.</nowiki>
# Review queued messages from servers that are now listed<br><nowiki>
Reviewing recently received emails that were queued by the mail server from hosts that are *now* listed on a dnsbl can be used to look for patterns that might help you tune your spam filter settings.</nowiki>
The command below will:
The command below will:
if [ $NEWDAYS ]; then DAYS=$NEWDAYS; fi; \
if [ $NEWDAYS ]; then DAYS=$NEWDAYS; fi; \
if [ $NEWTESTBL ]; then TESTBL=$NEWTESTBL; fi; \
if [ $NEWTESTBL ]; then TESTBL=$NEWTESTBL; fi; \
grep -h logging::logterse $(find /var/log/qpsmtpd -ctime -$DAYS) \
grep -h logging::logterse $(find /var/log/qpsmtpd /var/log/sqpsmtpd -ctime -$DAYS -name "@*" -o -name current) \
|grep -v dnsbl.903 \
|grep -v dnsbl.903 \
|tai64nlocal \
|tai64nlocal \
|awk -v DNSBL=$TESTBL -F"\t" '{split($1,intro," "); split(intro[8],ip,"."); split($9,hits," "); split( intro[2],time,"."); \
|awk -v DNSBL=$TESTBL -F"\t" '{split($1,intro," "); \
print "echo -ne \"" intro[1] " " time[1] "\t" $6 "\t" (hits[2]) \
intro[6] == "`" ? split(intro[7],ip,".") : split(intro[8],ip,"."); split($9,hits," "); \
"\tFrom: " gensub("[<>]","","g",$4) \
split( intro[2],time,"."); \
"\tTo: " gensub("[<>]","","g",$5) \
print \
"\tA: `dig +short " ip[4] "." ip[3] "." ip[2] "." ip[1] "." DNSBL \
"echo -ne \"" intro[1] " " time[1] \
"`\tTXT: \" ; echo -e \"`dig +short txt " ip[4] "." ip[3] "." ip[2] "." ip[1] "." DNSBL "`\""}' |bash | grep 127\.0</nowiki>
"\t" $6 \
"\t" ip[1] "." ip[2] "." ip[3] "." ip[4] \
"\t" (hits[2]) \
"\tFrom: " gensub("[<>]","","g",$4) \
"\tTo: " gensub("[<>]","","g",$5) \
"\tA: `dig +short " ip[4] "." ip[3] "." ip[2] "." ip[1] "." DNSBL " |tr \"\n\" \",\" |sed \"s/,$//\" `"\
"\tTXT: \" ; echo -e \"`dig +short txt " ip[4] "." ip[3] "." ip[2] "." ip[1] "." DNSBL "`\""}'\
|bash\
|grep 127\.0</nowiki>
===List Recent Emails with sending IP===
===List Recent Emails with sending IP===
<nowiki>echo -n "TLD to review: "; read TLD; qploggrep $TLD\> |tai64nlocal |awk '{print $1 " " $2 "\t" $4 "\t" $5 "\t" $6 "\t" $7}'</nowiki>
<nowiki>echo -n "TLD to review: "; read TLD; qploggrep $TLD\> |tai64nlocal |awk '{print $1 " " $2 "\t" $4 "\t" $5 "\t" $6 "\t" $7}'</nowiki>
===Count emails by TLD and disposition for today and yesterday===
This command will scan all qpsmtpd log files closed in the last day, pull out entries dated today or yesterday, then count the dispositions applied to each message by TLD (".com", ".org", etc):
Place the entire command below in your clipboard then paste it into command shell on your server. Adding "|sendmail -t emailuser@yourserver.tld" will email the report to the selected email address.
<nowiki>export LC_ALL=C; \
mydate=$(date "+%Y-%m-%d")\|$(date -d "yesterday" "+%Y-%m-%d"); \
cat -v $(find /var/log/qpsmtpd /var/log/sqpsmtpd/ -ctime -1 -type f -name "@*" -o -name current) \
|tai64nlocal |egrep $mydate | grep -v ^# | \
awk -v date="$mydate" -v tots=" {{Total}} " -F"[\t]" ' \
/logterse/ {split($4,ss,"."); ssn=0; for (i in ss) { ssn++}; \
sendtld=tolower( ss[ssn]); sub(">","",sendtld); \
tld=sprintf("%-20s",sendtld); plugin=sprintf("%-35s",$6); \
plugint=sprintf("%35s%-20s",$6" ","{Total}");\
countem=plugin tld; count[countem]++; count[plugint]++; count[tots]++; } \
END \
{ORS=""; print "Subject: Email Disposition on " date "\n\n\
Denying plugin or \"queued\" TLD Count Pct\n\
================================= ==================== ======= =====\n"; \
for (j in count) { pct=sprintf("%2.1f",(count[j]/count[tots])*100); \
j ~ /Total/ ? myORS= " (" pct "%)\n": myORS="\n"; \
printf "%s%9s%s",j,count[j],myORS |"sort -b" } }'
</nowiki>
Sample output:
<nowiki>Subject: Email Disposition on 2015-11-27|2015-11-26
Denying plugin or "queued" TLD Count Pct
================================= ==================== ======= =====
check_badmailfrom_patterns com 23
check_badmailfrom_patterns download 1
check_badmailfrom_patterns info 1
check_badmailfrom_patterns net 2
check_badmailfrom_patterns top 120
check_badmailfrom_patterns xyz 2
check_badmailfrom_patterns {Total} 149 (8.4%)
check_earlytalker 5
check_earlytalker {Total} 5 (0.3%)
check_goodrcptto com 10
check_goodrcptto email 1
check_goodrcptto {Total} 11 (0.6%)
check_spamhelo 3
check_spamhelo {Total} 3 (0.2%)
dnsbl < 5
dnsbl com 104
dnsbl in 2
dnsbl jp 1
dnsbl net 2
dnsbl top 76
dnsbl za 1
dnsbl {Total} 191 (10.8%)
queued com 183
queued net 11
queued org 2
queued za 2
queued {Total} 198 (11.2%)
rhsbl bid 16
rhsbl biz 10
rhsbl cc 2
rhsbl com 902
rhsbl date 14
rhsbl download 25
rhsbl in 1
rhsbl info 1
rhsbl net 10
rhsbl org 3
rhsbl racing 12
rhsbl top 198
rhsbl win 1
rhsbl xyz 12
rhsbl {Total} 1207 (68.4%)
{{Total}} 1764 (100.0%)
</nowiki>
----
----
[[Category:Howto]]
[[Category:Howto]]
[[Category:Administration:Monitoring]]
[[Category:Administration:Monitoring]]
[[Category:Mail]]
[[Category:Mail]]