Virus:Email Attachment Blocking
Virus & executable content blocking tutorial for qpsmtpd
This functionality has been incorporated into sme server v7.0 and more recent versions. See the Email panel in server manager.
Problem
Your sme server receives a lot of email with virus infected attachments and you want to reject it before it enters your server's mail system. You want to block email with certain types of file attachments to improve security of your server or reduce bandwidth use caused by unwanted or undesired large multimedia files. Current methods typically use Anti Virus detection software, which are processor & memory intensive.
Solution
This functionality allows incoming & outgoing messages to be rejected if the attached file has executable content, which matches specific file type patterns. A default pattern matching database is created with common executable file patterns, which cover the majority of currently known Windows type executable viruses. Patterns can be created for any file types to allow multimedia or other attachments to be rejected where the system management policy considers it appropriate.
Email messages are rejected if the attachment content matches an entry in the patterns database. By default this includes the majority of *.exe files, older v1.0 *.zip files and some *.gif files. This blocking applies to both incoming and outgoing smtp email messages, including the local network, in order to stop virus propagation. If these file types need to be sent using email, they should be compressed using WinZip (v2.0 format) or WinRAR or other suitable compression software, or alternatively shared on the local network use filesharing. Note that recent releases of compression software use the v2.0 zip format.
Messages with attachments that match the patterns database are rejected by the mail system, and as a result there is no further processing. In practice a large number of virus infected messages will be rejected, perhaps 95 % or more, depending on the type of virus infections you receive and your system exposure (email addresses).
In conjunction with RBL list blocking of spam messages you can expect a reduction in virus detections by Clamavis from typically hundreds per month to one message per month. The use of RBL list spam blocking also helps reduce virus infected email messages entering the server, probably due to the fact that some virus infected messages come from similar sources as spam messages.
This method works for servers configured as either Server & Gateway or Server Only as long as the mail server components are enabled (qpsmtp & qmail) and the server has access to the Internet via another sme server or firewall.
Additional Information
Pattern matching acts as a "gross filter" to reject many known virus types, but a regularly updated virus scanner is still required to catch new viruses. Once these new executable content viruses have been analysed, additional patterns can be created and added to the patterns database as required. It is envisaged that new patterns would be added infrequently.
This pattern matching feature should be used in conjunction with virus scanning software and spam filtering software, although these programs will have a lot less work to do. Pattern blocking should be compatible with other brands of virus & spam software based programs. They generally scan or filter the message after it has been accepted by the servers mail system. Pattern blocking occurs before the message is accepted, and if a matching occurs the message is rejected so it would never be scanned by secondary software based systems. Incompatibilities are therefore unlikely.
An additional feature I recommend to implement is "RBL List blocking" using qpsmtp, to reject spam messages from senders that are included on RBL lists. This technique will dramatically reduce the amount of spam entering the server. Additional patterns can be added to the database after install is completed. Also see separate section below for information on analysing, creating & adding patterns.
WARNING: Enable additional patterns with care. Verify that the patterns do not block attachment types that you wish to receive.
Enabling Pattern Matching
A menu box is in the server manager Email panel, which allows executable content blocking to be enabled or disabled. It is disabled by default. Use "Ctrl click" to highlight or unhighlight the various groups of file types, and then click the Save button to enable/disable pattern matching.
Analyzing and creating patterns
Common file patterns (or signatures or magic)
The standard patterns enabled by default are:
Windows executables seen in active viruses TVqQAAMAA TVpQAAIAA
Additional Windows executable signatures not yet seen in viruses TVpAALQAc TVpyAXkAX TVrmAU4AA TVrhARwAk TVoFAQUAA TVoAAAQAA TVoIARMAA TVouARsAA TVrQAT8AA TVoAAAEAAA
ZIP file signature seen in SoBig.E and mydoom UEsDBAoAA (this pattern is blocked - zip v1.0 format) UEsDBBQAA (this pattern is NOT blocked by default - zip v2.0 format
GIF file found in a previous Microsoft virus R0lGODlhaAA7APcAAP///+rp6puSp6GZrDUjUUc6Zn53mFJMdbGvvVtXh2xre8bF1x8cU4yLprOy
Extra patterns not included in default database that may be enabled if required for blocking of new viruses
A recent pattern identified for the virus
Worm.SomeFool.P
is
TVoAAD8AA
Identified as MS-DOS executable
Extra patterns not included in default database that may be enabled if required for blocking of multimedia files etc (long & short versions listed)
Note that these have not been thoroughly tested and may need further refinement to ensure they accurately represent the signature pattern for all occurrences of the particular file type
SCR screen saver files - MS-DOS executable (EXE)
Example: TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Pattern: TVqQAAMAAA
PIF1 - data
Example: AHhUYXggMTk5OCAgICAgICAgICAgICAgICAgICAgICCAAgAAWTpcSFN
PRlRcSFQ5OFxIVDk4LkVY
Pattern: AHhUYXgg
PIF2 - data Example: AMlIbDk5LmV4ZSAgICAgICAgICAgICAgICAgICAgICCAAIAAVDpccH
JpdmF0ZVxIc29mdFxITFxI Pattern: AMlIbDk5Lm
PIF3 - data
Example: AHhIYW5kaVJlZ2lzdGVyIDIwMDAgICAgICAgICAgICCAAgAAWTpcSHNvZn
RcSFJcSFIwMC5FWEUA
Pattern: AHhIYW5k
WAV sound file - data Example: UklGRiRwLgBXQVZFZm10IBAAAAABAAIAgLsAAADuAgAEABAAZGF0YQB
wLgAAAAAAAAAAAAAAAAAA
Pattern: Uk1GRiRwL
JPEG image data, JFIF standard 0.00, aspect ratio, 0 x 0 Example: /9j/4AAQSkZJRgABAgEBLAEsAAD/7RLSUGhvdG9zaG9wIDMuMAA4QklNA+0
AAAAAABABLAAAAAEA Pattern: /9j/4AAQSkZJRg
TIF - TIFF image data, little-endian
Example: SUkqAAgAAAAQAP4ABAABAAAAAAAAAAABAwABAAAAJgMAAAEBAwABAAAA
QAUAAAIBAwADAAAAzgAA
Pattern: SUkqAAgAAAA
PPT powerpoint presentation -Microsoft Office Document Example: 0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAA
DEAwAAIRgBAAAAAAAA
Pattern: 0M8R4KGxGuEA
WMV Windows Media Player video file - Microsoft ASF
Example: MCaydY5mzxGm2QCqAGLObH8PAAAAAAAACwAAAAECodyrjEepzxGO5
ADADCBTZWgAAAAAAAAAeeIB
Pattern: MCaydY5mzxGm
MPG mpeg1 video file - MPEG system stream data
Example: AAABuiEAAQAHgCgdAAABuwAMgCgdBeH/4OAuwMAgAAAB4AfcYC4xAA
GMUREAAXAxAAABsxYBIIME Pattern: AAABuiEAAQAHg
M2P mpeg2 video file - MPEG system stream data
Example: AAABukQABAAGBQFG//gAAAG7AAyAo38F4X/g4OfAwCAAAAHgB9qAw
Q0xAAG2QxEAAZojHmDnAAAB Pattern: AAABukQABAAGB
AVI video file - RIFF (little-endian) data
Example: UklGRpC0qQBBVkkgTElTVDYBAABoZHJsYXZpaDgAAABAnAAA5MJn
AAAAAAAQAAEAWggAAAAAAAAC Pattern: UklGRpC0qQBB
Determining file pattern signature or magic
To find out what the pattern or signature or magic for a file is, we need to run it through a base64 encoding routine and look for the appropriate strings in the first line of the output. That is, for "sane" files which have "magic" numbers at the start. We also can decode the file to find out what type of file it is. Published file specifications (where available) could also be referred to.
Copy a file to a folder on your sme server, say filename.zip
At the command prompt do
perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' <filename.zip | head -1
we get an output of
UEsDBAoAAQAAAMBOfzC356fxzVUAAMFVAAANAAAAZWxhaXZrZHVwLnNjckwHjDHTKYSGUE+SV
We need to pick a suitable substring to use as the pattern for this file type, for example:
UEsDBAoAA
We want the pattern string to be long enough to avoid "false positives" and short enough to catch all of that file type. Running the above command across a few files of a particular type will usually clearly show the appropriate substring.
To find out the file type details do
echo 'UEsDBAoAA' | perl -MMIME::Base64 -0777 -ne 'print decode_base64($_)' >/tmp/17.exe
then run "file" on the result
file /tmp/17.exe
the output is
/tmp/17.exe: Zip archive data, at least v1.0 to extract
which identifies the type of file
An alternative way of identifying the file pattern or signature for users of Clamavis-ng is to view the quarantined messages in /var/spool/amavis-ng/quarantine
Here is an extract from a quarantined infected message that mimics a zip file
File: 406a8bee~aad.msg Col 0 30787 bytes
mtohkeqkmfnipbfntepj
Content-Type: application/octet-stream; name="AttachedFile.zip" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="AttachedFile.zip"
UEsDBAoAAQAAAMBOfzC356fxzVUAAMFVAAANAAAAZWxhaXZrZHVwLnNjckwHjDHTKYSGUE+SV1OwspplLsSWrbYvwOvHVHYOYDOiVliyLlDWU2LYVELdEiwxkwOPVsk3+m/Ddl9U56v6+tbrdXPEBTv+yEH56h/R+Bbk54hUOLieVPW61QOD7YVXZilxgCAZ+SppPxWuKv2iCBuw5qQ5N/r7CISrWWEPaAzGYwUmuERoNMEo4TFm6yV2BqBhv+Y1e/SLz30EV6anGmvwvKiWaLfcjo8sfF3UDQ203TAV33kypvZDqAsF/g3O1rvbEf+K/pZpWjOy1A5S3OWF7IKsbNxQdwqWPvuO6XS6QHwLQAF+6q4LKdUFM89j+lnKR3bXaGU3v18YN862XIeJtEqW3Ulbj8MA33IBDoTQzpYQwGQm+?????????..
So to create a new pattern for this message we would use
UEsDBAoAA
which is the pattern corresponding to ZIPV1 file type
UEsDBAoAA: Zip archive data, at least v1.0 to extract
Enabling or disabling patterns
Let's say we want to add a pattern to the existing EXEFILES type (which you should do if you discover new patterns for common new MSDOS Executable type viruses)
A pattern analysed from email messages received is
TVoAAD8AA
As we wish to add this to the existing db entry we would do:
/sbin/e-smith/db mailpatterns set EXEFILES pattern Body TVqQAAMAA,TVpQAAIAA,
TVpAALQAc,TVpyAXkAX,TVrmAU4AA,TVrhARwAk,TVoFAQUAA,TVoAAAQAA,TVoIARMAA,
TVouARsAA,TVrQAT8AA,TVoAAAEAAA,TVoAAD8AA Description "MS-DOS executables"
Glob yes LineStart yes Status enabled
(the above is all on one line, watch the spaces are correct)
signal-event email-update
To check the entry is correct do:
db mailpatterns show EXEFILES
which gives an output of
EXEFILES=pattern
Body=TVqQAAMAA,TVpQAAIAA,TVpAALQAc,TVpyAXkAX,TVrmAU4AA,TVrhARwAk,TVoFAQUAA,TVoAAAQAA,TVoIARMAA,TVouARsAA,TVrQAT8AA,TVoAAAEAAA,TVoAAD8AA
Description=MS-DOS executables
Glob=yes
LineStart=yes
Status=enabled
Let's say we want to enable the pattern for PIF2 type files (which you should do if you wish to block some PIF attachments)
A pattern being tested for this file type is
AMlIbDk5Lm
So we do:
db mailpatterns set PIF2 pattern Body AMlIbDk5Lm Description "PIF2 data" Glob yes LineStart yes Status enabled
(the above is all on one line)
signal-event email-update
To check the entry is correct do:
db mailpatterns show PIF2
which gives an output of
PIF2=pattern
Body=AMlIbDk5Lm
Description=PIF2 data
Glob=yes
LineStart=yes
Status=enabled
The fields are as follows:
pattern - the type of the entry in the database (currently only the "pattern" type is used)
Body - the substring to match
Description - free format text to describe this pattern. This text will be used to display a menu
of patterns to enable/disable in a later version
Glob - whether to apply a wildcard match after the pattern
LineStart - whether to only match this pattern at the start of the line
Status - whether this pattern is currently enabled (i.e. blocked)
To disable the pattern do:
db mailpatterns setprop PIF2 Status disabled
signal-event email-update
The alternative but more correct approach is as follows:
The initialize-default-databases loads the db with fragments from /etc/e-smith/db. When new patterns are added to the master rpm, new fragments are also added.
To add a pattern to the default set in the rpm, we do:
mkdir -p /etc/e-smith/db/mailpatterns/defaults/PIF2/
and in that directory, create the following files/contents:
type/pattern Body/AMlIbDk5Lm Description/PIF2 data Glob/yes Status/enabled
then do
/etc/e-smith/events/actions/initialize-default-databases
which will load the default settings
To show all the patterns in the mailpatterns database & their status (enabled or disabled) do
db mailpatterns show
which will give an output similar to
(Note the last entry for ZIPV2 is disabled)
db mailpatterns show
EXEFILES=pattern
Body=TVqQAAMAA,TVpQAAIAA,TVpAALQAc,TVpyAXkAX,TVrmAU4AA,TVrhARwAk,TVoFAQUAA,TVoAAAQAA,TVoIARMAA,TVouARsAA,TVrQAT8AA,TVoAAAEAAA
Description=MS-DOS executables
Glob=yes
LineStart=yes
Status=enabled
GIF01=pattern
Body=R0lGODlhaAA7APcAAP///+rp6puSp6GZrDUjUUc6Zn53mFJMdbGvvVtXh2xre8bF1x8cU4yLprOy
Description=GIF file from old virus
Glob=yes
LineStart=yes
Status=enabled
ZIPV1=pattern
Body=UEsDBAoAA
Description=Zip archive data, at least v1.0 to extract
Glob=yes
LineStart=yes
Status=enabled
ZIPV2=pattern
Body=UEsDBBQAA
Description=Zip archive data, at least v2.0 to extract
Glob=yes
LineStart=yes
Status=disabled
Checking logs for effectiveness of blocking messages with executable content in the attachments
By reviewing /var/log/qpsmtpd/current and var/log/qpsmtpd/* you can see the entries for rejected messages and generally enough information as to why the rejection occurred, and therefore see the effectiveness of Pattern Matching blocking.
Note that you will only see these types of entries after blocking has been enabled and messages have been rejected.
If you do not see all of the types of entries shown below, it would either be due to not having the particular Pattern enabled or not receiving attachments with that type of content.
You can view date formatted logs using the Server Manager View log files panel
To see ALL the log entries do
grep "" /var/log/qpsmtpd/current | tai64nlocal
To see only the rejected message entries and the reason for rejection do
grep "We don't accept email with executable content" /var/log/qpsmtpd/current | tai64nlocal
(the above is all on one line)
Here is an example of some typical entries
Note: you will only see these entries after some messages have been rejected
2004-04-15 12:32:11.892522500 qpsmtp[23392]: 554 We don't accept email with executable content ZIPV1 (#5.3.4)
2004-04-15 15:23:40.765202500 qpsmtp[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)
2004-04-15 15:33:08.132041500 qpsmtp[29241]: 554 We don't accept email with executable content EXE12 (#5.3.4)
2004-04-15 15:33:09.021650500 qpsmtp[29265]: 554 We don't accept email with executable content PIF (#5.3.4)
Alternatively you could filter on the pattern type code to see how many messages with a particular type of executable content are being rejected eg
grep EXE01 /var/log/qpsmtpd/current | tai64local
2004-04-15 15:23:40.765202500 qpsmtp[28963]: 554 We don't accept email with executable content EXE01 (#5.3.4)
2004-04-15 15:33:08.132041500 qpsmtp[29241]: 554 We don't accept email with executable content EXE01 (#5.3.4)
2004-04-15 15:33:09.021650500 qpsmtp[29265]: 554 We don't accept email with executable content EXE01 (#5.3.4)
2004-04-15 15:33:24.986426500 qpsmtp[29274]: 554 We don't accept email with executable content EXE01 (#5.3.4)
Web sites for background information
These links may be of interest. Note that they do not specifically apply to sme server, so DO NOT implement them. They are listed for background information only.
http://qmail.planetmirror.com/top.html
http://qmail.planetmirror.com/top.html#microsoft
http://qmail.planetmirror.com/qmail-smtpd-viruscan-1.3.patch
Prior version of this Howto for sme6.x
Here is a link to an earlier Howto written for sme6.x. Please disregard all references to installing rpms as this does not apply to sme 7.x. There are some small changes in database arrangement between the older sme6.x db and the sme7.x db. Note also that there are many additional patterns in sme7.x. Note also that sme7.x uses qpsmtpd instead of smtpfront-qmail.
You will find it here old HowTo.