3,228
edits
Changes
From SME Server
Jump to navigationJump to searchno edit summary
Back to [[:Dansguardian]] wiki page
Back to [[:Dansguardian]] wiki page
====Blacklists====
The general procedure is to locate a suitable blacklist on the Internet, download the tgz file, uncompress it and move it to the /etc/dansguardian/blacklists directory. The SME server admin user would need to configure a cron job to regularly run the download & update script (see below).
There is a commercial blacklist available from URLBlacklist.com (commercial at cost list but free for first download)
There is a free blacklist available from http://www.shallalist.de/ (free for private or personal & non commercial users, registration is required for commercial use, although still free).
See full registration details here http://www.shallalist.de/licence.html Scripts for automating the shallalist download process are here http://www.shallalist.de/helpers.html
A current (at March 2012) blacklists.tar.gz is available from http://cri.univ-tlse1.fr/blacklists/download/blacklists.tar.gz
Here is a script to download and configure this list
If you wish to make dansguardian use squidguard blocking rules & have them updated weekly then add the following to the /etc/cron.weekly/dansguardian file
Please check the location of the blacklists is still current, if necessary search Google on "squidGuard blacklists" or "blacklists.tar.gz" to find a current location.
Create cron job
cd /etc/cron.weekly
pico -w dansguardian
Add the following lines
# blacklists update script for dansguardian
cd /etc/dansguardian
rm -f blacklists.tar.gz
wget -qnv http://cri.univ-tlse1.fr/blacklists/download/blacklists.tar.gz -O blacklists.tar.gz
tar -zxf blacklists.tar.gz
chown -R root.root blacklists
chmod -R 640 blacklists
find blacklists -name new\* -exec rm {} \;
rm -f blacklists/README
chmod ug+x blacklists
chmod ug+x blacklists/*
Then to save & exit
Ctrl o
Ctrl x
Change permissions on cron job & restart crond
chmod +x dansguardian
service crond restart
The scripts runs OK (manually instigated), but integration with Dansguardian not fully tested or documented (at 28 March 2012).
Also refer to this Forum post http://forums.contribs.org/index.php?topic=48449.new;topicseen which refers to list sites and an older blacklists update script from an earlier DG Howto.
Previously blacklists were available from mesd.k12.or.us but this site appears non functional at 28 March 2012
Previously there was a blacklist available from dungog.net which was originally installed with packages from dungog.net in an earlier Howto, but this appears no longer accessible.
=====Troubleshooting Blacklists=====
'''Why are sites not being blacklisted?'''
Keep in mind when using blacklists the interrelations between all blacklists, banned lists, exception lists, gray lists etc.
If sites are not being blacklisted even though the site is in a blacklist file, make sure you have added the path of the blacklist file or uncommented (removed the #) from the path of the blacklisted file from /etc/dansguardian/lists/bannedsitelist or bannedurllist. You must tell DansGuardian which blacklists to use!
'''Allow some blacklisted sites / Add my own blacklisted sites'''
If you have a site you want to allow or others you do not want to allow, read up for example on the exceptions lists. They override the banned lists. If you regularly update the blacklists with the method above, then any personal changes made in the blacklist folder /etc/dansguardian/lists/blacklists/* will be lost on the next update. Instead make changes to the various exception, banned, and gray list files in /etc/dansguardian/lists. Changes there will say put.
'''Send yourself email earning message'''
The above script is straight forward and clean and works well. It is suggested to try it first if you are setting up a cron job to regularly download new blacklists.
The below script could be a replacement for those who wish a little more functionality. It adds a log file to the wget command. This might be helpful if for some reason the new blacklist file is not being downloaded. Examining the log file might be useful in troubleshooting. This script also emails you if there is an error in downloading. Obviously this is nice to automatically inform you of possible problems.
First test the email portion to make sure it works. With all the spam filters etc out there it may or may not be caught by them. Put the following into a command prompt, (add your email.) This will confirm that the main part of the script works. You should receive an email from your server.
echo "See /var/log/blacklists_dl.log on" `uname -n` |/bin/mail -s'Blacklist DL Error' email@somewhere.com
If you choose to use or trial the lists from blacklist .com, download the tgz file, uncompress and move to the
If the above email test works it confirms that your email can be send. Now here is the script. Follow the instructions above for setting up the blacklists accept use this script instead.
/etc/dansguardian/blacklists directory. There is also a blacklist from dungog.net that was installed at the beginning of this HOWTO.
# blacklists update script for dansguardian
# Creates wget log and emails if error downloading.
cd /etc/dansguardian
rm -f blacklists.tar.gz
#If the following site stops allowing downloads you will need to find another
wget -v http://cri.univ-tlse1.fr/blacklists/download/blacklists.tar.gz -O blacklists.tar.gz -o \
/var/log/blacklists_dl.log
#If an error occurs during download then an email will be sent via the mail program and then will exit.
[ $? -gt 0 ] && echo "See /var/log/blacklists_dl.log on" \
`uname -n` |/bin/mail -s'Blacklist DL Error' email@somewhere.com && exit 1
tar -zxf blacklists.tar.gz
chown -R root.root blacklists
chmod -R 640 blacklists
find blacklists -name new\* -exec rm {} \;
rm -f blacklists/README
chmod ug+x blacklists
chmod ug+x blacklists/*
====dansguardian.conf & dansguardianf1.conf====
====dansguardian.conf & dansguardianf1.conf====
=====Show weighted phrases found=====
=====Show weighted phrases found=====
If enabled then the phrases found that made up the total which exceeds the naughtyness limit will be logged and, if the reporting level is high enough, reported.
If enabled then the phrases found that made up the total which exceeds the naughtyness limit will be logged and, if the reporting level is high enough, reported.
The logged message will look like this.
DENIED* Weighted phrase limit of 50 : 60 ((pink, lips)+(proxy, block)+(proxy, filter)+-main+-transparent+-tumor)\
GET 115503 60 Proxies, Pornography 1 403 text/css -
The 50 : 60 is the weight. The first number inicates your default allowable weight or naughtyness limit. The second number represents the weight for the site that the user went to. In this case the site is blocked because the second number representing the site is greater than the allowed limit. The weight based on the reasons given on the rest of the line.
=====Reverse Lookups for Banned Sites and URLs=====
=====Reverse Lookups for Banned Sites and URLs=====
=====exceptioniplist=====
=====exceptioniplist=====
This contains a list of client IPs who you want to bypass the filtering. For example, the network administrator's computer's IP.
This contains a list of client IPs who you want to bypass the filtering. For example, the network administrator's computer's IP.
=====exceptionmimetypelist=====
MIME stands for Multi-purpose Internet Mail Extensions. MIME types form a standard way of classifying file types on the Internet. Internet programs such as Web servers and browsers all have a list of MIME types, so that they can transfer files of the same type in the same way, no matter what operating system they are working in.
If a site does not display properly with Dansguardian it is possible that the mime type is not being allowed.
Look at the log file /var/log/dansguardian/access.log and view the message regarding the web site you are viewing. If it is a mime type that is being blocked you will see something like the following near the end of the line refering to the web site that is not resolving properly.
*DENIED* Banned extension: .com GET 0 0 Banned extension 1 403 application/json -
In this case the mime type is application/json. If you feel you want to allow this mime type you may add the following '''application/json ''' on a single line into the #/etc/dansguardian/lists/exceptionmimetypelist. This should be done carfully as now you are allowing this mime type. However it is not uncommon to add mime types.
Of course after any changes run the command ...
/etc/init.d/dansguardian restart
=====exceptionuserlist=====
=====exceptionuserlist=====
for more information.
for more information.
----
[[Category:Contrib]]
[[Category:Dungog]]
[[Category:Administration:Content Spam Virus Blocking]]
[[Category:Contrib:webfiltering]]
