Changes

From SME Server
Jump to navigationJump to search
20,146 bytes added ,  14:31, 21 March 2016
Created page with "{{Languages}} == Dansguardian web content filtering == {{Level|Medium}} === Version === {{ #smeversion: smeserver-dansguardian-panel}} === Foreword === Thank you to Stephe..."
{{Languages}}
== Dansguardian web content filtering ==
{{Level|Medium}}

=== Version ===
{{ #smeversion: smeserver-dansguardian-panel}}


=== Foreword ===
Thank you to Stephen Noble for releasing his work.

==Dansguardian, Web Content Filter==

Users on your LAN can have their web browsing filtered, to block objectionable sites,
to perform realtime virus scanning of browsing, or to satisfy a regulatory requirement.
Filtering of web content is performed by the DansGuardian program.

A word from the Dan behind DansGuardian, Please read http://dansguardian.org/?page=copyright2 and register and/or pay and/or donate for DansGuardian as you feel appropriate.

===Translations===
The dansguardian panel is now translated into most SME languages, refer to [[:Translations]]

===ClamAV and Updates===
Previous to dansguardian-2.10.0.3-4, when ClamAV was upgraded library versions could get out of sync, eg libclamav.so.2 to libclamav.so.3 gave yum update errors.

To solve this upgrade dansguardian to at least dansguardian-2.10.0.3-4

yum update --enablerepo=smecontribs dansguardian

:http://bugs.contribs.org/show_bug.cgi?id=5111

===smeserver-dansguardian===
This provides all the SME intergration to get dansguardian running,<br>
you will need to hand edit the configuration files in /etc/dansguardian to suit.
Documentation here http://wiki.contribs.org/Dansguardian should help.

yum install dansguardian smeserver-dansguardian

Alternatively you can purchase smeserver-dansguardian-panel

===smeserver-dansguardian-panel===
Provides a server-manager panel to help in the ongoing configuration.
You can use existing or make new SME groups to give users different levels of Filtering.

Other Features include<br>
Filter Groups are setout logically with each config file presented clearly<br>
A special ''everybody'' group exist to save time enter the same site for each group<br>
Enhanced denied access page alternatives are preconfigured for you<br>
Enhanced regexp checks are given as check box options<br>
Settings are saved in a SME Database to preserve changes during upgrades

yum install smeserver-dansguardian-panel [& optionally dungog-blacklists]

Access at \server-manager > dungog.net > Web Content Filter

====Overview====

[[Image:DansOverview2.png]]

=====Global Settings=====
[[Image:DansGlobal.png]]

=====Proxy Settings=====
[[Image:DansProxy.png]]

====Filter Group====

When the proxy access method is set to Authenticate, a user is required to enter
their user password before they can have access to the internet.
Or you can use Ident to authenticate your users which does away with the need to login,
NB. Ident can be misled by multiple logins on the same PC

With authenticated users you can filters users differently,
This is set by creating SME groups.
Select your SME groups on the server-manager dansguardian panel
(with ncsa use the proxy-user panel)
You can also make PC's banned or unfiltered by adding their IP address to the panel.

Users are part of the default filter group, until you create a 2nd filter group by selecting a group from the list of pre arranged SME groups.

To keep your setup uncomplicated you could use two groups.
One group can be more restrictive and the other less restrictive.

An example of a restrictive group is one that has a blanket ban on all sites,
then a white or grey list of allowed sites

A less restrictive group may have a high weighted phrase limit, and just blacklist
sites with ads, porn and warez

Each filter group can have their own [[:smeserver-dansguardian-panel#Custom_Access_Denied_Page|custom denied access page]]
=====Settings=====
[[Image:DansFilterGroup1.png]]

=====Lists=====
[[Image:DansFilterGroup2.png]]

====Lists====
=====Phraselists=====

Phrase lists are installed by default by DansGuardian

They are the brains behind dansguardian. These contain the
phrases that are checked on each web page page. A large selection of lists are available but you have to enable them for each filter group, select modify next to each
filtergroup, select phraselists from the table, and check the lists you wish to use.

You are encouraged to send feedback and forward any changes and additions that have general use to the [http://contentfilter.futuragts.com/phraselists Phraselist maintainer], he has a later set of phrases that you can manually install over the release version.

You can add separate phrases in the weighted/allow/deny records or create you own lists.
Create your own lists by making a new directory
mkdir /etc/dansguardian/lists/phraselists/mylist
Three files can be used, but weighted must exist for the group the be recognised.
weighted contains phrases that are scored and count towards the Weighted phrase limit
banned contains phrases that cause the page to be denied
exception contains phrases that allow the page to pass
now add this list to the internal database, from the command line
db phraselist set mylist list
where mylist is the name of your list & use your own description

There are over 30 lists, below is just the top of the page
[[Image:DansPhraselist2.png]]

=====Blacklists=====

You can if you wish install blacklists from mesd.k12.or.us or many other sources, including commercial lists like those available from [http://www.squidblacklist.org Squidblacklist.org - Blacklists For Squid Proxy & More.]
You can download a rpm from dungog.net/sme or
this can be updated or installed with rsync, run from the command line
or add /usr/bin/rsync-sgbl to cron, weekly or monthly. (sgbl=squidguard blacklist)
There is alternate commercial blacklist from URLBlacklist.com
You select which individual black/white/greylists to use for each filter group.

Although this is called a blacklist, the categories can be used as white or grey lists also. Being listed does not infer that the site is bad - these are just lists of sites.

If you choose to use or trial the lists from blacklist .com, download the tgz file,
uncompress and move to the /etc/dansguardian/blacklists directory.

You can create your own lists by making a new directory
mkdir /etc/dansguardian/blacklists/mylist
two files are used
domains contains whole sites eg mysite.com
urls contains parts of sites eg mysite/part
now add this list to the internal database
db blacklist set mylist list
where mylist is the name of your list & use your own description

[[Image:DansBlacklist2.png]]

=====General Lists=====
[[Image:DansLists2.png]]

Also see http://wiki.contribs.org/Dansguardian/ConfigFiles

*Banned, Exception and Grey Lists
These lists can override other settings such as weighted phrase or blacklists. They either allow or deny a page depending on the settings. The grey lists override the banned lists. The exception lists override the banned lists also. The difference is that the exception lists completely switch off *all* other filtering for the match. Grey lists only stop the URL filtering and allow the normal filtering to work.

You add records to the default lists in the ''Lists Configuration'' page.
If you have a lot records to add you can prepare a file and insert it into the template directory.
You are prompted with the file name on each page.

You can use symbolic links to expose the site config file into an ibay for easier access, you must be sure that anyone who edits the file knows to use a unix file format.

*Exceptionsitelist, Bannedsitelist, Greysitelist
Affects the hostname part of a URL eg yahoo.com or for finer control mail.yahoo.com
You can affect everything from the .us domain with .us or allow all things australian
by using just using .au

*ExceptionURLlist, BannedURLlist, GreyURLlist
Affects the parts of a domain eg abc.net.au/children or bbc.co.uk/cricket will affect
the childrens and cricket sections of the domains

*Exceptionphraselist, Bannedphraselist, Weightedphraselist
While checking the contents of a page will block or allow if these phrases are found.
This is slightly different to weighted phrases which scores the contents and won't
have an affect until enough the set limit is reached.

A word or phrase is enclosed by < sex> angle brackets, a leading or trailing space
inside the angle brackets is significant. eg [space]sex will not find middlesex

*Exceptioniplist, Bannediplist
Affects a PC on the local network with that IP address, Note. SMEserver can assign
a static IP based on a network card's MAC address via the hostname and addresses panel

*Exceptionuserlist, Banneduserlist
Affects a user when the proxy access method is set to Pam Auth, see the next
section for details, This is set by selecting a SME group.

*Exceptionvirusmimetype, Exceptionvirusextension, Exceptionvirussitelist, Exceptionvirusurllist
When virus scanning of browsing is enabled these files or sites are not scanned

*Bannedregexpurllist
Affects a URL that contains a pattern that is matched by a unix regular expression.
This is very powerful but also difficult to understand and get right if you don't
know your regular expression rules.

*Bannedfileextlist
Common catagories of files have been grouped so you only need to check a box
on the filter group page. You can ban other file types not included in that list.

*Bannedmimetypelist
Affects files of a defined mime type

*Greyurllist, Greysitelist
An example of grey list use is when in Blanket Block (whitelist) mode and you want to allow some sites but still filter as normal on their content. Another example of grey list use is when you ban a site but want to allow part of it. <br>
The greyurllist is for partly unblocking PART of a site<br>
The greysitelist is for partly unblocking ALL of a site<br>

====Access Denied====
When a page is blocked the denied usage screen is displayed.
The details of why the page was blocked can be brief or detailed depending on the settings.

The override bypass link is shown if the user is authenticated, the reporting level is set to report details and the bypass link is enabled in the filtergroup

Each filter group can have their own denied access page

[[Image:DansDenied.png]]

The denied access page can be stripped down to the bare minimum, x (blocked) + (bypass)

This version is available in the next release 2.9.9.1 with
db dungog setprop dansguardian deniedurl yourserver.net/cgi-bin/denied.pl
[[Image:DansDenied2.png]]

====Proxy Access and Browser Setup====
=====ldap=====
Authenticate against an LDAP server

BETA, from smeserver-dansguardian-panel-2.9-19

Tested with ldap on SME, may need refinement with MS Active Directory

This isn't 'Single Sign On'. The user is prompted for their LDAP/AD username and password. If users tick remember and save password this is only a small inconvenience.

Two tests need to be run to verify your LDAP settings and two db settings saved.

The settings are your ldap server hostname.domainname, just an IP will do
config setprop squid host ldap://k8.232.net

And your ldap server Distinguised Name
config setprop squid dn dc=232,dc=net

Test these are correct with

1. Authenticate against LDAP
/usr/lib/squid/squid_ldap_auth -b dc=232,dc=net -f uid=%s -h ldap://k8.232.net

the server waits for you to enter a username, then a space then the password, success with an OK

sam SamSam987^%$
OK

2. Retrieve filter group members, eg. for the group students, where the attribute of the users is memberUid
yum install openldap-clients

ldapsearch -x -LLL -H ldap://k8.232.net -b dc=232,dc=net cn=students memberUid
dn: cn=students,ou=Groups,dc=232,dc=net
memberUid: bernard
memberUid: stephen

Let us know if you need to change the command to connect, and we can add to smeserver-dansguardian-panel

see also
man squid_ldap_auth
man ldapsearch

eg if the LDAP server requires authentication, for squid_ldap_auth add something like -D cn=root,dc=232,dc=net -W /etc/ldap.pwd


set the browser to use <nowiki>http://proxy/proxy.pac</nowiki>, users are required to have valid accounts on the LDAP server and must enter their username/password to access the proxy.

=====pam=====
set the browser to use <nowiki>http://proxy/proxy.pac</nowiki>, users are required to have valid accounts on the server and must enter their username/password to access the proxy.

=====ncsa=====
set the browser to use <nowiki>http://proxy/proxy.pac</nowiki>, users are NOT required to have valid accounts on the server users must enter their username/password to access the proxy. Create a user password file and assign users to groups.

To add users to the NCSA database /home/e-smith/db/proxyusers

we have a panel [[:Dungog-proxyusers|dungog-proxyusers]]
yum install dungog-proxyusers

or ...

db proxyusers set stephen user password 6ecreT group staff
db proxyusers set jimmy user password wiggles group students

where groups staff and students are enabled in the dansguardian panel
as 2nd or 3rd filter group, bypass, banned or unfiltered

you can edit passwords and groups by
db proxyusers setprop password fruit5ly group students

after adding users
signal-event proxy-passwd

you may create or import a file in this format

stephen=user|password|6ecreT|group|staff
jimmy=user|password|lItt6kk|group|students
then
chmod 640 /home/e-smith/db/proxyusers
chown root.admin /home/e-smith/db/proxyusers

=====ident=====
set the browser to use <nowiki>http://proxy/proxy.pac</nowiki>, If you are using ident auth, you will require a ident client on your workstation. One windows ident client is available from: https://sourceforge.net/projects/retinascan.

In some cases, the Windows firewall blocks access to the ident client and you will have to add an exception in your firewall rules as follows: <br>
Control Panel > Windows Firewall > Exceptions > Add Port <br>
Name: ''auth'' > Port number: ''113'' > ''TCP''

=====transparent proxy=====
no browser setup is needed. will filter on 8080 or the port you nominate. Note, this can be bypassed by the user entering 3128 in their browser.

=====disable dansguardian=====
resets transparent proxy to 3128, remember to untick port blocking if you enabled it.

Your Operating system may allow you to lock down your browser proxy settings,
an alternative is to use the tick box in the panel to block ports 3128 to stop the filter being bypassed.

====Help====

=====Restarting Dansguardian=====
With a 'save & restart' Squid is restarted, Squid must restart before dansguardian, if it hasn't try 'save & reload' which doesn't restart squid or drop to command line and check. You can check if dansguardian is running with:
ps ax |grep dans
to start or stop from the command line see
dansguardian -h

Restarting dansguardian from the panel affects users differently depending on the button
the options are:

Restart
-Q kill any running copy AND start a new one with current options.

Reload
-r closes all connections and reloads config files by issuing a HUP,
but this does not reset the maxchildren option.

=====Custom Access Denied Page=====
*CGI
To create/edit a custom .pl you have two options <br>
create a new .pl file, dansguardianfN.pl and edit to suit <br>
cp /home/e-smith/files/ibays/Primary/cgi-bin/dansguardian.pl <br>
to /home/e-smith/files/ibays/Primary/cgi-bin/dansguardianfN.pl <br>
where N is the filtergroup number

or set a db value deniedurl which overrules the above method, see db section below

*HTML
to create/edit a custom .html <br>
You can edit a html template in /etc/dansguardian/languages / LANGUAGE / template.html <br>
LANGUAGE defaults to ukenglish but you can set with a DB command

make a copy relative to your filter level eg templatef2.html in your language directory

you can edit the default but it will be overwritten when you upgrade the Dansguardian rpm, so make a copy as templatef0.html which will be used if it exists

html template doesn't include a bypass link

=====DB settings=====
Not all settings can be set from the panel,
you can set these settings with db commands,
activate db settings with
signal-event dansguardian-reload


*Language support, see options in /etc/dansguardian/languages, default is ukenglish
db dungog setprop dansguardian language danish

*Set an alternate page denied url, eg. for filter group 2
db dungog setprop dansguardianf2 deniedurl 2321.net/cgi-bin/deniedf2.pl
then select and save this value in the filtergroup panel

*change default denied page
db dungog setprop dansguardian deniedurl 2321.net/cgi-bin/denied.pl

*to just change from the Primary domain to another of your domains
db dungog setprop dansguardian wsn 4545.org

*POST protection, eg. uploads, forms etc. <br>
Maximum Size of file allowed to be uploaded <br>
default is -1 (no restrictions) <br>
or enter a size in kb's eg. <br>
0 = complete block <br>
500 = 500 kb <br>
5000 = 5 mb <br>
db dungog setprop dansguardian maxuploadsize -1

*A shortcut to entering a set of banned extensions, where fX is the filtergroup f1-f5
db dungog setprop dansguardian bannedextfX exe on (executable)
db dungog setprop dansguardian bannedextfX macro on (macros and viruses)
db dungog setprop dansguardian bannedextfX arc on (archives)
db dungog setprop dansguardian bannedextfX time on (bandwidth wasting)


=====Time base restrictions=====
An alternative or additional method of control is to use a script to change db settings with cron,

see /usr/bin/dproxy for an example.

This would allow you to ban access to the internet for a group or to give unfiltered access. Make a copy of your altered script so it isn't overwritten by the next rpm update, and enable the changes with a cron job.

say your copy is /usr/bin/kidproxy<br>
give access at 17:00 with /usr/bin/kidproxy open<br>
then shutdown at 19:00 with /usr/bin/kidproxy close

=====MSN=====
To block MSN Messanger add the following to [mime types - Deny]

application/x-msn-messenger

=====Troubleshooting=====

*Switch off or modify firewalls which block port 8080 on the client PC

*A few users have had problems with transparent proxying, and we cant work out why, it's probably network issues. If this happens, which is uncommon, the best we can suggest is to use ident and set 8080 in your browser. Without adding an ident client you are assumed to be in the default filter group.

*If the 'denied access' page comes up as follows, it is a problem with the syntax of your edited denied page or denied page url.
DansGuardian - 400 Bad Request

* Bypassing the proxy selectively

You have Transparent Proxy enabled but want to allow this to be selectively bypassed. <br>
or you have devices eg TiVo that you want to bypass squid <br>
http://wiki.contribs.org/Firewall#Bypass_Proxy

the smeserver-adv-masq rpm in dungogMembers contains these fragments, and the db entries can be added in the <br>
''Modify status and proxy values.'' sub-panel

* Trusted sites that you want unauthenticated access to can be added to the 'Common' exceptionsitelist

ie Common > modify > a site > allow <br>
this will bypass dansguardian and squid authentication.

* Email if problems continue after running through these steps

check yum at the command line
yum update
and
yum update --enablerepo=smecontribs

check logs

/var/log/messages
/var/log/squid/access.log
/var/log/dansguardian/access.log

check if dansguardian is running
ps ax

what error does if give trying to start

make sure it is stopped
dansguardian -q

start it
dansguardian


check templates are expanded and restarted
signal-event dansguardian-save
wait for squid to restart
signal-event dansguardian-reload



=== Bugs ===
Please raise bugs under the SME-Contribs section in {{BugzillaFileBug|product=|component=|title=bugzilla}}and select the smeserver-dansguardian-panel component or use {{BugzillaFileBug|product=SME%20Contribs|component=smeserver-dansguardian-panel|title=this link}}.

{{#bugzilla:columns=id,product,version,status,summary |sort=id |order=desc |component=smeserver-dansguardian-panel|noresultsmessage="No open bugs found."}}


===Changelog===
Only versions released in smecontrib are listed here.

{{ #smechangelog: smeserver-dansguardian-panel}}
----
[[Category:Contrib]]
[[Category:Dungog]]
[[Category:Administration:Content Spam Virus Blocking]]
[[Category:Security]]
Super Admin, Wiki & Docs Team, Bureaucrats, Interface administrators, Administrators
3,254

edits

Navigation menu