GeoIP

From SME Server
Revision as of 08:20, 22 January 2020 by Trex (talk | contribs) (→‎Version)
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.


Maintainer

john crisp

stephdl Stéphane de Labrusse AKA Stephdl

Version

Contrib 10:
Contrib 9:
smeserver-geoip
The latest version of smeserver-geoip is available in the SME repository, click on the version number(s) for more information.



Warning.png Warning:
From MAXMIND site :

"Due to upcoming data privacy regulations, we are making significant changes to how you access free GeoLite2 databases starting December 30, 2019. Learn more on our blog." https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/

Quote Starting December 30, 2019, we will be requiring users of our GeoLite2 databases to register for a MaxMind account and obtain a license key in order to download GeoLite2 databases. We will continue to offer the GeoLite2 databases without charge, and with the ability to redistribute with proper attribution and in compliance with privacy regulations. In addition, we are introducing a new end-user license agreement to govern your use of the GeoLite2 databases. Previously, GeoLite2 databases were accessible for download to the public on our developer website and were licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.

Starting December 30, 2019, downloads will no longer be served from our public GeoLite2 page, from geolite.maxmind.com/download/geoip/database/*, or from any other public URL. See the section below Installation of V2 db for steps on how to migrate to the new download mechanism. End Quote


Description

The GeoIP plugin for qpsmtpd lets you know where your mail server is receiving mail from. If you're receiving too much spam from a particular location, this will help track it down. You can then use that info to reject connections from that place taking the load off your server.


Important.png Note:
Maxmind have removed support for their legacy v1 DBs and an update to v2 DBs is required. Please see the bugtracker link to bug #9033 below for more information


Installation of legacy geoip v1 db

those are still available for few time only for back compatibility purpose and to avoid a yum update mess, or if you do not trust third party repo, but be aware thos db are not updated anymore since April 2018 and were already only 80% accurate then.

yum install smeserver-geoip --enablerepo=smecontribs
config set UnsavedChanges no
signal-event geoip-update

update of geoip v1 db

if you have smeserver-geoip-1.1.2-7 already installed, this will keep you on the legacy version, only change new plugin comaptible with v1 and removal of cron update.

yum update --enablerepo=smecontribs

Installation and Updating of geoip v2 db

Important.png Note:
GeoIP2 will ONLY work on x64 CPU


Sign up for a MaxMind account (no purchase required) https://dev.maxmind.com/geoip/geoip2/geolite2/

Important - Note your login details and in particular your AccountID

Go to Services My Licence key and generate a licence key, carefully note the key details, multiple keys may be created.

The following config property keys and values will be used to set the geoip config db for ongoing updates see below

AccountID #######
LicenseKey xxxxxxxxxxxxxxx

Because the databases have been updated new perl modules are required to support GeoIP v2. The core modules have a large number of dependencies and we have decided that it is not practical for the Koozali team to maintain them all.

As a result you will now need the OpenFusion repo to install smeserver-geoip We will be syncing their mirror in due course to speed up installs and updates.

yum  --enablerepo=smeaddons install smeserver-extrarepositories-openfusion 
signal-event yum-modify 
config set UnsavedChanges no

Updating to v2 with v1 DBs installed

If you have the v1 DBs package smeserver-geoip-1.1.2-7 already installed and do not wish to update to v2 DBs you may perform updates safely, after making sure your smeserver-extrarepositories-openfusion is the latest, the following:

To carry out an install or update of the new v2 DBs you are required to firstly remove the exclude statement from the openfusion repo db

db yum_repositories delprop openfusion Exclude 
signal-event yum-modify

You may then go ahead with the following to either install or update a v2 DB package

yum install smeserver-geoip2 --enablerepo=smecontribs,openfusion
yum update --enablerepo=smecontribs,openfusion
config set UnsavedChanges no

A configuration db for geoip has been created as part of the install

# config show geoip
geoip=service
   status=enabled

Now add Maxmind AccountID and LicenseKey properties and keys to the geoip db config

db configuration setprop geoip LicenseKey "YOUR LIC KEY" AccountID "YOUR ACCT ID"

Fresh Installation of geoip v2 DBs

To carry out an install or update of the new v2 DBs you are required to firstly remove the exclude statement from the openfusion repo db

db yum_repositories delprop openfusion Exclude 
signal-event yum-modify

You may then go ahead with the following to either install or update a v2 DB package

yum install smeserver-geoip2 --enablerepo=smecontribs,openfusion
yum update --enablerepo=smecontribs,openfusion
config set UnsavedChanges no

A configuration db for geoip has been created as part of the install

# config show geoip
geoip=service
   status=enabled

Now add Maxmind AccountID and LicenseKey properties and keys to the geoip db config

db configuration setprop geoip LicenseKey "YOUR LIC KEY" AccountID "YOUR ACCT ID"

I installed the beta versions of smeserver-geoip v2

if you installed smeserver-geoip-1.2-1, smeserver-geoip-1.2-2, smeserver-geoip-1.2-3, then you have already a v2, but you still need to do as you were with legacy version and want to upgrade to v2. Otherwise you will not have the db updated anymore

db yum_repositories delprop openfusion Exclude
signal-event yum-modify

You may then go ahead with the following to either install or update a v2 DB package

yum install smeserver-geoip2 --enablerepo=smecontribs,openfusion
yum update --enablerepo=smecontribs,openfusion
config set UnsavedChanges no

A configuration db for geoip has been created as part of the install

# config show geoip
geoip=service
   status=enabled

Now add Maxmind AccountID and LicenseKey properties and keys to the geoip db config

db configuration setprop geoip LicenseKey "YOUR LIC KEY" AccountID "YOUR ACCT ID"

Update the db

(only working for geoip2 from 2019/01/03)

signal-event geoip-update

or if you prefer to reboot your server

signal-event post-upgrade; signal-event reboot

Updating the geoip database is performed by issuing the following command:

 signal-event geoip-update

This contrib adds a template to /etc/crontab to automatically update the database once a month. It's set to do so at midnight on the 5th of each month. If you have a paid subscription through the maintainer of this database (http://dev.maxmind.com/geoip/geoipupdate/) you can update more often. If you do want to update more often, you will have to create a custom template to handle the change. In addition, you will be supplied login credentials that have to be inserted into the file /etc/GeoIP.conf for the update to complete. The file is not templated so it will survive reboots. Use your favorite text editor to insert the information into this file.

More info on openfusion repo config

The openfusion repo now includes a number of packages as includes and Excludes, this is to prevent inadvertent system updates occurring when updating the old v1 DB package resulting in unstable or inaccessible systems, this is all they are for. These can be seen here:

db yum_repositories show openfusion

Testing

Now that the package and database are installed, we can test it (refer to Country Code list at end of page as required).

geoiplookup 216.17.211.37 

It should return:

GeoIP Country Edition: US, United States

It gives us the country code (US) and the long name (United States). Let's test it again with a domain name.

geoiplookup contribs.org 

Same result. So we know it works with ip addresses or domain names. Let's test it again around the world.

geoiplookup gormand.com.au 

It should return:

GeoIP Country Edition: AU, Australia 

One last time:

geoiplookup swerts-knudsen.dk 

It should return:

GeoIP Country Edition: DK, Denmark

Usage

Tracking e-mail

The qpsmtpd GeoIP plugin should now do its work. Check the qpsmtpd logs and you'll see the countries from where mail is sent.

cat /var/log/qpsmtpd/current 

We'll use a simple shell script to do the work then we'll run it.

First, create the the script.

vi geoipstats.sh

Insert the following: Code:

#!/bin/sh 
# Read the qpsmtpd log file.
# Read all of the countries and count them.  
cat /var/log/qpsmtpd/* | \
grep 'GeoIP Country:' | \
sed -e 's/^.*\(..\)$/\1/' | \
sort | uniq -c | sort -n 

Now run the script. It will show the number of messages sent by country code.

sh geoipstats.sh

See where your mail is coming from. Now ask the question, "why am I receiving thousands of email from RU -Russia? I don't even know anyone there." Good point. In addition, your server has to process all that mail, taking resources away from the server. In the next section we'll block the countries that we consider bad.

Blocking email

Add the values to the SME CADNHO db. In our case, Russia and Poland seem to causing issues. You can type in any country codes you wish.

config setprop qpsmtpd BadCountries RU,PL

Signal the email-update event.

signal-event email-update

No more mail from domains ending on .ru or .pl. The beauty of this is that the SME Server lookups happen locally on the local database rather than looking up the IP address via dns. This results in very fast responses. In addition, the plugin happens before most other plugins. This means the mail is dropped before the SME Server even has to check to see if it's on a blacklist or if it's spam.

Abbreviated Country Code List

A1      Anonymous Proxy
A2      Satellite Provider
AC      Ascension Island
AD      Andorra
AE      United Arab Emirates
AERO    members of the air-transport industry
AF      Afghanistan
AG      Antigua and Barbuda
AI      Anguilla
AL      Albania
AM      Armenia
AN      Netherlands Antilles (being phased out)
AO      Angola
AQ      Antarctica
AP      Asia/Pacific
AR      Argentina
AS      American Samoa
ASIA    Restricted to the Pan-Asia and Asia Pacific community
AT      Austria
AU      Australia
AW      Aruba
AX      Aland Islands
AZ      Azerbaijan
BA      Bosnia and Herzegovina
BB      Barbados
BD      Bangladesh
BE      Belgium
BF      Burkina Faso
BG      Bulgaria
BH      Bahrain
BI      Burundi
BIZ     Restricted for Business
BJ      Benin
BL      Saint Barthelemy
BM      Bermuda
BN      Brunei Darussalam
BO      Bolivia
BQ      Bonaire, Sint Eustatius and Saba
BR      Brazil
BS      Bahamas
BT      Bhutan
BV      Bouvet Island
BW      Botswana
BY      Belarus
BZ      Belize
CA      Canada
CC      Cocos (Keeling) Islands
CD      Congo, The Democratic Republic of the
CF      Central African Republic
CG      Congo
CH      Switzerland
CI      Cote d'Ivoire
CK      Cook Islands
CL      Chile
CM      Cameroon
CN      China
CO      Colombia
COM     Generic top-level domain
COOP    cooperative associations
CR      Costa Rica
CU      Cuba
CV      Cape Verde
CW      Curaçao
CX      Christmas Island
CY      Cyprus
CZ      Czech Republic
DE      Germany
DJ      Djibouti
DK      Denmark
DM      Dominica
DO      Dominican Republic
DZ      Algeria
EC      Ecuador
EDU     Educational Institutions
EE      Estonia
EG      Egypt
EH      Western Sahara
ER      Eritrea
ES      Spain
ET      Ethiopia
EU      European Union
FI      Finland
FJ      Fiji
FK      Falkland Islands (Malvinas)
FM      Micronesia, Federated States of
FO      Faroe Islands
FR      France
GA      Gabon
GB      United Kingdom
GD      Grenada
GE      Georgia
GF      French Guiana
GG      Guernsey
GH      Ghana
GI      Gibraltar
GL      Greenland
GM      Gambia
GN      Guinea
GOV     United States Government
GP      Guadeloupe
GQ      Equatorial Guinea
GR      Greece
GS      South Georgia and the South Sandwich Islands
GT      Guatemala
GU      Guam
GW      Guinea-Bissau
GY      Guyana
HK      Hong Kong
HM      Heard Island and McDonald Islands
HN      Honduras
HR      Croatia
HT      Haiti
HU      Hungary
ID      Indonesia
IE      Ireland
IL      Israel
IM      Isle of Man
IN      India
INFO    Generic top-level domain
IO      British Indian Ocean Territory
IQ      Iraq
IR      Iran, Islamic Republic of
IS      Iceland
IT      Italy
JE      Jersey
JM      Jamaica
JO      Jordan
JOBS    Reserved to serve needs of the international human resource management community
JP      Japan
KE      Kenya
KG      Kyrgyzstan
KH      Cambodia
KI      Kiribati
KM      Comoros
KN      Saint Kitts and Nevis
KP      Korea, Democratic People's Republic of
KR      Korea, Republic of
KW      Kuwait
KY      Cayman Islands
KZ      Kazakhstan
LA      Lao People's Democratic Republic
LB      Lebanon
LC      Saint Lucia
LI      Liechtenstein
LK      Sri Lanka
LR      Liberia
LS      Lesotho
LT      Lithuania
LU      Luxembourg
LV      Latvia
LY      Libyan Arab Jamahiriya
MA      Morocco
MC      Monaco
MD      Moldova, Republic of
ME      Montenegro
MF      Saint Martin (French part)
MG      Madagascar
MH      Marshall Islands
MIL     United States Military
MK      Macedonia, The Former Yugoslav Republic of
ML      Mali
MM      Myanmar
MN      Mongolia
MO      Macao
MOBI    consumers and providers of mobile products and services
MP      Northern Mariana Islands
MQ      Martinique
MR      Mauritania
MS      Montserrat
MT      Malta
MU      Mauritius
MUSEUM  museums
MV      Maldives
MW      Malawi
MX      Mexico
MY      Malaysia
MZ      Mozambique
NA      Namibia
NAME    individuals
NC      New Caledonia
NE      Niger
NET     Generic top-level domain
NF      Norfolk Island
NG      Nigeria
NI      Nicaragua
NL      Netherlands
NO      Norway
NP      Nepal
NR      Nauru
NU      Niue
NZ      New Zealand
OM      Oman
ORG     Generic top-level domain
PA      Panama
PE      Peru
PF      French Polynesia
PG      Papua New Guinea
PH      Philippines
PK      Pakistan
PL      Poland
PM      Saint Pierre and Miquelon
PN      Pitcairn
PR      Puerto Rico
PRO     Restricted to credentialed professionals and related entities
PS      Palestinian Territory, Occupied
PT      Portugal
PW      Palau
PY      Paraguay
QA      Qatar
RE      Reunion
RO      Romania
RS      Serbia
RU      Russian Federation
RW      Rwanda
SA      Saudi Arabia
SB      Solomon Islands
SC      Seychelles
SD      Sudan
SE      Sweden
SG      Singapore
SH      Saint Helena
SI      Slovenia
SJ      Svalbard and Jan Mayen
SK      Slovakia
SL      Sierra Leone
SM      San Marino
SN      Senegal
SO      Somalia
SR      Suriname
SS      South Sudan
ST      Sao Tome and Principe
SU      Soviet Union (being phased out)
SV      El Salvador
SX      Saint Maarten (Dutch part)
SY      Syrian Arab Republic
SZ      Swaziland
TC      Turks and Caicos Islands
TD      Chad
TEL     businesses and individuals to publish their contact data
TF      French Southern Territories
TG      Togo
TH      Thailand
TJ      Tajikistan
TK      Tokelau
TL      Timor-Leste
TM      Turkmenistan
TN      Tunisia
TO      Tonga
TP      Portuguese Timor (being phased out)
TR      Turkey
TRAVEL  entities whose primary area of activity is in the travel industry
TT      Trinidad and Tobago
TV      Tuvalu
TW      Taiwan, Province of China
TZ      Tanzania, United Republic of
UA      Ukraine
UG      Uganda
UK      United Kingdom
UM      United States Minor Outlying Islands
US      United States
UY      Uruguay
UZ      Uzbekistan
VA      Holy See (Vatican City State)
VC      Saint Vincent and the Grenadines
VE      Venezuela, Bolivarian Republic of
VG      Virgin Islands, British
VI      Virgin Islands, US
VN      Viet Nam
VU      Vanuatu
WF      Wallis and Futuna
WS      Samoa
XXX     the adult entertainment community
YE      Yemen
YT      Mayotte
ZA      South Africa
ZM      Zambia
ZW      Zimbabwe

Country Code Info Source:

http://en.wikipedia.org/wiki/ISO_3166-1
http://en.wikipedia.org/wiki/ISO_3166-1_alpha-2#Officially_assigned_code_elements

Troubleshooting

At April 2012 there may be some problems with countries not being blocked, possibly related to the way the geo database is updated for free users. User experience and opinion vary, and following a complaint, the original forum post about this has been deleted - http://forums.contribs.org/index.php/topic,48560.0.html

Users are advised to determine the effectiveness of the database for themselves.

Db compatibility with other softwares

Some software either use or depends on a geoip library, here is a table to help understand:

software repo Geoip 1 support Geoip 2 support notes on behaviour and default or settings to do
spamassassin smeos yes 3.4.2 or above
smeserver-mailsats smecontribs yes Yes with updates Requires updated perl-IO-Socket-INET6 and check_badcountries plugin: https://bugs.contribs.org/show_bug.cgi?id=10523
qpsmtpd plugin / smeserver-geoip smecontribs yes smeserver-geoip 1.2 and above new bad_countries is a fork of qpsmtpd geoip plugin, default to v2 unless v2 is not available then failback on v1
proftpd smeos only no no support for v2 yet see https://github.com/proftpd/proftpd/issues/605
apache mod_geoip smecontribs yes unknown v 1.2.10 requires libGeoIP.so.1 and GeoIP: probably only v1 db supported up there
apache mod_maxminddb yes https://github.com/maxmind/mod_maxminddb NEED BUILD
opensips-mmgeoip epel unknown
php-pecl-geoip epel yes no depends on libGeoIP.so.1
php*-php-pecl-geoip remi-safe yes no depends on libGeoIP.so.1
php-maxminddb remi no yes need to test if installs with base php. mostly not...
php*-php-maxminddb remi-safe yes depends on libmaxminddb
lighttpd-mod_geoip epel yes depends on libGeoIP.so.1
nginx-mod-http-geoip epel yes depends on libGeoIP.so.1 ; found a source for geoip2 https://github.com/leev/ngx_http_geoip2_module
python-GeoIP epel yes no
python-geoip2 python2-maxminddb no yes not available on CentOS 6 but 7.
python-pygeoip epel yes no Pure Python GeoIP API
uwsgi-plugin-geoip epel unknown
perl-Geo-IP smecontribs only no libGeoIP.so.1
perl-GeoIP2 openfusion no yes
perl-MaxMind-DB-* openfusion no yes
libmaxminddb epel 1.1.1 needed for mmdblookup
mmdblookup provided by libmaxminddb-devel (see https://bugzilla.redhat.com/show_bug.cgi?id=1663670)

Bugs

Please raise bugs under the SME-Contribs section in bugzilla and select the smeserver-geoip component or use this link .


IDProductVersionStatusSummary (2 tasks)
11675SME Contribs10.0rcCONFIRMEDMULTIPLE_RPM_OWNERS with core rpms
11546SME Contribs10.0rcUNCONFIRMEDGenerates FATAL PLUGIN ERROR [check_badcountries]: No record found for IP address x.x.x.x

Changelog

Only released version in smecontrib are listed here.

smeserver-geoip Changelog: SME 10 (smecontribs)
2021/03/18 Brian Read 1.2-18.sme
- Add expand template for the qpsmtpd peers [SME: 11023]

2021/03/14 Jean-Philippe Pialasse 1.2-17.sme
- merge legacy with main as we have few packages still using legacy [SME: 11023]

those are php*-pecl-geoip proftpd bind-libs* bind-utils.

2021/03/13 Jean-Philipe Pialasse 1.2-16.sme
- rebuild for SME10 [SME: 11023]

 make geoip2 default
create geoip-legacy package with old geoip1 stuffs
smeserver-geoip(-legacy)-update events
2020/10/06 Brian Read 1.2-15.sme
- Import to SME10 tree [SME: 11023]

2020/01/22 John Crisp 1.2-14.sme
- Change template from EditionID to ProductID

- fix 20databasestore has a trailing tilde