Difference between revisions of "SYSLOG Forwarding"

From SME Server
Jump to navigationJump to search
m
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
==Forwarding Log Information from SME to a SYSLOG Collector (SPLUNK)==
+
==SME8.x==
 +
====Fowarding syslog stream tot a remote host====
 +
Create the following a custom template directory on your source server (example 192.168.1.1)
 +
mkdir -p /etc/e-smith/templates-custom/etc/syslog.conf
 +
and copy the existing template fragments to this new custom template directory:
 +
/etc/e-smith/templates/etc/syslog.conf/auth
 +
/etc/e-smith/templates/etc/syslog.conf/authpriv
 +
/etc/e-smith/templates/etc/syslog.conf/daemon
 +
/etc/e-smith/templates/etc/syslog.conf/kern
 +
/etc/e-smith/templates/etc/syslog.conf/syslog
 +
you may copy any available template fragment for which you would like to send the syslog messages to the remote host.
  
 +
then modify the the copied template fragment in the custom dirctory as follows:
 +
authpriv.*                                      @192.168.1.170
 +
where 192.168.1.170 is the IP address of the remote host. Obviously this is an example and you should use the IP address of your real syslog collecting server.
 +
 +
To avoid unwanted mark messages to be send to the remote host, create the following custom template directory:
 +
mkdir -p /etc/e-smith/templates-custom/etc/sysconfig/syslog
 +
 +
and create the following template-fragment within this directory:
 +
touch /etc/e-smith/templates-custom/etc/sysconfig/syslog/10NoMARKs
 +
with the following content
 +
# we don;t want the MARK ticks
 +
SYSLOGD_OPTIONS="-r -m 0"
 +
 +
 +
The new templates need to be expanded by:
 +
expand-template /etc/syslog.conf
 +
expand-template /etc/sysconfig/syslog
 +
 +
And restart syslog:
 +
service syslog condrestart
 +
From here on, all syslog messages will be send to the remote host over port 514
 +
 +
On the remote host (192.168.1.170) there a 2 actions required to be able to receive remote syslog messages:
 +
* open UDP/TCP port 514
 +
* forwarding incomming traffic from our syslog server (192.168.1.1) on port 514 to localhost (127.0.0.1)
 +
 +
 +
=====Uninstall=====
 +
To uninstall the forwarding of syslog messages on your source server, remove the custom template directory:
 +
rm -f /etc/e-smith/templates-custom/etc/syslog.conf
 +
and restart syslog
 +
service syslog condrestart
 +
 +
 +
 +
====Forwarding syslog stream to a collector====
 +
Syslog information can also be send to a special agent on a remote host (e.g. Splunk, Logrythm)
 +
 +
=====Syslog forwarding for Splunk=====
 
If you would like to be able to collect and monitor your SME server using '''SYSLOG''', the following mini '''Howto''' shows how you can enable your SME to Forward Event Log information to an external '''SYSLOG''' Collector like SPLUNK.
 
If you would like to be able to collect and monitor your SME server using '''SYSLOG''', the following mini '''Howto''' shows how you can enable your SME to Forward Event Log information to an external '''SYSLOG''' Collector like SPLUNK.
 
Please note that this '''Howto''' assumes you have a running SPLUNK server (v6.xx) in your infrastructure with ports 514 and 9997 enabled. Visit http://www.splunk.com for more product information and installation documentation.  
 
Please note that this '''Howto''' assumes you have a running SPLUNK server (v6.xx) in your infrastructure with ports 514 and 9997 enabled. Visit http://www.splunk.com for more product information and installation documentation.  
Line 24: Line 73:
 
10. At this point you can test to make sure the connection between SME and SPLUNK is working by running the command: '''./splunk list forwarder'''
 
10. At this point you can test to make sure the connection between SME and SPLUNK is working by running the command: '''./splunk list forwarder'''
  
Next you will need to add which logs files will be forwarded and processed by SPLUNK. Run the following commands for SME logs (Steps 11-16).
+
Next you will need to add which logs files will be forwarded and processed by SPLUNK. Run the following commands for SME logs (Steps 11-17).
  
 
11. '''./splunk add monitor /var/log/messages -index main -sourcetype syslog'''
 
11. '''./splunk add monitor /var/log/messages -index main -sourcetype syslog'''
Line 38: Line 87:
 
16. '''./splunk add monitor /var/log/boot.log -index main -sourcetype syslog'''
 
16. '''./splunk add monitor /var/log/boot.log -index main -sourcetype syslog'''
  
17. Restart to the SPLUNK server to commit all changes: '''./splunk restart'''
+
17. '''./splunk add monitor /var/log/yum/yum.log -index main -sourcetype syslog'''
 +
 
 +
18. Restart to the SPLUNK server to commit all changes: '''./splunk restart'''
  
 
Within a few moments if you should begin to see SYSLOG updates to your SPLUNK server from SME over port 9997.
 
Within a few moments if you should begin to see SYSLOG updates to your SPLUNK server from SME over port 9997.
  
==Deployment considerations==
+
=====Deployment considerations=====
If you would like to seperate the data being collected from SME on your SPLUNK server, you can create a new index, ie: SME and replace the commands on lines 11-16 with:<br>  
+
If you would like to seperate the data being collected from SME on your SPLUNK server, you can create a new index, ie: SME and replace the commands on lines 11-17 with:<br>  
 
'''./splunk add monitor /''<pathtologs>''/ -index sme -sourcetype syslog'''<br>  
 
'''./splunk add monitor /''<pathtologs>''/ -index sme -sourcetype syslog'''<br>  
 
then you can create a report/filter or dashboard on the keyword "SME"
 
then you can create a report/filter or dashboard on the keyword "SME"
  
==Cleaning Data Indexes==
+
=====Cleaning Data Indexes=====
 
If you need to clear data that is being collected by SPLUNK from SME you can run the following command:<br>
 
If you need to clear data that is being collected by SPLUNK from SME you can run the following command:<br>
''on your SPLUNK server'': '''splunk clean evendata -index ''<indexname>'''''<br>(where ''<indexname>'' is main, or sme, etc.)
+
''on your SPLUNK server'': '''splunk clean eventdata -index ''<indexname>'''''<br>(where ''<indexname>'' is main, or sme, etc.)  
 +
(or command: splunk clean eventdata ''<indexname>'')
 +
[[category:howto]]

Latest revision as of 14:16, 20 September 2014

SME8.x

Fowarding syslog stream tot a remote host

Create the following a custom template directory on your source server (example 192.168.1.1)

mkdir -p /etc/e-smith/templates-custom/etc/syslog.conf

and copy the existing template fragments to this new custom template directory:

/etc/e-smith/templates/etc/syslog.conf/auth
/etc/e-smith/templates/etc/syslog.conf/authpriv
/etc/e-smith/templates/etc/syslog.conf/daemon
/etc/e-smith/templates/etc/syslog.conf/kern
/etc/e-smith/templates/etc/syslog.conf/syslog

you may copy any available template fragment for which you would like to send the syslog messages to the remote host.

then modify the the copied template fragment in the custom dirctory as follows:

authpriv.*                                      @192.168.1.170

where 192.168.1.170 is the IP address of the remote host. Obviously this is an example and you should use the IP address of your real syslog collecting server.

To avoid unwanted mark messages to be send to the remote host, create the following custom template directory:

mkdir -p /etc/e-smith/templates-custom/etc/sysconfig/syslog

and create the following template-fragment within this directory:

touch /etc/e-smith/templates-custom/etc/sysconfig/syslog/10NoMARKs

with the following content

# we don;t want the MARK ticks
SYSLOGD_OPTIONS="-r -m 0"


The new templates need to be expanded by:

expand-template /etc/syslog.conf
expand-template /etc/sysconfig/syslog

And restart syslog:

service syslog condrestart

From here on, all syslog messages will be send to the remote host over port 514

On the remote host (192.168.1.170) there a 2 actions required to be able to receive remote syslog messages:

  • open UDP/TCP port 514
  • forwarding incomming traffic from our syslog server (192.168.1.1) on port 514 to localhost (127.0.0.1)


Uninstall

To uninstall the forwarding of syslog messages on your source server, remove the custom template directory:

rm -f /etc/e-smith/templates-custom/etc/syslog.conf

and restart syslog

service syslog condrestart


Forwarding syslog stream to a collector

Syslog information can also be send to a special agent on a remote host (e.g. Splunk, Logrythm)

Syslog forwarding for Splunk

If you would like to be able to collect and monitor your SME server using SYSLOG, the following mini Howto shows how you can enable your SME to Forward Event Log information to an external SYSLOG Collector like SPLUNK. Please note that this Howto assumes you have a running SPLUNK server (v6.xx) in your infrastructure with ports 514 and 9997 enabled. Visit http://www.splunk.com for more product information and installation documentation.

1. Download SPLUNK Universal Forwarder from: http://www.splunk.com/download?r=header or direct download (you will need to establish a SPLUNK account, if you don't already have one -- you cannot WGET this file): http://www.splunk.com/page/download_track?file=6.1.1/splunk/linux/splunk-6.1.1-207789-linux-2.6-x86_64.rpm&platform=Linux&architecture=x86_64&version=6.1.1&product=splunk&typed=release&name=linux_installer&d=pro

2. Upload the SPLUNK .rpm package to your SME server via SFTP/FTP to /tmp

3. ssh to your SME server

4. cd /tmp

5. rpm -ivh splunk-6.1.1-207789-linux-2.6-x86_64.rpm (latest version as of this post)

6. cd /opt/splunkforwarder/bin

7. Enable boot-start init script: ./splunk enable boot-start

8. Start Splunk Forwarder service: ./splunk start

9. Enable Forwarding: ./splunk add forward-server <splunk-server-ip-or-hostname.domain>:9997

10. At this point you can test to make sure the connection between SME and SPLUNK is working by running the command: ./splunk list forwarder

Next you will need to add which logs files will be forwarded and processed by SPLUNK. Run the following commands for SME logs (Steps 11-17).

11. ./splunk add monitor /var/log/messages -index main -sourcetype syslog

12. ./splunk add monitor /var/log/secure -index main -sourcetype syslog

13. ./splunk add monitor /var/log/cron -index main -sourcetype syslog

14. ./splunk add monitor /var/log/maillog -index main -sourcetype syslog

15. ./splunk add monitor /var/log/spooler -index main -sourcetype syslog

16. ./splunk add monitor /var/log/boot.log -index main -sourcetype syslog

17. ./splunk add monitor /var/log/yum/yum.log -index main -sourcetype syslog

18. Restart to the SPLUNK server to commit all changes: ./splunk restart

Within a few moments if you should begin to see SYSLOG updates to your SPLUNK server from SME over port 9997.

Deployment considerations

If you would like to seperate the data being collected from SME on your SPLUNK server, you can create a new index, ie: SME and replace the commands on lines 11-17 with:
./splunk add monitor /<pathtologs>/ -index sme -sourcetype syslog
then you can create a report/filter or dashboard on the keyword "SME"

Cleaning Data Indexes

If you need to clear data that is being collected by SPLUNK from SME you can run the following command:
on your SPLUNK server: splunk clean eventdata -index <indexname>
(where <indexname> is main, or sme, etc.) (or command: splunk clean eventdata <indexname>)