Log Analytics Log Insight

Integrate Anything with Log Insight Using Syslog

Why did the Log Insight team choose syslog, invented way back in the 80s, as the input protocol for Log Insight?  The 80s was the decade that brought us CD-ROMs, Doppler Radar and Digital Cell Phones.  It’s hard to believe that the syslog protocol was only a part of sendmail at first.  Now it is available for every operating system with commercial software available from multiple vendors.  If you are wondering how syslog became so popular, it is because it is easy.  I’ll show you how easy it is to send syslog messages to Log Insight from a handful of scripting languages.

First, the basics: a syslog message is made from the priority, timestamp, hostname, tag and message – in that order – of the line you want to log.  The priority is a number that codes for the facility and the “level surrounded by brackets, we’ll use <150> in our examples and you can find the complete code table on Wikipedia http://en.wikipedia.org/wiki/Syslog.  The timestamp, hostname, tag and message are just as they appear in the logs, so what you send to the syslog port looks like this:

<Priority>Month DayOfMonth Hour:Minute:Second Hostname Tag: Message

Simply construct a UDP packet with that payload and send it to port 514 on your Log Insight server.  You can do this with NetCat for instance:

nc -u loginsight.example.com 514

And type a syslog message, press Ctrl-C

Screen Shot 2014-02-04 at 12.29.56 PM

Which shows up in Log Insight like this

Screen Shot 2014-02-04 at 12.32.14 PM

Super easy to do by hand, let’s turn that into a handy bash script I like to call logger.sh:

#!/bin/bash  server="127.0.0.1" port="514" prio="150" date=`date "+%b %d %H:%M:%S"` host=`hostname`  read message  echo "<$prio>$date $host $message" | nc -w 1 -u $server $port

That will let us send something in through the standard input and send it to Log Insight over syslog.  For instance, let’s log the output of the system ‘uptime’ command with

uptime | ./logger.sh

Screen Shot 2014-02-04 at 12.37.07 PM

Just for fun, I’ll extract the load average (if you want to know all the other things you can do with extracted fields visit the Log Insight Developer Center page )

Screen Shot 2014-02-04 at 12.38.30 PM

Set it to log every 10 seconds and make a quick graph:

while `true`; do uptime | ./logger.sh ; sleep 10; done

Screen Shot 2014-02-04 at 12.50.21 PM

We can do the same thing in other scripting languages too

Ruby

#!/usr/bin/ruby  server = '127.0.0.1' port = 514 prio = "150"  require 'socket'  s = UDPSocket.new host = Socket.gethostname time = Time.now.strftime("%b %d %H:%M:%S")  while line=gets       message = "<" + prio + ">" + time + " " + host + " " + line.chomp       s.send(message, 0, server, port) end

Perl

#!/usr/bin/perl  $server = "127.0.0.1"; $port = 514; $prio = 150;  use IO::Socket; use Sys::Hostname; use POSIX 'strftime';  $host = hostname(); $time = strftime '%b %d %H:%M:%S', localtime; $line = <STDIN>;  $s = IO::Socket::INET->new(     Proto    => 'udp',     PeerPort => $port,     PeerAddr => $server, ) or die "no socketn";  $s->send('<'.$prio.'>' . $time.' '.$host.' '.$line);

Python

#!/usr/bin/python  server = "127.0.0.1" port = 514 prio = "150"  import socket import fileinput import time  sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) host = socket.gethostname() t = time.strftime("%b %d %H:%M:%S") for line in fileinput.input():       sock.sendto('<' + prio + '>' + t + " " + host + " " + line, (server, port))

PHP

#!/usr/bin/php <?php  $server = "127.0.0.1"; $port = 514; $prio = 150;  $in = fopen('php://stdin','r'); $s = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP);  while ($line = fgets($in)) {   $message = "<".$prio.">".date('M j H:m:s')." ".gethostname()." ".$line;   socket_sendto($s, $message, strlen($message), 0, $server, $port); }?>

Testing them all out

Screen Shot 2014-02-04 at 1.20.36 PM

Screen Shot 2014-02-04 at 1.20.47 PM

Now that you see how easy it is, what will you log to Log Insight?  Come and tell us about it at http://loginsight.vmware.com/

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *