SMONITOR, Simple snmp based script for monitoring systems

Description

SMONITOR is a simple shell script designed to generate e-mail alerts when processes/disk/memory problems occur on a set of hosts running snmpd.

Source

The source is located in pimentech script package: http://www.pimentech.fr/en/technologies/outils , you can also download it here: ftp://ftp.pimentech.net/src/scripts/src/shell/smonitor

#!/bin/bash

case $DEBUG in
  1) set -x
     ;;
  *)
     ;;
esac

usage() {
	echo "smonitor : host, service and network monitoring program."
	echo "usage : smtpmonitor [ -h for help ] -f <smonitor.cfg>"
	echo 
	echo "config file syntax :"
	echo "host device_number max_percentage_allowed"
	echo "To find device numbers, do a"
	echo "snmpwalk -v2c  -c public <host> .iso.org.dod.internet.mgmt.mib-2.host.hrStorage.hrStorageTable.hrStorageEntry.hrStorageDescr"
	echo "or scli -c 'show system storage' <host>"
	echo 
	echo "This script is designed for crontab : it generates output only if one of max_percentages is reached."
}

while true ; do
  case $1 in
    -d) debug=1 ; set -x ; shift 1 ;;
    -h) usage ; exit 0 ;;
	-f) conffile=$2 ; shift 2 ;;
     *) break ;;
  esac
done

if [ -z "$conffile" ] ;	then
	usage
	exit 1
fi

hdmib=".iso.org.dod.internet.mgmt.mib-2.host.hrStorage.hrStorageTable.hrStorageEntry"
psmib=".iso.org.dod.internet.private.enterprises.ucdavis.prTable.prEntry.prErrMessage"

command='snmpget -v2c -Cf -Oqv -c public ${host} ${hdmib}.${item}.${device} | grep -v "No Such Object available"'

sed -e 's/#.*$//g' -e '/^[[:space:]]*$/d' -e 's/[[:space:]]+/ /g' $conffile | \
	while read host device maxpercentage ; do
		if [ $device = "proc" ] ; then
			snmpwalk -v2c -Oq -c public ${host} ${psmib} \
				| cut -d' ' -f2- \
				| sed -e '/^[[:space:]]*$/d' -e 's/^/'${host}' : /g'
		else
			size=`item=hrStorageSize eval ${command}`
			used=`item=hrStorageUsed eval ${command}`
			
			if [ -z "$size" -o -z "$used" ]
			then
				echo "${devicename}@${host} : no response from host"
			else
				percentageused=$((100*${used}/${size}))
				if [ ${percentageused} -ge ${maxpercentage} ] ; then 
					devicename=`item=hrStorageDescr eval ${command}`
					blocksize=`item=hrStorageAllocationUnits eval ${command}|cut -d' ' -f1`
					free=$(((${size}-${used})*${blocksize}/1024/1024))
					sizem=$((${size}*${blocksize}/1024/1024))
					echo "${devicename}@${host} : ${percentageused}% used (${free}M free / ${sizem}M total)"
				fi
			fi
		fi
done

Setup

SNMP Setup

On each host you want to monitor :

Install snmpd
Edit /etc/snmp/snmpd.conf :
- Add the line
```
com2sec readonly listener.host public
```
  where "listener.host" is the host where the listener script will run.
- For each process you want to monitor, add
```
proc NAME [MAX=0] [MIN=0]
```
  where NAME is the name of the process to check for. It must match exactly (ie, http will not find httpd processes).
Restart snmpd

SMONITOR Setup

You must have the snmp client package installed on listener host.

Configuration file

For processes monitoring, simply add the line

HOST proc

for each destination host.

For disk/swap monitoring, you have to find the node number in the snmp tree. On listener host, type the command:

snmpwalk -v2c -c public HOST .iso.org.dod.internet.mgmt.mib-2.host.hrStorage.hrStorageTable.hrStorageEntry.hrStorageDescr

You should have an output like that:

HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: /
HOST-RESOURCES-MIB::hrStorageDescr.2 = STRING: /dev/pts
HOST-RESOURCES-MIB::hrStorageDescr.3 = STRING: /mnt/sdb1
HOST-RESOURCES-MIB::hrStorageDescr.101 = STRING: Real Memory
HOST-RESOURCES-MIB::hrStorageDescr.102 = STRING: Swap Space
HOST-RESOURCES-MIB::hrStorageDescr.103 = STRING: Memory Buffers

If not, ensure that UDP port 161 is opened on HOST.

For instance, if you want to be warned when disk usage exceeds 80% on the root partition, 10% on the swap partition, add

HOST 1 80
HOST 102 10

Repeat this operation for each host.

Usage

Now try to launch smonitor:

smonitor smonitor.conf

Problems on your hosts will be reported on stdout, like this

hal : Too many apache2 running (# = 219)
mdm2 : Too many postmaster running (# = 265)

No output means no problems :)

That's it ! Additionally, you can put smonitor in your crontab to get e-mail alerts.

Documentation / tags :