SMONITOR, Simple snmp based script for monitoring systems

Description

SMONITOR is a simple shell script designed to generate e-mail alerts when processes/disk/memory problems occur on a set of hosts running snmpd.

Source

The source is located in pimentech script package: http://www.pimentech.fr/en/technologies/outils , you can also download it here: ftp://ftp.pimentech.net/src/scripts/src/shell/smonitor

#!/bin/bash
# DEPENDS: snmp mib-snmp-downloader
case $DEBUG in
  1) set -x
     ;;
  *)
     ;;
esac

usage() {
    echo "smonitor : host, service and network monitoring program."
    echo "usage : smtpmonitor [ -h for help ] -f <smonitor.cfg>"
    echo
    echo "config file syntax :"
    echo "host device_index max_percentage_allowed"
    echo "To find device index, do a"
    echo "snmpwalk -v2c  -c public <host> .1.3.6.1.2.1.25.2.3.1.3"
    echo "(last index)"
    echo "or scli -c 'show system storage' <host>"
    echo
    echo "This script is designed for crontab : it generates output only"
    echo "if one of max_percentages is reached."
}

while true ; do
  case $1 in
    -d) debug=1 ; set -x ; shift 1 ;;
    -h) usage ; exit 0 ;;
    -f) conffile=$2 ; shift 2 ;;
     *) break ;;
  esac
done

if [ -z "$conffile" ] ; then
    usage
    exit 1
fi

# .iso.org.dod.internet.mgmt.mib-2.host.hrStorage.hrStorageTable.hrStorageEntry
hdmib=".1.3.6.1.2.1.25.2.3.1"
# .iso.org.dod.internet.private.enterprises.ucdavis.prTable.prEntry.prErrMessage
psmib=".1.3.6.1.4.1.2021.2.1.101"

command='snmpget -v2c -Cf -Oqv -c public ${host} ${hdmib}.${item}.${device} | grep -v "No Such Object available"'

sed -e 's/#.*$//g' -e '/^[[:space:]]*$/d' -e 's/[[:space:]]+/ /g' $conffile | \
    while read host device maxpercentage ; do
        if [ $device = "proc" ] ; then
            (snmpwalk -v2c -Oq -c public ${host} ${psmib} 2>/dev/null \
                || sleep 1 || snmpwalk -v2c -Oq -c public ${host} ${psmib}) \
                | cut -d' ' -f2- \
                | sed -e '/^[[:space:]"]*$/d' -e 's/^/'${host}' : /g'
        else
            # hrStorageSize
            size=$(item=5 eval ${command})
            # hrStorageUsed
            used=$(item=6 eval ${command})

            if [ -z "$size" -o -z "$used" ]
            then
                # 2nd try
                sleep 2
                size=$(item=5 eval ${command})
                used=$(item=6 eval ${command})
            fi
            if [ -z "$size" -o -z "$used" ]
            then
              echo "${devicename}@${host} : no response from host after 2 tries"
            else
              percentageused=$((100*${used}/${size}))
              if [ ${percentageused} -ge ${maxpercentage} ] ; then
                  # hrStorageDescr
                  devicename=$(item=3 eval ${command})
                  # hrStorageAllocationUnits
                  blocksize=$(item=4 eval ${command}|cut -d' ' -f1)
                  free=$(((${size}-${used})*${blocksize}/1024/1024))
                  sizem=$((${size}*${blocksize}/1024/1024))
                  echo "${devicename}@${host} : ${percentageused}% used (${free}M free / ${sizem}M total)"
              fi
            fi
        fi
done

Setup

SNMP Setup

On each host you want to monitor :

  1. Install snmpd

  2. Edit /etc/snmp/snmpd.conf :

    • Add the line

      com2sec readonly listener.host public
      

      where "listener.host" is the host where the listener script will run.

    • For each process you want to monitor, add

      proc NAME [MAX=0] [MIN=0]
      

      where NAME is the name of the process to check for. It must match exactly (ie, http will not find httpd processes).

  3. Restart snmpd

SMONITOR Setup

You must have the snmp client package installed on listener host.

Configuration file

For processes monitoring, simply add the line

HOST proc

for each destination host.

For disk/swap monitoring, you have to find the node number in the snmp tree. On listener host, type the command:

snmpwalk -v2c -c public HOST .iso.org.dod.internet.mgmt.mib-2.host.hrStorage.hrStorageTable.hrStorageEntry.hrStorageDescr

You should have an output like that:

HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: /
HOST-RESOURCES-MIB::hrStorageDescr.2 = STRING: /dev/pts
HOST-RESOURCES-MIB::hrStorageDescr.3 = STRING: /mnt/sdb1
HOST-RESOURCES-MIB::hrStorageDescr.101 = STRING: Real Memory
HOST-RESOURCES-MIB::hrStorageDescr.102 = STRING: Swap Space
HOST-RESOURCES-MIB::hrStorageDescr.103 = STRING: Memory Buffers

If not, ensure that UDP port 161 is opened on HOST.

For instance, if you want to be warned when disk usage exceeds 80% on the root partition, 10% on the swap partition, add

HOST 1 80
HOST 102 10

Repeat this operation for each host.

Usage

Now try to launch smonitor:

smonitor smonitor.conf

Problems on your hosts will be reported on stdout, like this

hal : Too many apache2 running (# = 219)
mdm2 : Too many postmaster running (# = 265)

No output means no problems :)

That's it ! Additionally, you can put smonitor in your crontab to get e-mail alerts.

Commentaires

Comments

blog comments powered by Disqus