Cacti performance monitor on clusters

Cacti is a popular open-source graphing tool to measure system performance.
Initial setup of Cacti is manageable for a few systems, but if you need to deploy it across a set of many servers, it becomes very time-consuming to configure.

The puzzle: script the common steps for Cacti configuration to 40 servers in a cluster, avoiding as many mouse clicks as possible.

What the boss wants: disk utilization reports.  Someone in the developer group frequently overloads the hard disks with their script, causing high await. Automate the configuration of the iostat plugin to measure disk I/O.

This how-to assumes that you already have Cacti installed, and that you have one host already configured with the settings that you will deploy to several other systems.

The basic tasks needed for a cluster can be fit into a menu:

functions:
list_hosts
list_trees
list_graph_templates

Let’s try the script and see how quickly we can get Cacti configured across the cluster.

> cacti_menu.sh

Menu:
###################################
1. List existing hosts
2. List existing graph trees and templates
3. Add a new graph tree
4. Add new compute systems
5. Add hosts to trees
6. Add graphs for systems
7. Install iostat graph templates
8. Create iostat graphs for systems
###################################

> 3. Add a new graph tree

computes

> 4. Add new compute nodes

Enter the number of compute nodes.

> 48

Enter the number of the first compute node to add. (already did first one manually)

> 2

Enter the IP address range, leaving off the last octet, exampe: 192.168.0

> 192.168.0

Enter a name for these nodes. We’ll add a number to the end of the name.

> node

Select a template ID that these nodes will join, example: 3 for ucd/net SNMP host

> 3

Adding node2 (192.168.0.2) as “ucd/net SNMP Host” using SNMP v2 with community “public”
Success – new device-id: (3)

Adding node3 (192.168.0.3) as “ucd/net SNMP Host” using SNMP v2 with community “public”
Success – new device-id: (4)

… all nodes are now created in cacti.

> 6 Add hosts to trees

Enter the last ID number of compute nodes to add to tree

> 49

Enter the tree ID to add these hosts to:

> 2

Added Node node-id: (8)

> 7. Add graphs for hosts

First, list hosts:

Known hosts: (id, hostname, template, description)

1. 127.0.0.1  8  Localhost

2. 192.168.0.1  3  node1

3. 192.168.0.2  3  node2

49. 192.168.0.48  3  node48

Enter the last ID of compute nodes to graph:

> 49

Enter the first ID of compute node to graph:

> 3

Graph added – graph-id (8) – data-source-ids: (11,12,13)

All graphs are now created.

——————————————-

To go into the script, which I’ll post here:

#!/bin/bash
#Enter your cacti location, such as /usr/share/cacti or /var/www/cacti:
CACTI_LOC=/var/www/cacti
CINSTALL=/home/username/cacti_scripts

### Functions ###
function list_hosts {
 /usr/bin/php -q $CACTI_LOC/cli/add_tree.php --list-hosts
}
function list_trees {
 echo "Trees are:"
 /usr/bin/php -q $CACTI_LOC/cli/add_tree.php --list-trees
}
function list_graph_templates {
 echo "Graph templates are:"
 /usr/bin/php -q $CACTI_LOC/cli/add_graphs.php --list-graph-templates
}
# Display menu
echo "Cacti Script Menu"
echo "****************"
echo "Select an option:"
echo "1. List existing hosts"
echo "2. List existing graph trees and templates"
echo "3. Add a new graph tree"
echo "4. Add new compute nodes"
echo "5. Add hosts to trees"
echo "6. Add graphs for hosts"
echo "7. Install iostat templates"
echo "8. Create iostat graphs for systems"
echo "x. Press x to Exit script"
######
read OPTION
 case $OPTION in
1)
 echo "1. List existing hosts"
 list_hosts
 ;;
2)
 echo "2. List existing graph trees and graph templates"
 list_trees
 list_graph_templates
 ;;
3)
 echo "3. Add a new graph tree"
 echo "Enter the name for a new graph tree."
 read TREENAME
 echo "Creating tree $TREENAME"
 php -q $CACTI_LOC/cli/add_tree.php --type=tree --name="$TREENAME" --sort-method=manual
 ;;
4)
 echo "4. Add some new compute nodes"
 echo "Enter the number of compute nodes."
 read NUM_OF_COMPUTES
 echo "Number is $NUM_OF_COMPUTES"
 echo "Enter number of the first compute node to add."
 read FIRST_COMPUTENODE
 echo "$FIRST_COMPUTENODE"
 echo "Enter the IP address range, leaving off the last number, ex: 192.168.0"
 read COMPUTE_IPRANGE
 echo "$COMPUTE_IPRANGE"
 echo "Enter a name for these nodes. We'll add a number at the end of the name."
 read NODEDESC
 echo "Select a template ID that these nodes will join, example: 3"
 read TEMPLATEID
 echo "$TEMPLATEID"
# Add nodes/hosts
 for i in `seq $FIRST_COMPUTENODE $NUM_OF_COMPUTES` ;
 do
 /usr/bin/php -q "$CACTI_LOC"/cli/add_device.php --description="$NODEDESC$i" --ip="$COMPUTE_IPRANGE.$i" --template="$TEMPLATEID" --community="public" --avail=snmp --version=2
 done;
 ;;
5)
 echo "Enter last ID number of compute nodes to add to tree."
 read NUM_OF_COMPUTES
 echo "Number is $NUM_OF_COMPUTES"
 echo "Enter ID of the first compute node to add to tree."
 read FIRST_COMPUTENODE
echo "Enter the tree ID to add these hosts to."
 read TREEID
 for i in `seq $FIRST_COMPUTENODE $NUM_OF_COMPUTES` ;
 do
 /usr/bin/php -q "$CACTI_LOC"/cli/add_tree.php --type=node --node-type=host --tree-id="$TREEID" --host-id=$i
 done;
 ;;
 6)
 echo "Add graphs. List of graph templates is:"
 list_graph_templates
 list_hosts
 echo "Enter the last ID of compute nodes to graph:"
 read LAST_COMPUTENODEID
 echo "Enter first ID of compute node to graph:"
 read FIRST_COMPUTENODE
 for i in `seq $FIRST_COMPUTENODEID $LAST_COMPUTENODEID` ;
 do
 # ucd/net CPU usage:
 php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --graph-template-id=4
# ucd/net Load Average
 php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --graph-template-id=11
# ucd/net Load Average
 php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --graph-template-id=11
# ucd/net Memory Usage
 php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --graph-template-id=13

#interface eth0:
# running this next reindex line activates the eth0 monitor
 php -q $CACTI_LOC/cli/add_data_query.php --host-id=$i --data-query-id=1 --reindex-method=0
 php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=ds --graph-template-id=2 --snmp-query-id=1 --snmp-query-type-id=13 --snmp-field=ifName --snmp-value=eth0

# infiniband card ib0
php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=ds --graph-template-id=2 --snmp-query-id=1 --snmp-query-type-id=13 --snmp-field=ifName --snmp-value=ib0
done; 
;;
7)
 echo "Installing iostat templates"
 # Add all iostat templates into cacti now
 # Copy some add on files into local cacti install
 # Note: you will need to add your CACTI_LOC in add_template.php
 # something like: change include_once lines to $CACTI_LOC/...
 # Download add_template.php from the cacti site.
 pdcp -g servergroup $CINSTALL/iostat.pl /usr/local/bin/
 pdcp -g servergroup $CINSTALL/root_cron_iostat /tmp
 # Confirm whether you want to modify your root crontab here...
 pdsh -g servergroup "cat /tmp/root_cron_iostat >> /var/spool/cron/root"
 pdsh -g servergroup "/bin/rm /tmp/root_cron_iostat"
# Copy some add-on files into local cacti
 cp $CINSTALL/add_template.php $CACTI_LOC/cli/
 cp $CINSTALL/iostat.xml $CACTI_LOC/resource/snmp_queries/
for i in `ls $CINSTALL/iostat_linux_templates/ | grep xml`
 do
 /usr/bin/php $CACTI_LOC/cli/add_template.php $CINSTALL/iostat_linux_templates/$i
 done;
list_graph_templates
 ;;
8)
 echo "Iostat add-on graphs creation. List of hosts:"
 list_hosts
 echo "Enter the first system's cacti ID."
 read FIRST_ID
 echo "Enter the last system's cacti ID."
 read LAST_ID
 # Configure your disk here
 echo "Is your disk sda or sdb? "
 read DISK_NAME
 # Find the iostat graph template IDs
 echo "iostat graph template IDs found seem to be:"
 list_graph_templates | grep iostat | awk '{print $1}'
for i in `seq $FIRST_ID $LAST_ID`
 do
 /usr/bin/php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --snmp-query-id=10 --snmp-field=ioDescr --snmp-value=$DISK_NAME --graph-template-id=35
 /usr/bin/php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --snmp-query-id=11 --snmp-field=ioDescr --snmp-value=$DISK_NAME --graph-template-id=36
 /usr/bin/php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --snmp-query-id=12 --snmp-field=ioDescr --snmp-value=$DISK_NAME --graph-template-id=37
 /usr/bin/php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --snmp-query-id=13 --snmp-field=ioDescr --snmp-value=$DISK_NAME --graph-template-id=38
 /usr/bin/php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --snmp-query-id=14 --snmp-field=ioDescr --snmp-value=$DISK_NAME --graph-template-id=39
 /usr/bin/php -q $CACTI_LOC/cli/add_graphs.php --host-id=$i --graph-type=cg --snmp-query-id=15 --snmp-field=ioDescr --snmp-value=$DISK_NAME --graph-template-id=40
done
 ;;
x)
 echo "Exiting"
 ;;
 esac
 exit 0
#---------------

# Link to add_template.php

cacti ucd_net basic snmp config
cacti ucd_net basic snmp config

cacti_config2

Final config after adding iostat and NFS

cacti_config3

Advertisements