Tag Archives: monitoring

Posted on
by

BE-UnlimitedOk, so some of the regular readers of this blog will sense a bit of a theme with my recent posts, and get the feeling that I’m essentially trying to graph the world.

Guilty.

Anyway, I get my home internet through Be. The ADSL line around here was a little flakey some time ago, and after going through the 3rd splitter in as many weeks I got a BT engineer out to sort out the local switching equipment. Things are working fine and dandy now, however I thought it would be cool to keep an eye on the ADSL modem’s key stats, just in case I had any more problems.

Getting started

In the 3rd party contributor repository, there is a plugin called BeBoxSync, written by Alex Dekker. The documentation for this plugin appears to have only existed on his website, which appears to no longer be available and has expired from google cache.

The plugin consists of two perl scripts which use an expect script to pull information from the ADSL modem via telnet. These original scripts may work out of the box for you, however for my BeBox (Thomson TG585v7 modem running software version 8.2.7.7), I needed to make some changes to get it to work. Basically, I needed to get the expect script to probe for extended information (which is no longer provided by the adsl info command), and to look in a slightly different part of the output for some required data.

My modifications are available on github, and the originals are here. I’d suggest you try my version first, as the originals haven’t been maintained for a fairly long time.

Installation

First, you must install expect. Expect is a little tool that lets you script interactive sessions like telnet. It is quite often installed on default installations, but is considered rather oldskool, so may not be (it wasn’t on my Debian 6 server)…

apt-get install expect

Next, after you have downloaded the scripts to somewhere sensible, you will need to make the following modifications:

  • Edit beboxstats.expect and enter the IP address of your modem and your administrator password in the appropriate places.
  • Edit beboxstats and enter the absolute path to the beboxstats.expect script.
  • Edit beboxsync and do the same.
  • Ensure all three scripts are executable by the munin user

You can test things are working by executing each script from the terminal. You should see a whole bunch of data about your modem when executing the expect script, and the values for each key field listed in munin plugin format when executing the munin scripts. Check these values against the values on your modem’s stats page (default: http://192.168.1.254/cgi/b/dsl/dt/?be=0&l0=1&l1=0) to verify that you are getting the correct values reported.

Finally, link to your scripts from within your munin plugins directory in the usual way. If things are working, you should see some new graphs appear in your “network” section.

beboxsync-day beboxstats-day

As you can see from these images, I monitored a sharp drop in line quality with a corresponding drop in bandwidth. Still investigating the cause…

Posted on
by

In a previous post I wrote about a little bit of software which let you extract information from the output of the Weather Display weather station software.

Lately, I’ve been playing around with munin a fair bit, so I thought it’d be pretty neat to combine the two, and have munin keep an eye on my local airfield and monitor the winds, pressure and temperature. So, I had a quick skim of the plugin writing tutorial on their wiki, and bashed together some munin plugins.

Writing plugins for munin is actually remarkably straightforward, and very quickly my computer began generating some very pretty graphs…

wdwind_www_enstoneaerodrome_co_uk-day

wdtemp_www_enstoneaerodrome_co_uk-day

wdpressure_www_enstoneaerodrome_co_uk-day

* The outside temperature sensors at Enstone seems to have run out of batteries, hence the rather odd reading.

Hope this is useful to someone, code in the usual place!

» Visit the project on Github…

Posted on
by

MuninMunin is a powerful and highly customisable network monitoring tool. It lets you collect data about pretty much anything, from the number of apache processes running, to the number of IP addresses blocked by fail2ban, to the current temperature of your CPU.

All this data is collected and graphed over time, and provides you with a powerful insight into the performance of your infrastructure, and helps track down the root cause of any failure or outage.

I monitor just about anything I can, and one of the many things that I graph are the number of PHP errors appearing in my apache logs. This gives me an idea of the overall health of some of the platforms that I develop, as well as spotting instances where I introduce a regression which would otherwise not be spotted.

Powering this, I use a munin plugin called loggrep, which, as the name implies, works by grepping a log file for a certain bit of regular expression and returning a count of every instance found. This is simple but very effective, however, it took a little bit of fiddling to get it working.

The symptoms

You install munin, and all the other plugins are present and generating graphs, but those generated by loggrep are missing. You follow the munin troubleshooting steps and you find that while you can connect to to the node and the plugin appears to be running and collecting data:

host #: telnet localhost 4949
# munin node at host
fetch loggrep_foo
count.value 35
errors.value 12
.

The plugin does not appear in the list when you telnet in and type “list“.

The fix

After a lot of hacking about, I managed to fix this, but there are a number of things that can catch you out.

  1. Patch the loggrep plugin: Loggrep, by default, seems to mis-report the host_name variable, displaying a file path to the plugin, you’ll see what I mean if you run munin-run loggrep_foo config. Fix this by editing the munin loggrep plugin, which can usually be found in /usr/share/munin/plugins/, and change the line:

    (my $host_name = $0) =~ s|.*_([^_]+)_.*|$1|;

    to

    (my $host_name = $name) =~ s|.*_([^_]+)_.*|$1|;

  2. Check your config: Check your loggrep configuration… the plugin will silently fail if there are duplicate labels in a configuration section, or if it is missing any of the required environment variables. So, for example, if you’ve got two env.label_foo definitions defined either in the same section or in a section that overrides it, you will have problems.
  3. Check your plugin names: For some reason, probably related to the hostname regexp, when you create your symlink in /etc/munin/plugins/ names like loggrep_foo will work, but names containing two underscores like loggrep_foo_bar will not.

This got it working for me, but of course YMMV. Hopefully, if you’re having the same problem, this’ll save you a few hours hair pulling!