Tag Archives: munin

Posted on
by

In a previous post I wrote about a little bit of software which let you extract information from the output of the Weather Display weather station software.

Lately, I’ve been playing around with munin a fair bit, so I thought it’d be pretty neat to combine the two, and have munin keep an eye on my local airfield and monitor the winds, pressure and temperature. So, I had a quick skim of the plugin writing tutorial on their wiki, and bashed together some munin plugins.

Writing plugins for munin is actually remarkably straightforward, and very quickly my computer began generating some very pretty graphs…

wdwind_www_enstoneaerodrome_co_uk-day

wdtemp_www_enstoneaerodrome_co_uk-day

wdpressure_www_enstoneaerodrome_co_uk-day

* The outside temperature sensors at Enstone seems to have run out of batteries, hence the rather odd reading.

Hope this is useful to someone, code in the usual place!

» Visit the project on Github…

Posted on
by

The server-status page gives you a wealth of information about your apache server, and among other things it is necessary for the Apache munin plugin to work. However, by default it exposes sensitive data when run behind a squid reverse proxy.

In order to lock this down, you need to modify your /etc/squid/squid.conf file slightly…

acl no_stats urlpath_regex /server-status
http_access deny no_stats

This defines an ACL that will prevent squid from giving access to any request ending in /server-status. Because, in a reverse proxy configuration, it is squid that is hit when a client request a page from port 80, this prevents public access.

You’re not quite done though, because this setting assumes that your real web server (sitting behind the squid proxy on a different port) is NOT publicly visible. This can be achieved by placing it behind the firewall.

Munin

If you’re using munin to monitor your apache processes, you’ll need to make a small modification to the /etc/munin/plugin-conf.d/munin-node file, add/modify the following:

[apache_*]
env.url http://127.0.0.1:%d/server-status?auto
env.ports [apache server port]

This will tell the apache plugins where the server status page is, and what port the real apache instance is running on.

Posted on
by

MuninMunin is a powerful and highly customisable network monitoring tool. It lets you collect data about pretty much anything, from the number of apache processes running, to the number of IP addresses blocked by fail2ban, to the current temperature of your CPU.

All this data is collected and graphed over time, and provides you with a powerful insight into the performance of your infrastructure, and helps track down the root cause of any failure or outage.

I monitor just about anything I can, and one of the many things that I graph are the number of PHP errors appearing in my apache logs. This gives me an idea of the overall health of some of the platforms that I develop, as well as spotting instances where I introduce a regression which would otherwise not be spotted.

Powering this, I use a munin plugin called loggrep, which, as the name implies, works by grepping a log file for a certain bit of regular expression and returning a count of every instance found. This is simple but very effective, however, it took a little bit of fiddling to get it working.

The symptoms

You install munin, and all the other plugins are present and generating graphs, but those generated by loggrep are missing. You follow the munin troubleshooting steps and you find that while you can connect to to the node and the plugin appears to be running and collecting data:

host #: telnet localhost 4949
# munin node at host
fetch loggrep_foo
count.value 35
errors.value 12
.

The plugin does not appear in the list when you telnet in and type “list“.

The fix

After a lot of hacking about, I managed to fix this, but there are a number of things that can catch you out.

  1. Patch the loggrep plugin: Loggrep, by default, seems to mis-report the host_name variable, displaying a file path to the plugin, you’ll see what I mean if you run munin-run loggrep_foo config. Fix this by editing the munin loggrep plugin, which can usually be found in /usr/share/munin/plugins/, and change the line:

    (my $host_name = $0) =~ s|.*_([^_]+)_.*|$1|;

    to

    (my $host_name = $name) =~ s|.*_([^_]+)_.*|$1|;

  2. Check your config: Check your loggrep configuration… the plugin will silently fail if there are duplicate labels in a configuration section, or if it is missing any of the required environment variables. So, for example, if you’ve got two env.label_foo definitions defined either in the same section or in a section that overrides it, you will have problems.
  3. Check your plugin names: For some reason, probably related to the hostname regexp, when you create your symlink in /etc/munin/plugins/ names like loggrep_foo will work, but names containing two underscores like loggrep_foo_bar will not.

This got it working for me, but of course YMMV. Hopefully, if you’re having the same problem, this’ll save you a few hours hair pulling!