Posted on
by

This is just a quick post to nudge you towards a little plugin I wrote for Known which enforces a minimum password strength for user passwords.

The plugin works by calculating the entropy of the password based on NIST recommendations, and rejecting passwords where the entropy is too low.

By default, the minimum entropy is 44, however this can be changed through a configuration setting.

For this plugin to work, until my pull request is merged into the core code, you’ll need to apply patches available from my password validation branch.

Anyway, give it a kick about!

» Visit the project on Github...

Image “Password Strength” by XKCD

Posted on
by

The majority of web servers retain a vast amount of data about their visitors in the form of log files. Other processes running on the server, like the system log, MTA log, etc, also store a raft of information.

These logs are typically retained (although often rotated at regular intervals to save space) basically until the admin is looking to reclaim some disk space or the server is reinstalled, so, from a practical standpoint that’s “forever”. This is very much part of the tech industry’s dataholic “collect everything” culture, which I’m personally trying to wean myself off of.

Thing is, at first glance, retention seems like such a good idea (and limited retention can be, more on that later). You need logs to find out how your server is performing, and what if something goes wrong? However, they’re mostly just noise, and they go stale very quickly… when was the last time you needed to look at a 4 month old apache log file?

The reality is that the vast majority of the time you’re only really interested in the last couple of lines. Why keep the rest?

What question are you trying to answer?

Log files have there use; they are invaluable to diagnose specific and immediate problems along the lines of “My web site keeps giving me a white page!”, or “Why on earth won’t my firewall start?”, or “What was the last thing Apache did before it crashed?”.

However, to answer the perhaps more useful questions like, “Am I seeing increased traffic?” or “Are my hard drives healthy?”, or even esoteric questions like, “Did spring cleaning my server save me money?“, your raw logs really aren’t going to be much use to you.

To answer the questions you’re really interested in, you’re going to have to cook this data into something tasty.

What I do…

This is the approach I’m currently using for myself, and which I been recommending to my clients. Obviously you need to adjust this based on specific requirements, for example, one client I had in the past had a legal requirement to retain all logs off line (of course nobody ever looked at them but rules is rules).

  1. Retain raw logs for a day: keep your raw logs for a short period of time, this will let you get at the raw text of any error messages should anything on your server die.
  2. Run an infrastructure monitoring tool: instead of keeping raw logs, what you should be keeping is the higher level statistical information that is produced by analysing your logs (and other sources) produced by a tool like munin. These results have all the noise (and any sensitive information) removed, and are far better at helping you diagnose problems.

Using this approach I was in the past able to, among many other things:

  • Spot a failing hard drive on a customer’s server before it became a problem (because over time the frequency of errors on that specific drive was increasing).
  • Optimise caches within a feedback loop (I could track configuration changes with a corresponding increase or decrease in cached pages served).
  • Isolate the cause of an intermittent failure on a client site (by seeing what the server was doing at the time of the outage, I could see that the mysql query cache was becoming full causing queries to run slowly and apache to block).
  • Link an increasing number of errors back to a configuration change made months ago (I had logged the time and date of the config change, and could look back at my graphs to see that I first started seeing problems after this time. Reverted the change and everything was a-ok).
  • …etc…

In each case the information was in the raw logs, but good luck trying to find it.

There are many tools out there that can help you, but the basic principle is the same – process your logs into a more usable statistical form from which it’s easy to gain insights from, and ditch the unnecessary raw logs which are mostly noise.

Posted on
by

I have a lot of devices connected to my home network, and like most modern computers, they automatically keep their system clocks correct by use of the Network Time Protocol (ntp), to connect, over the internet, to a very accurate central time server.

Many of the devices on my network phoned home to pre-set servers operated by their manufacturers – Ubuntu machines phoned Canonical servers, Macs and iOS devices phoned Apple, Windows phoned Microsoft. Since, in this post Snowden era we live in, we know for a fact that if something can be used to track you it will, and since I’ve been working to make my network as non-leaky as possible, by stopping all but authorised outgoing network connections, this seemed like something I should change.

My home server itself provides NTP, so there was no need for any other clients to talk to NTP servers elsewhere on the internet. Great, although it presents us with a number of problems:

  1. There are a lot of devices, and I’m lazy, so I don’t want to go around reconfiguring each device.
  2. While some devices allow you to easily change the timeserver, there are a bunch that don’t.
  3. I also want devices to be able to continue syncing themselves when connected to other networks.
  4. It would be nice if visitors to my network were afforded the same protection!

Remapping using Shorewall

Thankfully, I have shorewall running on a Linux box between my network and the wider internet, and also thankfully, NTP is fairly identifiable as a protocol, as it operates over UDP port 123.

So, using DNAT, I was able to remap any request to an outgoing NTP server on UDP port 123 from the network to my local server, using this line towards the end of my /etc/shorewall/rules file.

...

DNAT            loc:!192.168.1.1         loc:192.168.1.1 udp     123

...

This has the effect of redirecting all NTP requests from the network, transparently, to my server, regardless of what servers the client believes it is talking to.

To verify, we can use the tool ntpdate -d to connect to a time server on a bogus, but still technically valid, IP address, e.g. 1.2.3.4 (the -d puts it in debug mode, so we won’t reset any clocks). The bogus IP will be transparently remapped, and we should get a lock.

Before remapping…

24 Mar 16:22:30 ntpdate[13715]: ntpdate 4.2.6p3@1.2290-o Tue Jun  5 20:12:09 UTC 2012 (1)
Looking for host 1.2.3.4 and service ntp
host found : 1.2.3.4
transmit(1.2.3.4)
transmit(1.2.3.4)
transmit(1.2.3.4)
transmit(1.2.3.4)
transmit(1.2.3.4)
1.2.3.4: Server dropped: no data
server 1.2.3.4, port 123
stratum 0, precision 0, leap 00, trust 000
refid [1.2.3.4], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  0:00:00.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  0:00:00.000
transmit timestamp:  d6dada51.643e780f  Mon, Mar 24 2014 16:22:41.391
filter delay:  0.00000  0.00000  0.00000  0.00000 
         0.00000  0.00000  0.00000  0.00000 
filter offset: 0.000000 0.000000 0.000000 0.000000
         0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

24 Mar 16:22:43 ntpdate[13715]: no server suitable for synchronization found

After remapping…

24 Mar 16:24:17 ntpdate[13736]: ntpdate 4.2.6p3@1.2290-o Tue Jun  5 20:12:09 UTC 2012 (1)
Looking for host 1.2.3.4 and service ntp
host found : 1.2.3.4
transmit(1.2.3.4)
receive(1.2.3.4)
transmit(1.2.3.4)
receive(1.2.3.4)
transmit(1.2.3.4)
receive(1.2.3.4)
transmit(1.2.3.4)
receive(1.2.3.4)
transmit(1.2.3.4)
server 1.2.3.4, port 123
stratum 3, precision -20, leap 00, trust 000
refid [1.2.3.4], delay 0.02592, dispersion 0.00000
transmitted 4, in filter 4
reference time:    d6dad7fc.de9a7567  Mon, Mar 24 2014 16:12:44.869
originate timestamp: d6dadabd.0543d5bb  Mon, Mar 24 2014 16:24:29.020
transmit timestamp:  d6dadabd.054a4033  Mon, Mar 24 2014 16:24:29.020
filter delay:  0.02597  0.02596  0.02594  0.02592 
         0.00000  0.00000  0.00000  0.00000 
filter offset: -0.00033 -0.00033 -0.00036 -0.00036
         0.000000 0.000000 0.000000 0.000000
delay 0.02592, dispersion 0.00000
offset -0.000361

24 Mar 16:24:31 ntpdate[13736]: adjust time server 1.2.3.4 offset -0.000361 sec

So, we can see that all network requests for ntp are now mapped to the local server on the quiet. A handy technique, and one I think I’ll start applying to some other protocols (I wonder if anyone has managed to reverse engineer icloud yet?).