Posted on
by

Helping out a friend and colleague, as well as stretching my programming muscles with a language I don’t often get to play with these days, I’d like to introduce LoveNote Server.

LoveNote is a simple abstract message queue server which lets you pass an message payload to one of a pool of endpoints and receive a webhook callback with the result. You can specify that this message be delivered ASAP, but crucially you can also specify a date and time for the delivery.

How it works

It works by POSTing a bit of JSON to a webhook provided by the server that contains a delivery time, an array of servers to try, the payload and an optional callback URL.

When received by the server, the message is queued. When the delivery time is reached the list of servers is randomised and the payload POSTed in sequence until all servers fail, or delivery is achieved. If a callback is specified, a report is then POSTed back to the callback as a JSON blob.

Why this is useful

Simply, this provides a common message passing framework with a unified event driven API, simplifying your architecture somewhat. It is especially handy if you wanted a message to be delivered some time in the future, for example for a credit card renewal or email reminder, where beforehand you’d probably have to write a dedicated server process.

All you now have to do is listen to webhook pings.

What still needs to be done

This is an early version and was written to help out a friend with a specific need, and although I’ve gone on to use it in a couple of client projects, there are still a fair amount of enhancements that could be made.

Some obvious ones are:

  • Message IDs: Currently messages in the queue are anonymous. It’d be handy to have message IDs since this would allow more sophisticated process control of scheduled messages.
  • Cancel Control: Basic message control to cancel future queued messages.
  • Make message queues persistent: Currently the queue is held in memory, which is simple and fast, but far from ideal. We should periodically flush the queue to a persistent storage so that no messages are lost if a server goes down.

Get involved and send your pull requests to the usual place!

» Visit the project on Github…

Posted on
by

StatsD, created by Etsy, is a simple Node.JS server that provides powerful way to collect numerous statistics about your web application, and to do so simply and quickly.

At Etsy, they graph everything, which they use to great effect.

Collecting statistics is cool because it gives you hard data about how your software is performing. This in turn gives you powerful analytical tools to dig deep into your code, and find out what’s really going on (especially true when combined with a visualisation tool such as Graphite). It let you see the impact of code or infrastructure changes, and to quickly identify problems. Perhaps most importantly, it is only when armed kind of hard data that you can even begin to grasp at the nettles of code optimisation, growth and scalability.

Introducing StatsD for Elgg

With this in mind, and because I needed to collect some stats for a couple of client projects, I’ve put up on Github the first version of an Elgg statsD module. When installed and configured, this module will interface with a statsd server and collect a whole bunch of statistics from your running Elgg site.

Out of the box the current version of the plugin can log:

  • Events & Hooks (which in turn give you things like user signup events and object creation)
  • Exceptions
  • PHP Errors, Notices and Warnings
  • Elgg popups (system messages and error messages)
  • Database calls
  • Script execution time

In addition you can record your own statistics by making a call to wrapper functions contained in the plugin itself.

All data will be logged into a custom “bucket”, which is by default derived from your Elgg site name. This lets you log statistics from multiple different sites to a common statsd server.

Installation is pretty straightforward once you’ve installed the base infrastructure. Follow the instructions for installing graphite, nodeJS and statsd from the various sites around the internet, and then upload the elgg-statsd plugin to your Elgg site’s mod directory.

Once activated, you can specify the statsd server you wish to log to and configure what statistics you want to record.

Have fun!

» Visit the project on Github…

P.S. If you try and set this up and are seeing errors in your graphite log along the lines of “create() takes at most 5 arguments (6 given)“, then you are likely falling foul of this bug.

My solution was to build Whisper from the latest code in master rather than the stable 0.9.x branch. This worked for me, but of course YMMV.

Posted on
by

iCal is a simple format for sharing calendar, event and todo list information between applications. It is widely supported by many many applications.

I have recently been hacking on a couple of projects which required me to implement the format, and during this I hit on a couple of less well documented gotchas.

So, I’ve jotted them down here in order to hopefully save you some time…

  • Line endings: The specification says that line endings should be CRLF only, and some less tolerant clients puke if they’re not.
  • No blank lines: Ensure that your output contains no blank lines, again some clients with puke or interpret these as the end of the file.
  • End in .ics (the biggy): This is important if you dynamically generate an iCal feed for use with Google Calendar. You should set header type appropriately, e.g.

    header("Content-Type: text/calendar");
    header('Content-Disposition: attachment; filename="calendar.ics"');

    But critically, if all your entries are showing up only as “Busy”, you must also end the URL you pass to Google calendar as http://foo.com/bar/whatever.ics

You may find one of these validator tools useful too.

Hope that helps!

Image “Calendar Logo” by Quadmod