Category Archives: Software

Posted on
by

URL unfurling is one of the names given to that funky thing that happens when you paste a web address into a post in Facebook (or other social network) and automatically get a preview of it – an image, the title and maybe some extract text.

This was a much requested technology that was sadly missing from Known for a long time, but for no longer!

In the latest builds you will automatically get URL unfurling occurring in status posts and likes/bookmarks. What’s more, you’ll get a URL preview when you’re editing your post.

Behind the scenes this tool makes use of a number of technologies, notably:

The Unfurling endpoint

This is an endpoint which is called by a client side javascript library.

When passed a URL, this endpoint will attempt to fetch and parse out header tags – title, open graph, facebook and twitter tags. It’ll also attempt to extract certain whitelisted OEmbed endpoints (currently only JSON endpoints are supported).

It will then render out in a pretty way – using oembed as preference, but if that’s not present (or not whitelisted) it’ll use Open Graph, and finally page title and description meta tags if nothing else is found.

Image proxy

If there is an image present in the open graph headers, this will be retrieved by a local caching image proxy. This proxy fetches and saves the image locally so that the remote site doesn’t get hit every time you refresh your page (this also helps protect your privacy).

Enjoy!

Posted on
by

Writing good software is hard, and often things go wrong in unexpected ways when software is deployed to live. A mantra that I’ve been trying to live by is to “never rely on your users/clients/customers reporting problems”, if anything this should be the absolute last thing to rely on.

I’ve previously talked about how I have been deploying metrics and fault reporting code for all my clients, regardless of what software they’re running, and how I’ve built support for these into Known core.

These reports produce a detailed insight into the code as it runs under real usage, as well as a detailed bug log should something fail. The fault reporting messages in particular have possibly been one of the most useful things I’ve ever done, and have already been responsible for discovering a number of rare failure modes on software that had thus far neither shown up in testing or had been reported.

Today I had a particularly thorny Javascript issue to debug, and it got me thinking about how I could capture javascript errors from client browsers, have errors logged in a central way, and even get metric and fault reporting as I currently do with PHP errors.

Turns out it’s actually quite easy, and in a nutshell you need to:

  • Write an endpoint which will log your error message in the appropriate way, sending a fault report as necessary
  • Write a bit of javascript code to call this endpoint, and listen to an error event, e.g.
window.addEventListener('error', function (error) { 
    var stack = error.error.stack;
    var message = error.error.toString();

    if (stack) 
    message += '\n' + stack;
    
    ... call your ajax endpoint...
});

The latest build of Known now has support for this: now, Javascript messages will get logged in your normal logs, metrics counted, and if you’ve enabled oops reporting, you’ll get an email when a client triggers a Javascript error.

Hopefully you’ll find this as useful as I have!

Posted on
by

Further to my last post, I’d like to introduce you to a working implementation of the stats gathering mechanism using Etsy StatsD and NodeJS.

StatsD is a Node.JS stats server created by the people at etsy to provide a simple way of logging useful statistics from software. These statistics are an invaluable way of monitoring the performance of your application, monitoring the performance of software changes and diagnosing faults.

This plugin gives you an overview of what is happening in your Known install by logging important system level things – events, errors, exceptions etc. This lets you get a very clear idea of how your Known network is performing, and quickly see the effect that changes have on your users.

Installation

  • Install Node.JS, either from github or the package manager for your OS
  • Install StatsD
  • Not required, but highly recommended, install a Graphite server for graph visualisation
  • Place this plugin in IdnoPlugins/StatsD
  • Add the following to your “`config.ini“` 
    statistics_collector = IdnoPlugins\StatsD\StatsDStatisticsCollector;
statsd_enabled = true;

Optionally, you can specify one or more of the following extra options, (although the defaults are usually ok):

statsd_host = localhost
statsd_port = 8125
statsd_bucket = some_name
statsd_samplerate = 1

statsd_samplerate is handy on really busy systems (see Statsd’s notes on the subject), but in a nutshell, setting this to something like 0.1 (capture one in every 10 count or timer events) is handy if you find StatsD being overloaded.

If everything is working, you should now be happily graphing some useful stats.

» Visit the project on Github...