Category Archives: Software

Posted on
by

Part of my day job is writing and maintaining a fairly massive and complex piece of software which has become mission critical for various large scientific infrastructures around Europe.

For reasons that will be familiar to anyone who’s built platforms that suddenly become successful, certain things were left out when building the software. One of these was any kind of monitoring.

We are of course building this out “properly”, however one of the simple things I did early on turned out to be a really big win. This was to simply catch all fatal errors and exceptions being thrown, and then pipe them to a slack channel set up for the purpose. With the help of clerk, this can be made easier.

Before adding the slack monitoring we’d often be surprised by error – receiving garbled reports third hand as they were escalated from a user email, through the administration team, and to us. By which time the detail has been lost, and any logs long since rotated away.

Our devs live in slack, and as a multinational team who’s members frequently travel, this has become the nervous system of the organisation. Now, my team is no longer surprised, and can jump on issues instantly.

Very very simple to set up, but turned out to be a big win. Here’s how

First, capture fatal errors in your application

I wrote about this before, about capturing WSOD errors, but assuming you’re using PHP, this is all about registering an exception and error handler for your application.

Create a slack app

Select “Incoming Webhooks” from application menu

Next, create a slack application and add it to a monitoring channel in your work workspace.

Add an “Incoming webhook” for your app, that posts to your monitoring channel. This will give you a URL, anything POSTed to which will end up in your channel.

Posted text supports Markdown for formatting, which is handy if you want to post raw error messages etc, or links to data dumps.

Link the two

Finally, from your error handler, POST the error / stack trace / etc to your slack channel.

Conclusion

Obviously, this should be no replacement for proper monitoring. Proper monitoring can provide historic information and statistics about the overall health of your platform.

However, in the absence of this… this may be a quick win that you can implement without too much effort. Certainly for us, this proved to be invaluable, and allowed us to quickly diagnose and fix faults we were previously unaware of.

Posted on
by

Just a quick one, you’ve been coding up your REST api, and are trying to use a Bearer authorization token (as obtained from an OAuth2 handshake), and it’s just not working.

If you send your access token as a GET or POST value things work fine however.

You point your head at httpbin.org and to see what your client is sending, and low and behold, the bearer token is present and correct.

You scratch your head and dump the contents of $_SERVER to a log, and to your surprise, nothing. No Authorization header is present!

To save you many a frustrating hour, here’s the answer. Turns out that Apache will strip any authorisation header it doesn’t recognise, which is basically anything that’s not basic auth.

So, you need to put it back in yourself. Do so by putting the following into your .htaccess

RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule .* - [e=HTTP_AUTHORIZATION:%1]

Posted on
by

As part of (one of) my day jobs, I have had to yet again bash together a set of REST APIs. This is so we can start wiring up some proper micro services AWS style scalable architecture into the monolithic beast that is the current incarnation of the software I’m working on.

Anyway, here are a few gotchas for this if you intend to start using proper REST style HTTP verbs (PUT/PATCH/DELETE), rather than doing everything via GET and POST like most everyone.

No easy way to access variables

If you’re familiar with the standard $_POST mechanism to access passed variables, you’ll be disappointed that PHP doesn’t by default provide a nice way of access these for PUT and PATCH.

So, you’re going to have to extract them yourself. Not overly tricky, but irritating:

So, for example:

parse_str(file_get_contents('php://input'), $patch);
if (is_array($patch)) {
  $this->data = array_merge($this->data, $patch);
}

Requests not coming through

If you find that your APIs work fine on your local machine but break when deployed, you might want to check your server configuration.

It is quite common for web servers (especially on shared hosts) to block access to HTTP verbs other than most common GET and POST. Modsecurity’s default config definitely blocks these methods.

You should also check that any proxies or load balancers that you have in front of your REST endpoint. These may need some configuration tweaks as well.

Hopefully this will save you some time and frustration!