On the Indiewebcamp wiki, there’s a page discussing HTTPS, the support for which is strongly recommended. As I’ve mentioned previously, at this stage all non-encrypted communication forms (including traditional port 80 HTTP) should be considered deprecated and dangerous.

Indieweb compatible sites are encouraged to get a higher level as possible, and thanks to some prodding, I’ve finally moved both this blog and my feed over to HTTPS only, with HSTS and forward secrecy.

This got me thinking, perhaps it would be worth adding a “Level 7” (or perhaps Level 6.5) to this, and to suggest that Indieweb sites should also be made available as .onion hidden services on Tor?

Pros

  • Anonymity. Would go a large way towards protecting communication metadata (who know’s whom), which is a goal we should move towards in a world of endemic selector based surveillance.
  • Encryption. Traffic within the tor network is end to end encrypted, and there is some discussion of whether this renders HTTPS unnecessary.

Cons

  • Tor has nothing to do with HTTPS, although it is encrypted. However, the HTTPS levels page seemed a good place to put the suggestion.
  • Could be seen as endorsing one service. Tor is Free software and is pretty much the only game in town when it comes to anonymity networks, but does that constitute a silo? Probably not, but is a point for discussion.
  • No certificates for .onion. There are currently no certificate providers available for .onion domains. But, this may not be a problem.

Anyway, just mooting this as a point for discussion.

I’ve previously documented how I’ve previously used Known to track system events generated by various pieces of hardware and various processes within my (and my client) infrastructure.

Like many, I use a UPS to keep my servers from uncontrolled shutdowns, and potential data loss, during the thankfully rare (but still more common than I’d like) power outages.

Both my UPS’ are made by APC, and are monitored by a small demon called apcupsd. This demon monitors the UPS and will report on its status, from obvious things like “lost power” and “power returned”, but also potentially more important things like “battery needs replacing”.

While some events do trigger an email, other messages are written to the console using the wall command, so are less useful on headless systems. Thankfully, modifying the script is fairly straightforward.

Setting up your script

First, set your script as you did for nagios. Create an account from within your Known install, and then grab its API key, then put it in a wrapping script.

#!/bin/bash

PATH=/path/to/BashKnown:"${PATH}"

status.sh https://my.status.server apc *YOURAPICODE* >/dev/null

exit 0

I need to Pee

The next step is to modify /etc/apccontrol to call your wrapper script.

I wanted to maintain the existing ‘post to wall’ functionality as well as posting to my status page. To do this, you need to replace the definition for WALL at the top of the script, and split the pipe between two executables.

To do this you need a command called pee, which is the pipe version of tee, and is available in the moreutils package on debian based systems. So, apt-get install moreutils.

Change:

WALL=wall

To:

WALL="pee 'wall' '/path/to/wrapperscript.sh'"

Testing

To test, you can run apccontrol directly, although obviously you should be careful which command you run, as some commands fire other scripts.

I tested by firing:

./apccontrol commfailure

Happy hacking!

Like many people out there who have servers, I use cross site backups for the purposes of disaster recovery.

While I also use “proper” backup software as well (which maintains a historic delta of changes, among other things), I’m a big fan of redundancy, so like many other server administrators out there, I also perform regular site to site copies using Rsync over SSH.

Rsync, for those who aren’t familiar, is a file copy tool, which, after the first copy, will only send changes during subsequent updates. This makes it a very efficient tool, especially when used over an internet connection. SSH stands for Secure Shell, and is an encrypted connection between two computers.

Anyway, to enable rsync from server A to server B, it is common to perform the login via key. This means that on Server A you’d generate a SSH keypair for your backup user, then copy the public key that was generated into the ~/.ssh/authorized_keys file for your backup user on Server B. There’s a good guide to doing that here.

Because rsync is going to be executed automatically via cron script, it is necessary to create the key file without a password.

Backups are running, so what’s the big?

Ok, so the key for your backup user isn’t password protected, but that doesn’t mean that anybody can log in as that user, they’d still need to have a copy of the cryptographic key.

If someone is able to break into Server A and steal your backup user’s key, they will be able to log in to Server B as that user. If that user is a non privileged user, that isn’t absolutely terrible (although they could, for example, read your /etc/passwd file, or anything else you’ve not locked down) and you can revoke this key at any time. However, since strength in depth and compartmentalisation are two concepts you should always remember when building a secure system, it’d be nice if it was possible to do something about it.

Thankfully there is.

Jail time

The solution is to configure Server B’s ssh client to place the backup user’s sessions into what’s called a “Jail” using a Linux tool called chroot. When the backup user logs in, they will not have access to any other parts of the system, other than what’s in their jail cell.

It’s a little fiddly to set up, but most of the hard work is done by SSH.

  • First, install chroot: “`apt-get install dchroot debootstrap“`
  • Configure your SSH server
    • Open up “`/etc/ssh/sshd_config“`
    • Make sure there’s a line that says
      Subsystem sftp internal-sftp
    • At the end of the file, tell SSH to create a chroot jail for your backup user:
      Match User backup-user
              ChrootDirectory /home/backup-user
              AllowTcpForwarding no
      

      Note, because of the way chroot works, you’ll need to make sure the chroot directory is owned by ROOT, even if it’s actually the home directory of your backup user.

    • Save, and restart your SSH server.

This gets you part of the way, you should now be able to SFTP into Server B using your backup user, and a normal SSH session will be refused. When connected, you will be restricted to the location set in ChrootDirectory.

Unfortunately, rsync needs more than this, and in order to copy files it’ll need access to the shell (I’m assuming bash), as well as the rsync application itself, together with whatever libraries are required.

Therefore, it becomes necessary to create a partial chroot image in the backup user’s chroot directory. You could do this the traditional way (e.g. by using something like debootstrap), which will create a mirror of your base operating system files in the chroot jail. However, this generally takes a few hundred megabytes at least, and if all you want is to copy some files, you don’t want to give access to more than you need.

Instead, I opted to create a skeleton chroot jail by hand.

The goal here is to mirror the filesystem of your server inside the chroot jail, so that if a file exists in /foo/bar, then you need to copy it to /home/backup-user/foo/bar, and make sure it’s owned by root.

  • Copy bash from “`/bin/bash“` to the directory “`/home/backup-user/bin/“`
  • Copy rsync (on my system this was in “`/usr/bin“`)
  • Next, you need to copy the symbolic link libraries to which these files are linked against. You can use the tool “`ldd“` to interrogate the executable and get a list of files to copy, e.g:
    root@server-b:/home/backup-user# ldd /bin/bash 
        linux-vdso.so.1 =>  (0x00007fff52bff000)
        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f412810a000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4127f06000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4127b79000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f4128340000)
    

    Copy the files which have directories into the appropriate locations, e.g. “`/lib/x86_64-linux-gnu/libtinfo.so.5“` should go into “`/home/backup-user/lib/x86_64-linux-gnu/“`

  • Do the same for “`/usr/bin/rsync“`

All being well, your rsync backups from Server A should now function as before, however the backup user will not be able to touch anything outside of the chroot jail.

Hope that’s useful to you!