I have previously talked about speeding up your site by using Squid as a reverse proxy to cache served pages. This is a great thing to do, but presents a problem now that all sites are moving over to HTTPS, since for various technical reasons reverse proxies can’t really handle HTTPS.

These days the standard way of doing this seems to be using Varnish as a cache, and Squid seems to be a little “old hat”, however I have several client estates which were set up before Varnish came on the scene, so I needed a solution I could get up and running very quickly.

Terminating HTTPS

Thankfully, the solution is very similar whatever reverse proxy you’re using. The solution is simple, you need to install something that terminates and handles the HTTPS session before sending it to your proxy. The simplest way to do this is to install NGINX and configure it to handle HTTPS sessions.

1) Disable Apache’s handling of HTTPS (if you’ve got an existing, un-cached, HTTPS server).

2) Install the basic nginx apt-get install nginx-light

3) Configure nginx to forward to your proxy (which you have previously configured to listen on port 80)

server {
        listen 443 ssl;
        server_name www.example.com;
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

        location / {
            proxy_pass http://127.0.0.1:80;
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header X-Forwarded-Port 443;
            proxy_set_header Host $host;
        }
}

After restarting nginx, you should be able to see https requests coming in on your squid proxy logs.

Gotchas

The biggest gotcha that you’re going to hit is that if you’re checking whether a request is HTTPS in your app (e.g. for automatically forwarding from insecure to secure), you’re not going to be able to use the standard protocol checks. The reason being is that HTTPS is being terminated by nginx, so by the time the session hits your app, it will not be seen as secure!

To perform such a test, you’re instead going to have to check for the X-Forwarded-Proto header instead ($_SERVER['HTTP_X_FORWARDED_PROTO'] in PHP).

Tor (which stands for The Onion Router), is a powerful anonymity service originally developed by the US Navy, which helps protect citizens around the world from abuse and monitoring. A VPN also offers the same service, if you’re not familiar with the tool, you can read about the VPN meaning here.

Most people use Tor via the Tor Browser, which simplifies setup, and I encourage you to use that where you can. This article discusses going one step further, and routing all traffic through Tor via the use of a HTTP proxy.

In the UK, the passage of the disastrous #IPBill places everyone under suspicionless surveillance, and I have client confidentiality to consider. So, as a matter of due diligence, I wanted to ensure that when my ISP’s surveillance database was inevitably hacked, the information the Russian Mafia got was of limited use.

This was easy enough to set up.

Install and configure Tor

The first step is to install Tor; not the browser, but the software the browser talks to in order to make it’s connection. On Debian based systems, apt-get install tor.

Tor comes with a SOCKS proxy, so enable support by editing /etc/tor/torrc and uncomment the line:

SocksPort 9050

If this is a network server, you may want to enable an external proxy on your network as well (for example, I have an always on Raspberry Pi running a tor proxy for all the various iOS devices on my home network).

Chain a simple HTTP Proxy

Once that’s done, you’ll have a SOCKS proxy up and running that’ll route anything it gets through Tor. Many things (e.g. the aforementioned iOS devices) won’t talk SOCKS. To solve this, I use a light weight HTTP proxy called polipo to create a HTTP proxy wrapper for Tor’s SOCKS proxy.

Install polipo: apt-get install polipo

Then configure the proxy to chain to Tor’s SOCKS proxy, modify /etc/polipo/config:

...

allowedClients = 127.0.0.1, 192.168.1.0/24 # Expose your network (modify accordingly)

socksParentProxy = "localhost:9050"
socksProxyType = socks5

proxyAddress = "0.0.0.0"    # IPv4 only

...

On my Raspberry Pi, I also disable the caching by adding diskCacheRoot = "" to the config, as this prevents polipo from filling up the SD card and breaking the proxy.

Restart both tor and polipo, and now you should have both a tor SOCKS and HTTP proxy.

Change your browser settings

Finally, you need to configure your browser (or your entire system) to use this proxy. This is different depending on what you’re using, but on Ubuntu you can set global proxy settings in your system settings, which will route all traffic over your new proxy (default port 8123).

iOS devices have a per-network proxy configuration found in your wireless network configuration (click on the little “i” icon).

Verify everything is working by visiting check.torproject.org.

Tor is a powerful anonymity tool that lets people communicate and share ideas without disclosing who they are or what they’re viewing to prying eyes.

One cool thing that the network lets you do is create a site as a hidden service, which is a service accessible only from within the tor network, is end to end encrypted and can be viewed entirely anonymously via a .onion domain.

I thought it could be cool to be able to run Known as a hidden service, and potentially build an entirely distributed social network, one that resists metadata surveillance as well as content surveillance. Anyway, baby steps…

Install Tor

The first thing you need to do is install tor. I don’t mean the tor browser, although you’ll need that to visit your hidden service, I mean the underlying tor program itself. On a debian based network this is really easy:

apt-get install tor

Next, you need to configure your hidden service(s) by editing the configuration in /etc/tor/torrc:

HiddenServiceDir /var/lib/tor/hidden_service/
HiddenServicePort 80 127.0.0.1:80
HiddenServicePort 443 127.0.0.1:443

Restart tor and you will find a file containing your .onion domain name in /var/lib/tor/hidden_service/hostname. More details on configuring tor hidden services can be found on the tor project website.

Install Known

Download and install Known in the usual way.

My setup for my test onion site is Apache2 + modules + MariaDB (because I wanted to try Maria out). You of course can run nginx with mysql or mongodb, it won’t matter too much for tor.

To run through the Known installation procedure, you’ll need to visit your site via the tor browser. Find the name of your hidden service in /var/lib/tor/hidden_service/hostname (which will look something like xa7txi2q5okg36f6.onion), fire up your tor browser and visit that URL. All being well, you should see the Known installation.

Configuring Known to use tor

By default, Known will attempt to connect to web services and send webmentions directly. This will usually work if the machine you’re running your hidden service on is also directly connected to the internet, however you should consider routing this through tor since this communication could unmask your service.

So, you need to configure tor as a proxy, and then configure Known to use it.

First, edit torrc and add/uncomment the line:

SocksPort 9050

Next, configure Known’s web services to use it. Add the following to your Known config.ini file:

proxy_string = 'localhost:9050'
proxy_type = 'socks5-hostname'
disable_ssl_verify = 'yes'

These commands tell Known’s webservice calls to use the tor proxy to connect to the internet, and to use tor for DNS resolution in order to get access to .onion domains. This is important so that you are able to, among other things, send webmentions to other .onion sites from within the tor network. The last line disables ssl verification for HTTPS certificates (see the note below).

Note, for these settings to work you need to be running a version of Known that includes my curl settings patch.

Restart tor, and now all web service calls (which use Known’s web service library) should be sent via your tor proxy.

Known issues

There are some things you will need to consider when running Known as a hidden service:

  • The default PuSH hub is the public one at https://withknown.superfeedr.com/. This may cause unacceptable leakage of activity, so you may want to consider leaving this blank or using the Known native PuSH hub.
  • For the same reason, you may want to consider turning off emails, or sending emails through something like Mixmaster.
  • Only things that use Known’s web services libraries will use the proxy. Third party plugins (especially syndication plugins which use third party libraries, e.g. Facebook) will still try to connect directly. You may consider not using these plugins.
  • Currently there are no valid TLS certificates available for .onion domains, so you have to self sign if you want HTTPS. However, a .onion address is end to end encrypted in any case and there is some lively discussion about whether certificate authority based HTTPS is necessary or even desirable for hidden services. That you can’t get valid SSL certs for hidden services is a known problem (as opposed to a Known problem), so it’ll be resolved eventually one way or another.

Anyway, happy hacking!