Proxy golang web server with relayd on OpenBSD 5.7

If you are new to OpenBSD, you may wonder why the httpd daemon that in the base install does not provide the ability to proxy web requests. The reason is that this capability is already provided in base by another daemon called relayd.

This post gives you a quick recipe for how to setup relayd to proxy http requests to a golang web server running as a non-priviledged user. Note that this barely scratches the surface of what relayd can do; for more info, you can read

• Basic relayd Config (cont)
• Recent work in OpenBSD relayd
• Relayd: a load balancer for OpenBSD - SlideShare
• relayd/etc/relayd.conf
• relayd(8)
• relay.conf(5)

Steps:

1. Add a new user, accepting defaults.
2. As that user, start up golang web server, listening on port 8080.
   

   $ cat > srv.go
   package main
   
   import (
           "fmt"
           "log"
           "net/http"
           "time"
   )
   
   func withalog(h http.Handler) http.Handler {
           f := func(w http.ResponseWriter, r *http.Request) {
                   fmt.Printf(
                           "%s - - [%s] \"%s %s %s\"\n",
                           r.Header["X-Forwarded-For"],
                           //r.RemoteAddr,
                           time.Now().Format("02/Jan/2006 03:04:05 EST"),
                           r.Method,
                           r.URL,
                           r.Proto)
                   h.ServeHTTP(w, r)
           }
           return http.HandlerFunc(f)
   }
   
   func main() {
           handler := http.FileServer(http.Dir("/usr/share/doc"))
           log.Fatal(http.ListenAndServe(":8080", withalog(handler)))
   }
   ^D
   $ go build srv.go
   $ ./srv &
   $ curl http://127.0.0.1:8080/../..
   [] - - [04/Jul/2015 10:20:21 EST] "GET / HTTP/1.1"
   <pre>
   <a href="mg/">mg/
   </pre>
   $
   

3. Configure relayd to forward 80 to 8080
   

   # cat > /etc/relayd.conf
   http protocol "littleproto" {
           return error
           match request header append "X-Forwarded-For" \
               value "$REMOTE_ADDR"
   }
   
   relay myproxy {
           listen on 192.168.30.6 port 80
           protocol "littleproto"
           forward to 127.0.0.1 port 8080
   }
   ^D
   # relayd -n
   configuration OK
   #
   

4. Set it to start on reboot.
   

   # echo relayd_flags="" >> /etc/rc.conf.local
   #
   

5. Start it and test.

   # relayd
   # curl http://192.168.30.6
   <pre>
   <a href="mg/">mg/
   </pre>
   

Questions

What does return error do in http protocol?

Tells relayd to return an error pages. If you don't specify the style option (see man page), this is the default styling:

If I just want a http reverse proxy, do I need to configure PF?

No.

What's the differences between relay httpproxy {... and redirect www {... , both of which are in /etc/examples/relayd.conf.

Not 100% sure. From reading relay.conf(5), it says that a redirection does stateful forwarding, and a relay is for general purpose TCP proxy.

Your own private git remote

I have some documents (writing, financial data, some code) that I want to keep private. This describes how to setup a git remote repository on your home network, and to back it up to the cloud (daily) with tarsnap.

Definitions:

office

The host that holds your remote git repo.

lap

The computer where you do your programming.

Here are the steps:

1. Create a git-private user on office. On OpenBSD, that is
   adduser git-private
2. Create ssh key-pair on lap
   

   mark@lap:~$ ssh-keygen
   Generating public/private rsa key pair.
   Enter file in which to save the key (/home/mark/.ssh/id_rsa): /home/mark/.ssh/id_rsa_git-private
   Enter passphrase (empty for no passphrase): 
   Enter same passphrase again: 
   Your identification has been saved in /home/mark/.ssh/id_rsa_git-private.
   Your public key has been saved in /home/mark/.ssh/id_rsa_git-private.pub.
   The key fingerprint is:
   cc:c7:c8:ab:36:9b:1a:d1:61:d1:2f:9a:06:4b:97:16 mark@lap
   The key's randomart image is:
   +--[ RSA 2048]----+
   |      ..         |
   |      E..        |
   |      oo .       |
   |    oo+=.o.      |
   |   ..=.oS.o      |
   |    ..+  o       |
   |    ..  .        |
   |     .oo         |
   |    .o+o         |
   +-----------------+
   mark@lap:~$
   

3. Authorize key.
   scp ~/.ssh/id_rsa_git-private.pub git-private@office:~/.ssh/authorized_keys
4. On lap, associate private key with office
   

   cat >> ~/.ssh/config
   Host            office
    Hostname office
    IdentityFile    ~/.ssh/id_rsa_git-private
    User            git-private
   ^D
   

5. Set up bare remote git repo on office.
   

   mark@lap:~$ ssh git-private@office
   git-private@office:~$ git init --bare myrepo.git
   

6. On lap, add remove repo as remote.
   

   mark@lap:~$ git remove -v
   mark@lap:~$ git remote add origin git-private@prod:/home/git-private/myrepo.git
   mark@lap:~$ git push -u origin all
   

7. To securely backup encrypted versions of your remove, see my Recipe for setting up backups with tarsnap (OpenBSD 5.7).

Recipe for setting up backups with tarsnap (OpenBSD 5.7)

Tarsnap is a wicked smart backup service. (The author started studying math at university at age 13!) It only saves the deltas since your last backup, which means you can always do a full backup but you use much less space. Tarsnap (somehow) stores enough info that it makes every backup act like a full backup; for example, even if you delete all previous backups the last one will still be a full.

It provides you a simple command-line client (which makes it easy to script in cron), is secure (Hi, NSA!) and costs a bit more than the base Amazon S3 service it uses to store your files.

For the canonical directions, see http://www.tarsnap.com/gettingstarted.html.

Create tarsnap account and install client

1. Create your account: https://www.tarsnap.com/register.cgi
2. Click on the link in the "Tarsnap registration confirmation" email
3. Compile the tarsnap client (you must compile)

   # ftp https://www.tarsnap.com/download/tarsnap-autoconf-1.0.35.tgz
   # tar xzvf tarsnap-autoconf-1.0.35.tgz
   # cd tarsnap-autoconf-1.0.35
   # ./configure 
   ...
   # make all install clean
   

4. Login to your account at https://www.tarsnap.com/manage.cgi, then add money:
   • Click "Add funds to your account" link
   • Enter a payment amount of $5 (he shows account balances to 18 decimal places ... in one day, my balance went from $5.000000000000000000 to $4.986442789553223132. :)
   • Enter a credit card or use PayPal.
   • In a few minutes, you will get a "Tarsnap payment processed" confirmation email.

Create tarsnap key and make your first backup

Note: KEEP YOUR TARSNAP KEY. If you lose it, you cannot get your backed up data. Yes, that is correct.

What I did was encrypt it using my keybase.io account and then copied that encrypted file around to a bunch of different places.

5. $ tarsnap-keygen --keyfile /root/tarsnap.key --user mkbucc@gexample.com --machine office
6. (From another host ...)

   $ scp office:/root/tarsnap.key .
   $ brew install keybase
   $ keybase login
   Your keybase username or email: markb
   Your keybase login passphrase: ***********************************
   run scrypt [----------------------------------]
   info: Made directory '/Users/mark/.cache/keybase'
   info: Updated file: /Users/mark/.cache/keybase/session.json
   info: Updated file: /Users/mark/.config/keybase/config.json
   info: Creating temporary keyring dir: /Users/mark/.cache/keybase/tmp_keyrings
   $ keybase encrypt markb tarsnap.key
   $ scp tarsnap.key.asc 
   $ cp tarsnap.key.asc ~/Documents
   $ keybase logout
   info: Removing file: /Users/mark/.cache/keybase/session.json
   $
   

7. Ok, back on server ... create the first backup.

   # mkdir /var/cache/tarsnap
   # cat > /usr/local/bin/backup
   #! /bin/sh -e
   
   /usr/local/bin/tarsnap --keyfile /root/tarsnap.key  --cachedir /var/cache/tarsnap -c  -f $(hostname)-$(date +%s) /home
   ^D
   $ chmod +x  /usr/local/bin/backup
   $ backup
   ... wait, first one takes a while ...
   $ du -sh /var/cache/tarsnap
   196K    /var/cache/tarsnap
   $
   

8. Wrapper script to list backups.

   $ cat > /usr/local/bin/backuplist
   #! /bin/sh -e
   
   /usr/local/bin/tarsnap --keyfile /root/tarsnap.key --list-archives | sort
   ^D
   $ chmod +x /usr/local/bin/backuplist
   # backuplist 
   office-1435712869
   #
   

9. Finally, add to daily cron job

   # cat >> /etc/daily.local
   #! /bin/sh
   
   /usr/local/bin/backup
   ^D
   #
   

It took me longer to write up this blog entry than it did to set this up. Literally.

Enjoy!

OSX, Microsoft Intellipoint Mouse (Explorer Touch) and Acme editor

Doh!

My last post was about trying to get a three-button mouse that works in OSX for the acme editor. I thought I had to use xwindows.

But it turns out, all I had to do was configure the mouse to have OSX handle the middle click. Like so:

On the Microsoft Explorer Touch mouse, you also need to press the middle button more towards the back of the mouse than towards the front to get that middle click to fire.

This is the mouse I am thinking about getting for work, where I do use Linux and so will use i3 and X-Windows, so that work is not all for naught.