About Me

I'm one of those people who genuinely loves technology.

As a kid I took things to bits to see how they worked - telephones a speciality. Thankfully, with time, "see" became "understand", "fix" became "improve" and taking an active role in technological innovation became a central theme of my life. This is what I do now...

I make new things, mostly in software.

In 2015 I started investigating the various container ecosystems and found SmartOS (an Open Solaris derivative) to be a great substrate for container deployment with terrible industry support. I ended up writing a lot of glue code, and eventually set about writing a "simplest possible" container infrastructure - including porting to Linux and AWS.

It turns out that this is quite a lot of work. Who knew?

Past engagements, projects and pastimes include voxel and particle databases for Weta Digital; using OpenGL to put the weather on TV; And a kids' gameshow; Winning a hackathon; Writing prosumer software for the mac; Twice; Being CTO in a venture funded startup; Some time obsessed with video compression; Some time obsessed with load balancing; Working as a Solution Architect; Running an outsourced development project; and at some point I probably had a job.

Tell me about

What's your current skillset?

A backend technology stack and decades of experience.

Where have you worked and what on?

Experience. It's not very ... linear.

Open Source

I open sourced some useful things.

A digital footprint

Rantings, ravings and the occasional fact.

Skillset

Python 3

I started with Python a few years ago for it's clarity, portability and the need for rapid iterations. I've spent the last year purely on Python, developing cross platform and remotely using PyCharm, and documenting with Sphinx. Specific experience includes Django, Bottle (microservices), boto3, zmq, multiprocessing, libnacl, and scripting Unix with a combination of subprocess and ptyprocess modules.

AWS

I've recently been using AWS programmatically through boto3 (Python). Specifically I've been working with EC2; IAM profiles; Security groups; CloudFormation; CloudWatch (logging); SSM; and Elastic File System. Some useful utilities have been spun out as open source.

Container Technology

I've spent the last (approximately) two years working towards an understanding of container technology and writing a simplest possible implementation. The process has left me with an understanding of what is required to build container infrastructure, and (perhaps more to the point) why all the parts are necessary.

From a technology viewpoint I've spent much of my time working with SmartOS, a Solaris derivative optimised to be a container hypervisor. Unfortunately it proves difficult to cloud host, and the mainstream has caught up with many of its innovations so I have ported my work to Linux. Areas of expertise I have gained during this process include...

Isolation (cgroups)...
is a major component of container technology. I have been using systemd-nspawn and also have considerable experience with Zones.

ZFS...
with it's ability to create a 'tree' of file systems, and zero cost snapshot/rollback, has proven to be superb for container file systems. The Linux and Solaris implementations are very nearly identical.

IP networking...
is the hardest part of the whole ecosystem. SmartOS uses Crossbow, which has an emphasis on completness, object orientation and performance. A subset of these features implemented for Linux are found in iproute2 and nftables. I have found that a series of subnets and static routes (over GRE tunnels) is both fast and simple to automate.

I'm also lucky to have experience and a broad understanding of IP networks under more usual circumstances.

Cryptography...
is obviously more important now than it has ever been. My work in this area has centred around using libsodium (a prescriptive public key cryptography library). I can never use passwords again.

Solaris/Linux...
Porting libraries and applications from Linux to SmartOS proved to be a large portion of my work - likewise the port back to Linux. This has left me with a broader understanding and appreciation of Unix, and experience with bringing applications live on Solaris derivatives.

Compiled Languages, C/C++, Objective-C and GLSL.

I have a long history with compiled languages, including:

  • C++ on and off for at least twenty years. C++ is still my go-to tool for algorithm and high performance development, and has great portability (haters gonna hate). I love the direction the language has taken in the last few years.
  • Objective-C for around ten years. I've written (or co-written) three ground up products for macOS, all of which are still in the market and all of which are still being maintained.
  • OpenGL, not a compiled language I know, but excellent for live graphics. I have a little experience with GLSL.

Miscellaneous

I, of course, have the usual mix of supporting skills you would expect: postgresql, bash, html, css, mysql, php, git, mercurial and subversion.

Experience

A word about this timeline: For much of my career I have worked for a company, often mine, but have done work on behalf of another. Similarly MixTape was a full time project that became part time (and led to spinoff work) and some smaller projects have been omitted completely. This timeline is thus not exactly linear and describing it as such is something of a challenge...

10.2015 -

20ft.nz - Container Infrastructure Provider.

20ft is a reaction to the current complexity of container infrastructure. As a multi-tenant infrastructure it's able to instantiate containers without first creating a VM cluster; supports physical location as a first class object; and has a Python SDK to provide agility where traditional orchestration solutions create a static deployment. Early applications are for development WIP (including digital agencies) and for the testing and CI/CD sectors.
01.2009 -

Atomic Droplet - Independent Software Vendor.

I develop, support and publish MixTape, an elegant and 'prosumer' centric music mixing application. Initially for sale through a home grown ecommerce facility, it's now available exclusively through the Mac AppStore and I participate in various co-marketing arrangements. A portion of the codebase was also licensed to a third party.
03.2015 - 06.2015

SilverStripe - Senior Solution Architect

My job description was extremely broad, but my time with SilverStripe was split between: acting as a 'first touch' technical contact for new clients; working with the account management team on writing proposals; and I joined a scrum team for an entire (but short) project.
11.2014 - 02.2015

Fronde - Solution Architect

I primarily worked with a client creating an HA backend using the Microsoft stack. The project was made more interesting by a broad definition of data and an agile approach towards its use.
06.2013 - 09.2014

Metra Weather - Software Engineer (C++, GLSL)

I maintained and enhanced Weatherscape XT, a live graphic solution for media industries across several counties including the UK, Korea and Thailand. I worked mostly with geometry, frame delivery and software architecture.
01.2011 - 02.2013

Weta Digital - R&D Software Engineer (C++)

For Weta I developed a voxel database and interpolation framework with storage requirements into the billions. I am credited as an R&D engineer for TinTin and The Hobbit.
01.2008 - 10.2008

shinywhitebox - Software Developer (Objective-C, GLSL)

I developed an OpenGL capture and compresson pipeline for live compression of HD streams.
02.2007 - 12.2007

Atomic Droplet - Self-Serve Advertising Venture.

Atomic Droplet was originally formed to create an internet advertising service with a unique reverse auction model. An early 'ajax' application, the technology worked but was not commercially successful.
09.2005 - 09.2006

Virtual Katy - CTO.

Initially bought in to help with raising venture capital, my previous company was acquired (for stock) and I became the company's Chief Technical Officer. We developed software that was the undoubted leader in it's niche and we were awarded the PAR Excellence Award. The company was eventually acquired by a group in Singapore.
12.2003 - 06.2004

Kiwa International - Outsourced Development.

My company was the sole provider of technology services for the VoiceQ ADR product. We ran the project from whiteboard to delivery including hiring contract staff from overseas. The software won a Hollywood Post Alliance Special Recognition Award for Engineering Excellence and my clients were honoured as Maori Innovation Icons. Development continued on an 'as needed' basis until 2009.
05.2002 - 07.2005

Periscope - h264 video conferencing development.

I created an R&D project to investigate the use of the (then new) h.264 video compression protocol in video conferencing. Despite having the only realtime software encoder in early 2003, efforts to commercialise the technology into the education sector failed.
09.2002 - 06.2005

Periscope - network infrastructure consulting.

I performed an independent comparison of 2G mobile data networks; and maintained infrastructure for CreativeHQ and Medialab South Pacific.
10.2000 - 05.2001

Afterswish - network load balancing.

My company created some load balancing IP which was eventually acquired by Allied Telesis for deployment in Japan.
02.1994 - 10.1999

Work in the UK

My early career consisted of my first company; work with C++ and MFC; some algorithm development and a short but instructive time working for Cable and Wireless.

Open Source contributions

Many of these are portions of 20ft that I have open sourced.

Messidge

A message passing framework for distributed systems needing a public gateway.
Messidge is a secure client server rpc. Not only does it enforce authentication and encryption, but is able to guarantee the provenance of a particular message. Messages are routed to a traditional controller; synchronous, asynchronous and streaming replies are trivial; and are all multiplexed over a single connection. Having a background in containers, Messidge also has a concept of nodes - being services provided across a private LAN (or VPC) that can be announced as part of a resource offer when the client connects.

Litecache

Query cache and async write for sqlite databases.
Specifically designed for use with slow (cloud hosted) drives and non-blocking application. The query cache is reset whenever the database is written to - even if written from an external application.

AwsOrNot

Classes for logging and key/value that work identically whether running on AWS or not.
A set of classes for logging and/or kv value storage that work 'identically' whether or not they are currently running on AWS - and using a drop-in compatible subset of the boto3 api. When running natively the platform's native features are used with no code changes.

Imgserv

Image server.
A lightweight SmartOS compatible disk image server. Runs as an SMF service.

Chime

Interactive art
Chime was interactive art for Kiwiburn 2016. It contains code to interface with a Novation Launchpad and a class for playing a number of samples precisely in sync.