Companies large and small are singing praises for infrastructure as code and immutable infrastructure. Younger companies are using tools around Docker and containers to promote and mandate immutable infrastructure. These are all great, but if you are a lone bootstrapper burning the midnight oil, it does not seem like the end justifies the means (at least not to me). Not only do these tools require constant upkeep, as this domain is so rapidly changing, but there are often subtle and intricate ‘features’ that spring up when particular versions are paired, or on some Linux kernels, or with some file system types, or, or, or. As a developer who spends a lot of time promoting immutable infrastructure on the job, I really wanted to find a practical way to achieve similar results for my bootstrapping world. This meant attention to the following concerns:
In the context of our requirements
Packer, Vagrant, VirtualBox, and Asgard are all free and open source products. While Amazon Web Services is certainly not the cheapest way to host your infrastructure, it is one of the cheapest ways to quickly and easily convert a project to using immutable infrastructure. In this context, as well as the growth opportunities AWS offers, this is a cost burden I am willing to accept.
VirtualBox is cross-platform and has easy installation. With Vagrant installed, using VirtualBox becomes even easier.
Packer is Go project, and it is distributed as a bundle of pre-compiled binaries. Even so, I plan to include packer binaries and some boilerplate packer config files in a project that accommodates my own needs for immutable infrastructure on a dime. There is a challenge here in that you must somehow convert your infrastructure to code, but Packer leaves a bootstrapper in good shape to implement this in the most convenient way that Packer supports (shell, Chef, Puppet, Ansible, Salt, Custom). If everything is hand-implemented, probably flipping over to a handful of shell scripts will be fastest, though some of those other tools are certainly worth investigation when there is time (as if…).
Asgard is a JVM project, distributed as a standalone jar file or a deployable war. It requires some knowledge of the JVM and servlet containers, and you also benefit from having basic Amazon Web Services familiarity. Alongside the packer installation, I am including Asgard install and a walk-through for getting up and running in the previously mentioned project.
AWS is daunting to newcomers. This is unfortunate, but I am willing to make an exception for AWS here since we really only need to stumble through some basic AWS tasks as a preliminary step. Once we configure a user and retrieve their credentials, Asgard takes care of the bulk of our AWS needs.
The newest player here is Packer, and while I have certainly hit some bumps in earlier versions Packer is moving along nicely and has well-established support for basic Amazon Machine Image (AMI) creation needs. Things can get tricky if you are using brand new AWS features or less-popular Packer options, but those are easy to work around and for a basic AMI builder they should not pose any challenges.
Vagrant and VirtualBox have been staples for my development for many years and they continue to get better. I can not imagine developing for the web without Vagrant and some virtualization technology.
Asgard was open sourced by the Netflix OSS team in 2012, and was already powering infrastructure internally. It manages a lot of production servers for a lot of companies.
AWS is always in the mix when we discuss cloud technologies and/or providers. Their backwards compatibility in the face of relentless product enhancement and growth is impressive. Even though AWS is mandated by the Asgard decision, it would be a strong contender nonetheless.
The real concern here is Asgard (or any comparable solution). I am going to install and run this locally, so it is at least as private as my project code.
My solution is to create a virtual machine for local use that will interact with my AWS account from my development machine. I plan to have a single vm for my project, ideally created with some configuration management system. I can use this for development and testing, and when I am ready to deploy I can run a separate virtual machine with this immutable infrastructure toolset.
The infrastructure management virtual machine (also created with some configuration management system) will run Asgard and Packer. I am going to keep it simple and use Asgard over the default tomcat port on the virtual machine, and ssh into the virtual machine to kick off Packer builds. Once Packer builds an AMI, I can locate the AMI in Asgard and trigger a deployment.
I made noteworthy concessions in this solution:
This is getting cumbersome, so I plan to break it into 4 or 5 posts. They will look like
If you made it this far, you might enjoy a book I am writing on this topic: Immutable Infrastructure with Netflix OSS.