ben tedder : code things

Continuously integrated static sites in 9 minutes

Stop wasting time. Mostly.

If you've ever worked on a Rails site, you know the joys of working with a beautiful syntax like Ruby. But if you've worked on a Rails site, you also know the joys of deployment. As the gemfile gets larger, your deploy time seems to grow exponentially. In a continuous integration environment where code is being pushed dozens of times per day, this can mean a lot of waiting.

45 minutes of my life...gone (4-5 times a day)

A few months ago at Lonely Planet we were at the point of waiting a good 45 minutes between merge to master and seeing our builds live.

While we were sitting there waiting for our builds to go through, we started thinking of ways to speed the process up. We did things like offload our image assets to S3 (which did help). We talked about ways to cache gems on the server. We talked about trimming out unnecessary gems. But in the process we landed on probably the most important piece of optimization we could perform.

Leave Rails.

No, we have not left rails as a company, not by any means. But there are projects, as you surely know, that are not suited for a rails app. In particular we were dealing with a lot of short-lived, 1-2 page, •database-free* pages. These pages were being churned out at a rate that was quickly making them unmanageable. They were being stored in one rails app, all deriving dependencies from the same gems (Rizzo for example). This led to pages quickly becoming a burden to maintain.

It's all just static

Turns out, a portion of our sites and pages weren't that complex. We were waaaay overcomplicating not only the deploy process, but the development process. At the end of the day, we were pumping out sites that needed little more than HTML, CSS, and javascript.

So, putting our heads together, we came up with a suite of tools and workflows that have enabled us to spin up sites (from concept to repo to scaffold to jenkins integration and deploy) in 9 minutes (my personal record).

Our tools

Our basket of tools for creating these static sites is fairly simple, but did take a few iterations to get right. We've since wrapped everything in a Yeoman generator to make our lives oh so simple. Watch this space to see if we get to the point of open-sourcing that generator.

On to the tools:

HTML

Rather than writing plain HTML, we wanted to take advantage of templating languages, building out pieces of a site from JSON files, and sharing partials. We settled on an open source project called Assemble. While it's still in the early stages of development, Assemble lets us run our entire front-end workflow from a Gruntfile. As a front-end developer, this was very enticing. We didn't need ruby even (see Jekyll, another option we considered).

CSS

We are very much a SASS shop at Lonely Planet. And in building our static sites we wanted to keep that approach. So we write all of our CSS in the SASS syntax and compile it with, you guessed it, Grunt. The one concession we've made here is to use the ruby sass and sass-globbing gems. It makes life a bit easier and only adds a few seconds of build time. Ideally we'd have a gem-free environment. But compromise is what keeps the world spinning.

javascript

The first edition of our scaffold started out using the more node/EC6 way of doing javascript modules and using browserify to compile for the browser. However, we had to ditch that approach because we were extremely tied (in a good way) to our internal style guide, Rizzo. Because Rizzo handles all the javascript with requirejs, we followed suit. Personally I quite like requirejs. It gets tricky when you introduce it to the Rails asset pipeline, but as an AMD solution, I like it. We're not necessarily using it for its asynchronous ability, but for dependency injection it's nice.

Dependencies and build tools

We're using bower for dependencies where we need them. We use Grunt and a myriad of npm packages for compiling everything. We use yeoman to package it up and scaffold it out. We use github to host the code. We use jenkins for continuous deployment. And once you get the process down, 9 minutes is all it will take.

Deploying a static site

Because we're using Jenkins for our continuous deployment/integration for everything else at Lonely Planet, we wanted these static sites to follow the pattern and be deployed like everything else. But because these sites were static, we came to the conclusion that we didn't even need to put them on an EC2 instance. We are able to host everything in an S3 bucket, put fastly in front of it, and have blazing-fast, completely cached sites served from an edge server near you!