Geek and Artist

Self-rating on the Spolsky/Limoncelli tests

Tom Limoncelli just released his own version of The Joel Test – except his one is for sysadmins. I was only vaguely aware of Joel Spolsky’s test and only just read through it and rated my current team, and I’m glad to say we are just about at twelve for twelve. It hasn’t really been on my radar until now as this is the first time I can actually say I’ve been on a development team. Given all the buzz around DevOps, this is quite an exciting phase of my career as we are basically a development team of Systems Engineers, working on automation and all the things sysadmins dream about.

We actually spent a good amount of time planning out our product, developing our ways of working and a few sprints down (yes, we are using Agile) things are generally going pretty smoothly and enjoyably. The team is a handful of geeks with a range of interests not just related to pure sysadminery so it makes for a pretty enjoyable place to be right now.

The only areas I can say we don’t entirely meet Joel’s criteria are:

• Daily Builds. We develop in Python and Ruby and our test suites are pretty quick, so we build on every commit (or push for Git). Instead of having a single monstrous test suite for everything we split things up pretty aggressively and have all the simple tests done in the first stage of the pipeline, and more intensive tests done in later stages so as to enable fast feedback. I’ve written (and talked) about this a few times before.
• We don’t have testers. We do however have a rigorous code review mentality, actually do pairing when writing code (it’s totally worth it) and test each other’s work constantly. I know this doesn’t meet up to Joel’s testing criteria but we’re only a team of five or six right now. Hopefully this will change in future (although we do have a very willing army of beta testers right now).
• Candidates don’t yet write code in interviews, but this is something we definitely intend to change. Sadly the DevOps movement is still young and it is extremely difficult finding people who are both competent sysadmins and coders. Usually they will be one or the other and I have read several times now how it is easier training a developer to be a good sysadmin than vice-versa (and this certainly on face-value would seem to make most sense). If you want to prove me wrong, send me your CV and come in for an interview!

So overall I think we’re doing pretty well. On Tom’s (extremely comprehensive) test I have to say we (or rather, my old team now) don’t fare so well. To be completely fair, we maintain a huge array of services, maintain environments which span development VMs where developers germinate the first seedlings of an idea to the production machines on which they serve millions of users world wide, datacenters not only local and abroad for R&D but in many countries around the world serving our production services. Some of us even manage to fit on-call time into our crazy lives.

We are sometimes at the mercy of development teams pressing to get their products live so I can’t say we have a strangle-hold on all of Tom’s test items – but if we did, what would be the challenge and fun that keeps us coming back day after day? I also see Tom’s test differing significantly from Joel’s. If you score 10 or less on Joel’s test you are in pretty big trouble. Tom’s test has only a handful of such critical items among the 32 questions, so it is much easier to fall into the trap of either thinking you are doomed (and you’re not) or thinking you are fine (and you’re not). Not to say that his test is any less valuable, and of course such things are very useful for applying good DevOps practices to sysadmin teams.

So my advice is, take a look at what you didn’t answer yes to and think about what you can do to get there. I can personally vouch for many of the items being painful to implement when you don’t already have them, but once you have it done you won’t regret it and you won’t look back (unless you maintain your documentation in LaTeX… never again!)

Tags: devops, joel spolsky, tom limoncelli