Writings on various topics (mostly technical) from Oliver Hookins and Angela Collins. We currently reside in Sydney after almost a decade in Berlin, have three kids, and have far too little time to really justify having a blog.
If you have previously read anything on this blog (directly, rather than via RSS) you will have noticed that the visuals have been updated. This is the culmination of a very long and arduous process - entirely my responsibility and self-inflicted - and as a result, it is running via Netlify, built with Gatsby and stored in Contentful. For styling I relied heavily on Tachyons and for the sake of completeness, used a header image from Unsplash.
If you didn't see it before, this is what it looked like:
It was hosted on Wordpress since about 2010 when I started blogging, initially on a former employer's shared hosting service, and then later I moved it to a VPS I was managing myself. It has automatically upgraded through the various versions of Wordpress as they became available, and used the Decoder theme, which felt modern when I chose it but is really starting to look dated now.
In order to understand the challenges the engineers were experiencing, I went through a few online courses, completed all of the exercises on ES6Katas, and built a few small dashboards. These initially were very horribly coded - creating DOM elements manually and populating the page piece by piece. Jan pointed me in the direction of Flexbox Froggy which helped immensely in the styling of some of my build status dashboards and generally helped me understand how styling for the web works. We are really spoiled today with built-in development tools in the browser and dynamic editing of the DOM!
More recently, I learned React since it seemed to be gaining traction (and we've selected it as our future standard framework for our web app at Contentful), and I dutifully migrated the various dashboards I'd created over to that. It drastically simplified the code and made it more modular (and hopefully more maintainable, should anyone be so adventurous). I had made a somewhat false start on migrating the blog off Wordpress towards the end of 2016, but it had involved a Node service retrieving content live from the Contentful API and server-side rendering with Jade (now Pug) templates.
After watching Khaled's excellent four part tutorial on Gatsby I resolved to finally get it done.
Back in 2016 I made a first attempt, using the tooling that was around at the time which was the wordpress-exporter. Wordpress itself facilitates data exporting by letting you download an XML-encoded dump of the database. The format is pretty horrible, and the result in my Contentful space looked like a 1:1 facsimile of Wordpress's data structure, which is fairly generic and one-size-fits-all. I did one initial import of the data into a space and started developing the previously mentioned Node service, and built a couple of simple views. It felt a bit difficult and some other priorities popped up so that effort got shelved until earlier this year.
In the meantime my writing output dropped off significantly, partially due to working in management rather than as an individual contributor, and also due to lack of time. The couple of blog posts I decided to make ended up going still in Wordpress which felt like continuing a bad habit. This year I started thinking of some new topics to write about but felt that I didn't want to put them into the Wordpress site - hence jumping back into the data migration. I pulled Gatsby into the old codebase before realising I wanted a more understandable content model to work with. Content Modelling is a pretty key concept in our product, so not only did I want to do it the right way but also wrap my head around exactly how my content would fit together.
I decided it would be fun to use our very alpha Go SDK to write a data migration tool, and the resulting code is here on Github. The SDK didn't support some key features like adding entries, specifying your own IDs for some entities like Content Types, unpublishing assets etc. I ended up making my own fork of the SDK and adding the functionality I needed (although it needs some work to be merged upstream). My workflow was basically this:
I went through many cycles like this, deleting everything from the space, adding everything back, checking the results in the web app, tweaking something and repeating. Everything was done serially so some operations like adding/deleting tags took a while. Later I added code to re-write URLs that were inside of the post body data to relative URLs (especially for links between blog posts that I use a lot). I also used this "rewrite table" to generate a redirects file for Netlify to use - I am now using "slugs" in the post URLs whereas the old Wordpress site used a query parameter to refer to the post ID. Old links from search engines or RSS feeds will redirect to the new location of the post in question.
Another interesting aspect of the migration is that usually people are interested in converting from Markdown to HTML, not the other way around. Pandoc solves that problem, being able to convert between any two given formats, from a rich list of options. My migration tool just executes another process for
pandoc on temporary files, reads the output in again and loads the result into the CMA as an entry (after rewriting URLs and removing other cruft in the output).
Last week, once I was fairly happy with the results, I did one last final import of all of the content and today I've spent several hours combing through each post checking for broken links and other manual fixes that the tool wasn't easily able to do itself. Hopefully the job is done now and I won't have to reimport again, but if so, I have 90% of the automation encoded. Perhaps someone else can use it as well.
As mentioned above, I made an initial start with Gatsby in the same repository in which I had developed the Node app, but it became quickly evident that I was not going to want (or need) the backend component, and I also needed more of the content structure and entries available to continue building out the website. I used some placeholder content to start building the React components and overall layout. Had this been perhaps two or more years ago I'd have been tempted to use Bootstrap for much of the layout and styling (in fact I've used this in the past for some dashboards and simple front-ends to infrastructure systems) but I'd heard of Tachyons as a lighter-weight alternative to Bootstrap so I decided to give that a try. I started out styling components myself, and gradually moved styles to standard classes already present in Tachyons. There are a couple of things missing which I styled "manually" - text shadows (not box shadows) and some fine tuning of padding/margins. I feel I actually have a pretty good grasp on these elements now, which is nice after such a long time.
The GraphQL querying of the Contentful API was a little tricky to work with as I'd not worked with GraphQL before, but Khaled's tutorial series referenced above helped immensely with this (as well as some personal assistance from him answering my questions 😍). I also borrowed his strategy on pagination of the index page. Font and colour selection are two aspects I have not trained my eye enough for (probably), but I can live with my choices! Similarly, I'm not a "pixel perfect" styler, but prefer to ensure there are no obvious misalignments or visual gaps.
The other dramatic improvement in this area was in hosting. Netlify has been gaining a lot of traction (we use it for our main website at Contentful, and it received a mention recently on the Thoughtworks Tech Radar. Deploying a Gatsby site to Netlify is even simpler than deploying my old Node app to Heroku, and is what seems like a perfect fit for a site like this. Let's Encrypt certificates are supported (and configured/renewed automatically), DNS management is also part of the offering and integrates seamlessly, and I was able to hook it up to the development pipeline by both linking to the Github repository and sending webhook notifications from Contentful on entry publishing. Amazingly, I think I don't even have very extensive use of their features, which go far beyond this. The redirects file as mentioned above is also great for not losing clicks to old URLs from the Wordpress site.
I was conscious that I didn't want to re-implement all of the functionality of the Wordpress site, to keep the work involved to a minimum. The admin side is taken care of by the Contentful webapp, but outside of that, everything is up for grabs. Here is the list of what didn't make the cut:
Given that the "project" in a sense should have been done in the early months of me starting at Contentful, it is way overdue, and needed to be kept lean. Obviously some knowledge pre-requisites stood in the way, but I'm glad I could stay relatively strict with myself about feature-set and get it completed quickly once I'd set my mind to it.
Not much else to say here except that the combination of Contentful, Gatsby and Netlify is a good one and I can recommend it. Styling and front-end development is not to be feared, and can be mastered at least to the point where one can build a content-driven site such as this. React and similar frameworks make this a lot easier, I have to say.
I hope with this long-overdue site migration I can get back to writing a few more technical posts, hopefully at a rate of slightly more than one a year! Any comments, questions or feedback - you'll have to direct to me on Twitter for the moment - I'm happy to chat more about my experiences or answer any questions if you want to go about a similar migration or site build.
Thanks for reading.