More on Bundler and RPMs

by Oliver on Friday, June 22nd, 2012.

So I was going to post a comment on the original blog post which I linked to from here but Facebook connect was broken and I don’t feel like setting up yet another account </1stworldproblems>… but there was a slight development.

I attempted to use the same methodology we had already followed with the first app, to another app so that this one was also packaged using Bundler and RPM. Needing to confirm that all was well before I committed the changes I did some testing in a CentOS virtual machine in Vagrant. To be expected, with a deployment bundle of a decent few gems the package size comes out at around 15MB. I committed the changes and the produced RPM from the Jenkins build job was 50MB. Why?

Initially, I suspected subtle differences in Bundler gem versions, library path differences etc but these ended up being dead-ends. What was happening, however, was that the gems were being installed into apprepo/vendor/ruby/1.8, including the excluded groups. I am assuming this is a necessity for the tests and other build-time checks to run, but I certainly didn’t want them to be packaged with the RPM which can rely on just the gem cache.

As it turns out, Bundler has some “smart” code around user permissions – specifically around what commands you can run through sudo. A standard Vagrant box will have unrestricted sudo access for the vagrant user, so it can install gems anywhere. Bundler uses this fact to its advantage and will install them into the standard /usr/lib64/ruby/gems/1.8/gems/ path. Hence, when it comes time to package up the gems as an RPM, these files are not in the app build path and the RPM stays a slim 15MB.

In our build pipeline which uses a standard user account on a fairly normal CentOS install, the jenkins user has no such permissions and thus has no option but to install them into the vendor directory along with the other Bundler artifacts. The solution was simply to exclude this directory from being packaged, although I’m still not entirely sure why we didn’t hit this problem the first time around. Nevertheless, bearing in mind these few gotchas, we now have a system in place that makes it a snap to add more gems and maintain a well-packaged and stable application from development to production.

Tags: , , , ,

Friday, June 22nd, 2012 Tech 2 Comments

Bundler, gems and RPMs

by Oliver on Wednesday, June 6th, 2012.

Recently I was working with one of our valued Thoughtworkers on an application we were trying to not only develop in a sane way, but package and deploy to production with just as much sanity. The status quo seems to favour bundler on the development side, but RPMs on the production side (if you judge these decisions based on what developers and ops folk prefer, generally).

After a reasonable amount of WTFing, we actually managed to get it working reasonably well. If you have Ruby apps with gem dependencies and want to develop and push to production with equal ease I suggest you read the blog post on the subject by Philip Potter here: http://rhebus.posterous.com/rpm-ruby-and-bundler

Tags: , , , ,

Wednesday, June 6th, 2012 Tech 1 Comment

Dependencies stored in grey matter

by Oliver on Wednesday, February 2nd, 2011.

I have a Zotac ZBox which I use an my HTPC, and generally it works pretty well. One thing that is slightly troublesome is the HDMI audio, which seems to rely on having the ALSA backports modules package installed in order for it to work. Remembering this is key, though, since when the package is installed it does not automatically get updated when you upgrade the kernel package.

Most packaging systems rely on the fact that you only have the one instance of a package around at a time, and as time passes you upgrade these packages between versions (a notable exception is the Gem packaging system). Kernel packages are the exception to the rule and not only can you have several present on your system at a time, but this is usually desirable so that you can test stabliity, rollback etc. For this reason the version number of the kernel creeps into the package name, making it effectively treated as a unique package (and since the file paths differ, it is unique on disk as well). The DEB package format handles upgrades by way of a meta-package which pulls in the latest version of the kernel package. RPM uses some other magic I can’t recall right now.

In the case of the linux-backports-modules-alsa package, the same idea applies. However where the kernel meta-package pulls in the newer kernel package when there is an update available, it can’t do the same for this package since not everybody wants it installed automatically. Since I do want it installed automatically but am not in a position to change the packages, this puts me in a slightly irritating position. Ideally there would be some hook that I could use to pull in the latest version of this package whenever a new kernel package is installed (and in fact there is, in /etc/kernel/postinst.d/) but anything triggered synchronously with the kernel upgrade will fail since the dpkg transaction is not yet complete and starting a new one will be blocked.

The trigger could in fact schedule an at job to install the newer alsa package a few minutes later, but I don’t like the asynchronous nature of this method and the likelihood of failure (what if I reboot immediately after to activate the new kernel?) although I can’t see an obvious alternative. Does anybody have any suggestions?

The work around for this to prevent having to remember to install the latest version is to make use of the kernel package maintainer hook directory: /etc/kernel/postinst.d. Scripts in this directory are called after installation of a new kernel with the first parameter being the new kernel version in uname -r style format.

Tags: , , , , , , ,

Wednesday, February 2nd, 2011 Tech No Comments