adrift on a cosmic ocean

Writings on various topics (mostly technical) from Oliver Hookins and Angela Collins. We have lived in Berlin since 2009, have two kids, and have far too little time to really justify having a blog.

I'm done with shell scripting

Posted by Oliver on the 12th of February, 2011 in category Thoughts
Tagged with: bashprogrammingpythonrubyscriptingshell

I think I will call this week the last I use shell script as my primary go-to language. Yes, by trade I am a systems administrator but I do have a Bachelor of Computer Science degree and there is a dormant programmer inside me desperately trying to get out. I feel like shell script has become the familiar crutch that I go back to whenever faced with a problem, and that is becoming frustrating to me.

Don't get me wrong - there is a wealth of things that shell script (and I'm primarily referring to BASH (or Bourne Again SHell) here rather than C SHell, Korn SHell or the legendary Z SHell) can do, even more so with the old-school UNIX tools like grep, sed, awk, cut, tr and pals. In fact, if you have the displeasure of being interviewed by me, a good deal of familiarity with these tools will be expected of you. They have their place, that is what I am trying to say, but the reflex of reaching for these tools needs to be quietened somewhat.

The straw that broke my camel's back in this instance was this sorry piece of scripting by yours truly. It's not an exemplary piece of "code" and I think that demonstrates how little I cared about it at this point. I was briefly entertained by the idea of implementing a simple uploader for Flickr in shell script, and I did actually manage to write it up in a fairly short amount of time, and it did then successfully upload around 4GB of images. The problem was that while the initial idea was simple enough, the script took on a life of its own (especially once the intricacies of Flickr's authentication API were fully realised) and became much more complex than initially envisaged.

Despite this, I had started out with the goal of making a reasonably "pure" shell uploader, and stuck to my guns. What I should have done, was call it quits when I started parsing the REST interface's XML output with grep - that was a major warning sign. Now I have a reasonably inflexible program that barely handles errors at all and only just gets the job done. I had a feature request from some poor soul who decided to use it and I was actually depressed at the prospect of having to implement it - that's not how a programmer should react to being able to extend the use of his/her work!

From a technical standpoint, shell is a terrible excuse for a "language". The poor typing system, excruciating handling of anything that should be an "object" when all you generally have to work with are string manipulation tools, and a "library" that is basically limited by what commands you have available on the system. I know that I have probably barely plumbed the depths of what BASH is capable of, but when the basics are just so hard to use for what are frequently used programming patterns, I don't really see the point.

So from next week, I've decided to reach for Python or Ruby when I have to code something up that is more than a few lines' worth, or of reasonable complexity. Not that I don't already use Python and Ruby when the occasion calls for it, but I think that those occasions are too few and far between. Shell scripting is an old-school sysadmin crutch and it is time to fully embrace the DevOps mentality and get into serious programming mode.

© 2010-2018 Oliver Hookins and Angela Collins