Archive for November, 2010

Reasons to love PostgreSQL

by Oliver on Sunday, November 21st, 2010.

I’ve been hacking on a small Ruby on Rails project to summarise and then prettily display pmacct traffic data. It’s not an original idea by any stretch and in fact is directly inspired by a system we had at my last job. For obvious reasons I have to at least reimplement the idea myself, and while the original was written in Python using PostgreSQL, mine will be using Ruby and MySQL.

One small pain point I’ve already encountered is the lack of IP address functions in MySQL. PostgreSQL at least as far back as 7.4 has supported IP address functions and in the case when you need to be comparing addresses and summarising based on local subnets they are sorely missed. It really makes me sad, and wonder why even the latest MySQL does not have any support for these functions.

Granted, having built-in support for certain functions does not always help you. It is well known that you can often gain performance for your application by sorting records in the code rather than in the database. However, without even having the option there to allow you to make a comparison, it is impossible to say which way would be better. I know that in terms of convenience and LOC savings, I’d prefer the functions to be in the database engine.

Tags: , , , ,

Sunday, November 21st, 2010 Tech No Comments

Preparing (slowly) for Winter

by Oliver on Saturday, November 13th, 2010.

Last year in Berlin we had a fairly terrible Winter, with temperatures going down to -30C which we were completely unprepared for. Ange was pregnant and neither of us had really adequate clothing although we both got sturdy boots and some thermals to stop from freezing altogether. This year we vowed to be more prepared, and one of the most popular brands that we see on the streets is Jack Wolfskin. Yes, it is a “name brand” and I think a lot of people just wear it for the logo rather than the quality.

However, they do actually make some pretty good stuff. I picked up a pair of gloves to stop my hands from freezing on the ride to and from work and they keep my hands nice and toasty the whole journey. I don’t expect that we’ll become as bad as some of the people out there (I’ve heard of entire families getting around entirely clad in JW gear) but I think we’ll look into some other items to keep the cold out properly this winter.

Tags: , , ,

Saturday, November 13th, 2010 Germany No Comments

My love/hate relationship with Ruby

by Oliver on Wednesday, November 3rd, 2010.

I consider myself a failed programmer, having never really excelled at it during University and only really having come to terms with some of the concepts several years later. I’ve always liked programming but at some point years ago I decided I didn’t want to be a programmer/developer so that was that. Since cementing myself in the realm of Systems Administration I’ve come to miss the programming that I was once so terrible at (and probably still am), but I never have quite enough time to catch up what I’ve missed. The programming landscape seems to have changed so much in the years subsequent to my joining the workforce so it seems like an ever increasing amount of new things to learn.

While working at Anchor I came to grips with Python which was at the time the “standard” language for the company (although I see now that their website is probably running on Ruby on Rails). I like Python, and find it logical and convenient (if not the best supported language out there at the moment). Ruby is actually not so much the new kid on the block any more but still has all of the Fanboyism that it gained a few years ago (if not more). Like the die-hard Mac users, Ruby programmers will defy all logic to defend their beloved language.

Critics of Ruby have made their opinions known far and wide around the Internets so I won’t repeat them here. I actually quite like Ruby because it is easy to use, has a huge collection of Gems to add functionality (and all-important code-reuse) and it is the language of Puppet which is my favourite configuration management tool, so I have to use Ruby to interface with it. I can get by with Ruby, but I also hate so many things about it.

One of the favourite lines of Ruby fans is how efficient Ruby is with simple string handling, thanks to the feature known as symbols. These are basically just a string of characters (with certain limitations) prefixed by a colon character, like :symbol. The efficiencies come from only storing the one copy of a symbol in memory at any time, even if it is used in many different places. I was intrigued by this claim when I first read it and set out to test the theory.

  100000000.times do
    foo = :abcdefghijklmnopqrstuvwxyz

That’s my basic testing framework. It is probably very naive, but I was looking for simplicity. To get an idea of how miniscule the “efficiencies” we gain, we have to run this loop 100 million times just to see numbers that differ significantly. The first time I ran this test over a year ago, I got slower results using symbols than using strings (“abcdefghijklmnopqrstuvwxyz” or ‘abcdefghijklmnopqrstuvwxyz’ rather than the symbol above) and laughed long and hard. I’ve now just retested and got the following results:

Symbol: 44.661 seconds
Single-quoted string: 53.224 seconds
Double-quoted string: 53.276 seconds

Wow, there actually is a benefit in using symbols. But bear in mind, we only saved about 9 seconds over 100 million invocations. You would have to be doing some pretty serious symbol use to gain performance from this. Ruby fans will take exception to this saying that the point of symbols is not for performance but for memory consumption, to which I would respond that Ruby has far more serious memory issues than in handling a few duplicate strings. Seriously.

The reason I tested single- and double-quoted strings is due to Ruby needing to check for interpolated variables within the double-quoted string. I had expected there to be more of a difference in performance but clearly there is not.

Out of interest I tried the same loop test in Python:


i = 1
while i <= 100000000:
    foo = 'abcdefghijklmnopqrstuvwxyz'
    i += 1

How long did it take? 20.634 seconds.

Tags: , ,

Wednesday, November 3rd, 2010 Tech No Comments