Functional schmunctional...

[r0g]

I remember being forced to do a bit of functional programming in ML back
at Uni in the mid 90, the lecturers were all in a froth about it and I
must admit the code was elegant to look at. The problem was the dog slow
performance for anything half practical, especially with recursion being
the technique de jour. The HP workstations we were using were really
quite beefy for their time yet simple ML stuff would just murder the CPU
and RAM. Added to that, recursive stuff could be extremely confusing to
debug so I wrote it off as a nice idea in concept but a waste of my time
for anything practical.

So fast forward 15 years to this very afternoon... I needed a routine to
convert IP address strings to INET numbers and vice-versa and I remember
writing one a couple of years back when I first picked up Python. So I
went and dug it out.  Looking back at old code can be a bit dispiriting
though and this was no exception...

def inet2ip(n):
  p = (n/16777216)
  q = ((n-(p*16777216))/65536)
  r = ((n-((p*16777216)+(q*65536)))/256)
  s = ((n-((p*16777216)+(q*65536)+(r*256))))
  return str(p)+"."+str(q)+"."+str(r)+"."+str(s)

def ip2in(a):
  li = a.split('.')
  assert len(li) == 4
  a = int(li[0])*16777216
  b = int(li[1])*65536
  c = int(li[2])*256
  d = int(li[3])
  return a+b+c+d


I thought "OMG, how naive and inelegant that looks, I can do far better
than that, after all I know things like list comprehensions and
map/reduce now!".  I had also noticed over the last year or two a lot of
chatter about functional programming especially in conjunction with python.

It seems to me that procedural programming has taken quite a kicking
against the more OO and functional methodologies over the last few years
and given my nothing-but-positive experience with OO of late I was
reconsidering the strength of my old opinions on FP. Maybe it was, as my
lecturers thought, the paradigm of the future and I was being a
crotchety old stick in the mud by spurning it as slow and confusing.

With this in mind I set about applying my hard won Python knowledge and
wisdom to the rewriting of these functions, not for any particularly
practical reasons, just to see what improvements I could wreak on them
several years down the line.

The IP string to INET number was first and I decided to throw list
comprehensions at it...

def ip2inet(a):
  li = a.split('.')
  assert len(li) == 4 or len(li) == 6
  return reduce(add,[int(li[e])*(256**((len(li)-1)-e)) for e in
xrange(0,len(li))])

Now, that did made it completely unreadable but I thought, "it's pretty
much all one line now so I've gained some conciseness and at least it
should run a little faster". After all list comprehensions are often
fast aren't they? Well actually no, It runs at less than half the speed
of the previous function and now it's entirely lexically impenetrable.

Shit :-(

So far so bad... As inauspicious as these beginnings were I figured I'd
plug on anyway and try a functional approach to the INET number to IP
string function - the above experience having reinforced in me the
notion that clarity is paramount. Functional programming is all about
the clarity and elegance right?

I figured it might be worth trading a few cycles for at least,
especially as the machine I'm using here in 2009 has more RAM and CPU
grunt than whole Uni computer department did back in 1994. Not only that
I would only need to do 4 or 6 recursions in this algorithm and so I
shouldn't need to worry about the exponential inefficiency problems of
yore...

def inet2ip(n, l=[], c=4):
  if c==0: return ".".join(l)
  p = 256**( c-1 )
  l.append( str(n/p) )
  return inet2ip( n-(n/p)*p, l, c-1 )

The thing is, that's not really any clearer though is it?  I'd wager if
you didn't know already it would take you longer to figure out this
version than it would the original.  Still, "at least" I thought "it's
more concise than the last one and it scales to IPv6 far more
elegantly". I figured as long as the performance wasn't too lousy I
would keep it and so I quickly ran a little benchmark to see how much of
a performance hit I would be taking by doing so. The results for 10000
iterations of each were as follows...

0.113744974136 seconds for old INET->IP method
27.7441999912 seconds for new INET->IP method

:-/

FFS!

It just goes to show that...

1) Thinking you're a smart arse and actually being a smart arse are two
quite different things.

2) You should always take received wisdom with a good pinch of salt.

3) People aren't exaggerating when they say Python function calls are
expensive are they!


Roger Heathcote.