Writing more efficient code

[Jon Harrop]

gonzlobo wrote:
> I picked up Python a few weeks ago, and have been able to parse large
> files and process data pretty easily, but I believe my code isn't too
> efficient. I'm hoping dictionaries will help out, but I'm not sure the
> best way to implement them.
> 
> I've been using a bunch of nested if/elif/else statements to select
> slices (0317 & 03de) from a file, then parse the data (aa, hh, bb,
> d2-d9) into parameters (a = airspeed, h = heading) & flags.
> 
> #sample file contents
> 0000007d 03 0317 aa aa aa aa aa hh hh hh bb bb
> 0000007e 06 03de d2 d3 d4 d5 d6 d7 d8 d9 10 11
> 
> # some pseudo code
> if PID == '03de':
>     flapsCmd = int(d3, 16)
>    if flapsCmd == 0xc0:
>       <flaps up code>
>    elif flapsCmd == 0x03:
>       <flaps down code>
> if PID == '0317':
>    airspeed == 'combine aa for airspeed & multiply by 0.1'
>    heading == 'combine hh for heading'
>    mach == 'combine bb for mach & multiply by 0.01'
> 
> Might dictionaries help in this case... say Label0317(parameterName,
> slice (d3), scaleFactor(0.1))... I'd like to use them if they'll
> replace the dozens of nested conditionals.

Sounds like you want pattern matching over lists:

  | 0x03 :: 0xde :: 0xc0 :: t -> <flaps up code>
  | 0x03 :: 0xde :: 0x03 :: t -> <flaps down code>
  | 0x03 :: 0x17 :: t -> ...

That's OCaml code which finds patterns in sequences of bytes, just like the
ones you're searching for.

> I have roughly 75 
> different parameters to decode from a file containing ~2.5 million
> lines of data.
> 
> I know my pseudo code lacks details, but hopefully I'm getting my
> point across...
> 
> (I suppose switch/select/case statements would help a bit, but python
> doesn't seem to use them... not to start a religious war or anything).

Pattern matching is "switch" on steroids.

OCaml's pattern matcher is very sophisticated and very fast. You'll probably
shrink your code by several fold whilst also having it run a few orders of
magnitude faster and having it statically checked, so it will be more
reliable.

You might want to look at any language with pattern matching: OCaml, SML,
Haskell, F#, Mathematica etc. If you're running Windows then F# is a .NET
language...

-- 
Dr Jon D Harrop, Flying Frog Consultancy
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists/index.html?usenet