About alternatives to Matlab

[Jon Harrop]

Carl Banks wrote:
>> Optimized Python is 14% slower than badly written OCaml.
> 
> I'd call that roughly the same speed.  Did you use any sort of
> benchmark suite that miminized testing error, or did you just run it
> surrounded by calls to the system timer like I did?

System timer, best of three.

> If the latter, 
> then it's poor benchmark, and 10% accuracy is better than you can
> expect.  (Naive tests like that tend to favor machine code.)

Yes, they are roughly the same speed. My objection was that this OCaml
program was not the unoptimised one, it was the badly written one. The
unoptimised OCaml/F#/C++ are all several times faster than this OCaml, and
much shorter and simpler too.

>> Unoptimised but well-written OCaml/C/C++ is 2.5-2.8x faster than the
>> fastest Python so far whilst also requiring about half as much code.
> 
> Frankly, I have a hard time believing anyone would naively write the
> Ocaml code like the optimized version you posted.

That wasn't optimised. I haven't optimised any of my solutions. The OCaml I
posted was a naive convolution with some unnecessary allocation.

> You'd know better than I, though.

When asked to convolve an array with a 4-element array (h0..h3), I think
most programmers would write (pseudocode):

  for i in [0 .. n-4]
    x[i] = h0 y[i] + h1 y[i+1] + h2 y[i+2] + h3 y[i+3]

in most languages. That is the essence of this benchmark.

You can express it even more succinctly with a dot product:

  for i in [0 .. n-4]
    x[i] = h . y[i:]

In this case, you'd want a cyclic array type too.

>> That's my point, using numpy encouraged the programmer to optimise in the
>> wrong direction in this case (to use slices instead of element-wise
>> operations).
> 
> Ok, I can see that.  We have a sort of JIT compiler, psyco, that works
> pretty well but I don't think it's as fast for large arrays as a good
> numpy solution.  But it has more to deal with than a JIT for a
> statically typed language.

Ok. Perhaps starting a Python JIT in something like MetaOCaml or Lisp/Scheme
would be a good student project?

>> Which begs the question, if you were writing in a compiled language like
>> F# would you ever use slices?
> 
> If I was writing in F#?  I absolutely would, because some problems are
> most cleanly, readably, and maintainably done with slices.  I don't
> have any speed-lust, and I don't waste time making things fast if they
> don't need to be.  I would rather spend my time designing and
> implementing the system, and optimizing things that actually need it.
> I don't want to spend my time tracking down some stupid indexing error.

Ok. Can you name some problems that are solved more easily with slices than
with indexing? I may have just answered my own question by posting some
really concise pseudocode that uses slices.

> Will any other F# people do it?  I don't know if they all have
> speed-lust, or if they'd all deliberately avoid slicing to prove some
> kind of point about functional programming and/or non-machine-compiled
> languages, but I assume there'd be some who would do it for the same
> reasons I'd do it.

My concern is that slices are being pulled into F# because they are popular
in languages like Python and Matlab but I've yet to see an example where
they are actually a good idea (in the context of F#, i.e. a compiled
language where indexing is as fast or faster).

>> What makes numpy convenient?
> 
> ...it scales up well...

What do you mean?

> ...it expresses mathematical concepts straightforwardly...

It didn't express this mathematical concept very straightforwardly.

> And a lot of the time, you don't have to worry about indexing.

Right. Ideally you want to factor that out without losing anything. Indeed,
if you do factor the C++ code you should get back to the underlying
mathematical definitions. Ideally, you'd want to write those directly and
have them executed efficiently.

>> > No problem, numpy will be fast enough for my needs.
>>
>> Ok. But you'd rather have a compiler?
> 
> Python *is* compiled, buddy.  It has a VM that runs bytecode.

I meant a native-code compiler, of course.

> But, would I rather have a seperate compile to *machine code*, when an
> automatic compile to VM code is fast enough?
> 
> Not remotely.  Compilers are a PITA and I avoid them when it's
> possible.

I can envisage a JIT Python compiler that would spot statically typed code,
compile it with no user intervention and then run it. I implemented such a
thing for Mathematica a few years ago.

>> Why not drop to C for this one function?
> 
> Why would I?  The assumptions clearly stated that numpy is fast enough.
>  Why would I take several hours to write a C extension when I could do
> it with numpy in about 20 minutes, *and it's fast enough*?

Is it very difficult to interface Python to other languages efficiently?

> But let me offer you some advice: if you wan't to lure people from
> Python, speed isn't going to cut it.  Most people who chose Python
> (and, let's face it, most people who use Python chose it themselves)
> already knew it was slow, and chose it anyways.  Pythonistas prefer
> Python because it's clean, readable, well-supported, and practical.
> Practical might be the sticking point here.  If you were to get a
> Python prompt, and type "import this", it would print the Zen of
> Python, a set of principles by which the language is designed.  One of
> them is "Practicality beats purity."  And let's face it, functional
> languages are (or seem to be) all about purity.  Notice that there's no
> Zen that says "Fast is better than slow".  If you want us to swtich to
> Ocaml, you better figure out how it supports the Zen of Python better
> than Python does.

I'm keen on the practical aspect. I intend to take some of the excellent
Python demos out there and port them to languages like F# in order to
illustrate how they too can be practical languages.

-- 
Dr Jon D Harrop, Flying Frog Consultancy
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists