Benefits of moving from Python to Common Lisp?

[Daniel Lakeland]

In article <fbc0f5d1.0111150352.54896c2b at posting.google.com>, "Tim
Bradshaw" <tfb+google at tfeb.org> wrote:

> Paul Rubin <phr-n2001d at nightsong.com> wrote in message
> news:<7xy9l9ne1b.fsf at ruckus.brouhaha.com>...
>> 
>> However, most Lisp compilers have depended on type declaration pragmas
>> to generate good code, AFAIK.  Python would need a language extension
>> to support something like that.
> 
> I don't think this is really true for many interesting programs.  It is
> very definitely true for *some* kinds of programs, for instance things
> that do heavy numerical work, and almost all of the trivial, stupid,
> non-benchmarks that people use to decide that `language x is faster than
> language y' so they can flame on inappropriate newsgroups (cll readers
> will be familiar with this).
> 
> However a lot of programs spend a lot of time doing stuff where the
> typechecking overhead seems to be in the noise.  Anything that's
> building and manipulating large complex data structures is probably
> spending a lot of its time chasing pointers and worrying much more about
> cache misses and so on than exact details of types.
> 
> As an example, I have a system (in Lisp) which parses structures from a
> stream, amd then does various manipulations on them.  It has bnasically
> no declarations in it.  I spent a little time profiling it, and someone
> else did some more work.  He found a few places where type declarations
> help some - mostly declarations that things are better-sorts-of-arrays
> so array ndexing can do better (the parser reads into buffers which are
> arrays because a lot of what it generates is strings).  We got between 1
> and 5% speedup.  We might not have found the big problems of course, but
> really, it looks like type declarations don't help here much at all.

On a similar type of program I worked on in CMUCL it would run plenty fast
at home on my 233 MHz PII and then when I took it to work and ran it on a
400 Mhz Sparq or something like that, it would be a dog.

Turned out that consing was a big problem, and the x86 had a generational
collector, whereas the Sparq had a mark-sweep.

changing some functions to be non-consing, and changing the interval
between GCs to be about equal to the expected image size made it MUCH
faster. (I was GC'ing every 2MB on an image that had about 25-30 MB of
static in-RAM data, Every 10 seconds or so I'd cons up about 100 MB, so 
GCing every 30 MB makes consing amortized to O(n) instead of O(n^2).

With the generational collector, consing was already O(n).


-- 
Daniel Lakeland
dlakelan at endpointcomputing.com
http://www.endpointcomputing.com