Python Memory Manager

[Paul Rubin]

Pie Squared <PieSquared at gmail.com> writes:
> It seems to me that another, perhaps better strategy, would be to
> allocate a large heap space, then store a pointer to the base of the
> heap, the current heap size, and the beginning of the free memory.
> When you need to 'allocate' more room, just return a pointer to some
> location in the heap and increment the start-of-free-memory pointer.
> That way, allocation really IS free, more or less. Wouldn't that be
> more efficient? Perhaps I'm missing something.

The problem here is with a high allocation rate, you have to GC a lot
more often, which typically involves copying live data.  So now you
have to figure out how to reduce the amount of copying using
generational schemes, (these days) come up with ways to make all this
work in the presence of parallel threads, etc.  It sounds like you're
new to the subject, so for now I'll summarize the situation by saying
Python uses a fairly simple scheme that's a reasonable match for its
implementation as a small, medium performance interpreter; however,
higher performance GC'd language implementations end up doing much
more complicated things.  See the Jones and Lins book that I
mentioned, and there's an even older one by Appel, and the Wikipedia
article on garbage collection may have some links you can look at.
Also, Structure and Interpretation of Computer Programs (full text
online) includes a simple implementation of a copying GC, that might
be more tutorial.

> As a side note, I'm new to Usenet, so I'm not exactly sure... are
> 'tangents' like this - since this IS a Python newsgroup, after all - okay?

It's reasonably on topic, compared with last week's long digression
about the dimensional analysis of the Kessel Run.