threading support in python

[Paul Rubin]

"sjdevnull at yahoo.com" <sjdevnull at yahoo.com> writes:
> > Part of the win of programming in Python instead of C is having the
> > language do memory management for you--no more null pointers
> > dereferences or malloc/free errors.  Using shared memory puts all that
> > squarely back in your lap.
> 
> Huh?  Why couldn't you use garbage collection with objects allocated in
> shm?  The worst theoretical case is about the same programatically as
> having garbage collected objects in a multithreaded program.

I'm talking about using a module like mmap or the now-AWOL shm module,
which gives you a big shared byte array that you have to do your own
memory management in.  POSH is a slight improvement over this, since
it does its own ref counting, but that is slightly leaky, and POSH has
to marshal every object into the shared area.  

> Python doesn't actually support that as of yet, but it could.

Well, yeah, with a radically different memory system that's even
more pie in the sky than the GIL and refcount removal that we've
been discussing.

> In the interim, if the memory you're sharing is array-like then you
> can already take full advantage of multiprocess solutions in Python.

But then you're back to doing your own memory management within that
array.  Sure, that's tolerable for some applications (C programmers do
it for everything), but not exactly joy.  

And as already mentioned, the stdlib currently gives no way to
implement shared memory locks (file locks aren't the same thing).
POSH and the old shm library do, but POSH is apparently not that
reliable, and nobody knows what happened to shm.