2.6, 3.0, and truly independent intepreters

[Jesse Noller]

On Wed, Oct 29, 2008 at 8:05 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On approximately 10/29/2008 3:45 PM, came the following characters from the
> keyboard of Patrick Stinson:
>>
>> If you are dealing with "lots" of data like in video or sound editing,
>> you would just keep the data in shared memory and send the reference
>> over IPC to the worker process. Otherwise, if you marshal and send you
>> are looking at a temporary doubling of the memory footprint of your
>> app because the data will be copied, and marshaling overhead.
>
> Right.  Sounds, and is, easy, if the data is all directly allocated by the
> application.  But when pieces are allocated by 3rd party libraries, that use
> the C-runtime allocator directly, then it becomes more difficult to keep
> everything in shared memory.
>
> One _could_ replace the C-runtime allocator, I suppose, but that could have
> some adverse effects on other code, that doesn't need its data to be in
> shared memory.  So it is somewhat between a rock and a hard place.
>
> By avoiding shared memory, such problems are sidestepped... until you run
> smack into the GIL.

If you do not have shared memory: You don't need threads, ergo: You
don't get penalized by the GIL. Threads are only useful when you need
to have that requirement of large in-memory data structures shared and
modified by a pool of workers.

-jesse