[pypy-dev] Thinking about the GIL
Bob Ippolito
bob at redivi.com
Wed Mar 16 18:30:29 CET 2011
On Wed, Mar 16, 2011 at 9:13 AM, Timothy Baldridge <tbaldridge at gmail.com> wrote:
>> one alternative approach is to have a separate VM in each thread. Then pass
>> messages between them. Works well, and no GIL in each VM. You have to have
>> clean code that allows you to have a separate VMs in a process. However,
>> it's easier to make your code be able to run in separate VMs, than to recode
>> it to allow concurrent thread access to all data structures. This way is
>> easier to implement than removing a GIL.
>>
>
> This is what Erlang does, but unfortunately it scales very poorly when
> you have large datasets (as mentioned above). If you want 4 threads
> pounding on 1GB of data, either they need to ask a 5th thread for each
> data item, or each thread needs its own copy of that 1GB dataset. Even
> embarrassingly parallel things such as raytracing are very hard in
> this model.
If the data is constant you can share it in Erlang by generating a
module for it and putting it in the code server (a hack, but it works)
or simply using a binary. These are zero-copy in Erlang.
-bob
More information about the Pypy-dev
mailing list