[Cython] CEP1000: Native dispatch through callables

[Dag Sverre Seljebotn]

On 04/14/2012 08:10 PM, Robert Bradshaw wrote:
> On Sat, Apr 14, 2012 at 1:56 AM, Stefan Behnel<stefan_ml at behnel.de>  wrote:
>> Dag Sverre Seljebotn, 14.04.2012 10:41:
>>> Greg Ewing wrote:
>>>> Dag Sverre Seljebotn wrote:
>>>>
>>>>> 1) It doesn't work well with multiple interpreter states. Ok, nothing
>>>>> works with that at the moment, but it is on the roadmap for Python
>>>>
>>>> Is it really? I got the impression that it's not considered feasible,
>>>> since it would require massive changes to the entire implementation
>>>> and totally break the existing C API. Has someone thought of a way
>>>> around those problems?
>>>
>>> I was just referring to the offhand comments in PEP3121, but I guess that PEP had multiple reasons, and perhaps this particular arguøent had no significance...
>>
>> IIRC, the last status was that even after this PEP, Py3 still has serious
>> issues with keeping extension modules in separate interpreters. And this
>> probably isn't worth doing anything about because it won't work without a
>> major effort in all sorts of places. And I never heard that any extension
>> module even tried to support this.
>>
>> I don't think we should invest too much thought into this direction.

A shame; short of getting rid of the GIL, multiple interpreter states 
would be my favourite shared-memory parallel computation approach, as 
they could share NumPy buffers (and other C-level structures), without 
worrying about allocating in process-shared memory (and most data 
structures, like std::map, won't even work (or, work portably and 
reliably) in process-shared memory anyway).

Multiple seperate interpreter states would be a very nice way of getting 
the benefits of multi-threading without the disadvantages.

>
> I had never even heard of this PEP before this thread, but this
> certainly seems reasonable to me. Aside from this, there is some value
> with the inlined signature in that a pure C library can easily support
> the ABI as well.

Yes -- I think both "sides" of this discussion prefer their approach out 
of aesthetics more than performance :-)

I'll post a revamped CEP in a minute to at least try to sum them up.

> Has anyone done any experiments/timings to see if having constants vs.
> globals even matters?

It'd be interesting to see; won't have time myself until Monday earliest.

Dag