[Cython] CEP1000: Native dispatch through callables

[Stefan Behnel]

Dag Sverre Seljebotn, 17.04.2012 14:53:
> On 04/17/2012 02:40 PM, Nathaniel Smith wrote:
>> On Tue, Apr 17, 2012 at 12:54 PM, Dag Sverre Seljebotn wrote:
>>> I don't believe doing interning right without a common dependency .so is
>>> all
>>> that easy. I'd love to see a concrete spec for it (e.g., if you use Python
>>> bytes in a dict in sys.modules['_nativecall'], the bytes objects could be
>>> deallocated before callables containing the interned string -- unless you
>>> Py_INCREF once too many, but then valgrind complains -- and so on).
>>
>> I don't understand. A C-callable object would hold a reference to each
>> interned string it contains, just like any other Python data structure
>> does. When the C-callable object is deallocated, then it drops this
>> reference, and if it there are no other live C-callable objects that
>> use the same signature, and there are no callers that are statically
>> caching the signature, then it will drop out of the intern table.
>> Otherwise it doesn't.
> 
> Thanks!, I'm just being dense.
> 
> In fact I was the one who first proposed just storing interned
> PyBytesObject* way back in the start of the thread, but it met opposition
> in favour of an interned char* or an allocated "string id"; perhaps that's
> why I shut the possibility out of my mind.
> 
> Would we store just PyBytesObject* or a char* in addition? Is bytes a
> vararg object or does it wrap a char*? If the former I think we should just
> store the PyBytesObject*.

I had originally thought that callables would include C implemented
functions, which would make this difficult for non-Cython code, because
they don't normally have destructors in CPython. If we rule that out and
restrict ourselves to callable extension types, binding the lifetime of yet
another object to their lifetime would not hurt.

Stefan