[Cython] [Python-Dev] C-level duck typing

[Nathaniel Smith]

On Mon, May 28, 2012 at 11:55 AM, mark florisson
<markflorisson88 at gmail.com> wrote:
> On 28 May 2012 11:41, Nathaniel Smith <njs at pobox.com> wrote:
>> On Mon, May 28, 2012 at 10:13 AM, mark florisson
>> <markflorisson88 at gmail.com> wrote:
>>> On 28 May 2012 09:54, mark florisson <markflorisson88 at gmail.com> wrote:
>>>> On 27 May 2012 23:12, Nathaniel Smith <njs at pobox.com> wrote:
>>>>> On Sun, May 27, 2012 at 10:24 PM, Dag Sverre Seljebotn
>>>>> <d.s.seljebotn at astro.uio.no> wrote:
>>>>>> On 05/18/2012 10:30 AM, Dag Sverre Seljebotn wrote:
>>>>>>>
>>>>>>> On 05/18/2012 12:57 AM, Nick Coghlan wrote:
>>>>>>>>
>>>>>>>> I think the main things we'd be looking for would be:
>>>>>>>> - a clear explanation of why a new metaclass is considered too complex a
>>>>>>>> solution
>>>>>>>> - what the implications are for classes that have nothing to do with the
>>>>>>>> SciPy/NumPy ecosystem
>>>>>>>> - how subclassing would behave (both at the class and metaclass level)
>>>>>>>>
>>>>>>>> Yes, defining a new metaclass for fast signature exchange has its
>>>>>>>> challenges - but it means that *our* concerns about maintaining
>>>>>>>> consistent behaviour in the default object model and avoiding adverse
>>>>>>>> effects on code that doesn't need the new behaviour are addressed
>>>>>>>> automatically.
>>>>>>>>
>>>>>>>> Also, I'd consider a functioning reference implementation using a custom
>>>>>>>> metaclass a requirement before we considered modifying type anyway, so I
>>>>>>>> think that's the best thing to pursue next rather than a PEP. It also
>>>>>>>> has the virtue of letting you choose which Python versions to target and
>>>>>>>> iterating at a faster rate than CPython.
>>>>>>>
>>>>>>>
>>>>>>> This seems right on target. I could make a utility code C header for
>>>>>>> such a metaclass, and then the different libraries can all include it
>>>>>>> and handshake on which implementation becomes the real one through
>>>>>>> sys.modules during module initialization. That way an eventual PEP will
>>>>>>> only be a natural incremental step to make things more polished, whether
>>>>>>> that happens by making such a metaclass part of the standard library or
>>>>>>> by extending PyTypeObject.
>>>>>>
>>>>>>
>>>>>> So I finally got around to implementing this:
>>>>>>
>>>>>> https://github.com/dagss/pyextensibletype
>>>>>>
>>>>>> Documentation now in a draft in the NumFOCUS SEP repo, which I believe is a
>>>>>> better place to store cross-project standards like this. (The NumPy
>>>>>> docstring standard will be SEP 100).
>>>>>>
>>>>>> https://github.com/numfocus/sep/blob/master/sep200.rst
>>>>>>
>>>>>> Summary:
>>>>>>
>>>>>>  - No common runtime dependency
>>>>>>
>>>>>>  - 1 ns overhead per lookup (that's for the custom slot *alone*, no
>>>>>> fast-callable signature matching or similar)
>>>>>>
>>>>>>  - Slight annoyance: Types that want to use the metaclass must be a
>>>>>> PyHeapExtensibleType, to make the binary layout work with how CPython makes
>>>>>> subclasses from Python scripts
>>>>>>
>>>>>> My conclusion: I think the metaclass approach should work really well.
>>>>>
>>>>> Few quick comments on skimming the code:
>>>>>
>>>>> The complicated nested #ifdef for __builtin_expect could be simplified to
>>>>>  #if defined(__GNUC__) && (__GNUC__ > 2 || __GNUC_MINOR__ > 95)
>>>>>
>>>>> PyCustomSlots_Check should be called PyCustomSlots_CheckExact, surely?
>>>>> And given that, how can this code work if someone does subclass this
>>>>> metaclass?
>>>>
>>>> I think we should provide a wrapper for PyType_Ready, which just
>>>> copies the pointer to the table and the count directly into the
>>>> subclass. If a user then wishes to add stuff, the user can allocate a
>>>> new memory region dynamically, memcpy the base class' stuff in there,
>>>> and append some entries.
>>>
>>> Maybe we should also allow each custom type to set a deallocator,
>>> since they are then heap types which can go out of scope. The
>>> metaclass can then call this deallocator to deallocate the table.
>>
>> Custom types are plain old Python objects, they can use tp_dealloc.
>>
> If I set etp_custom_slots to something allocated on the heap, then the
> (shared) metaclass would have to deallocate it. The tp_dealloc of the
> type itself would be called for its instances (which can be used to
> deallocate dynamically allocated memory in the objects if you use a
> custom slot "pointer offset").

Oh, I see. Right, the natural way to handle this would be have each
user define their own metaclass with the behavior they want. Another
argument for supporting multiple metaclasses simultaneously I guess...

- N