[Cython] [Python-Dev] C-level duck typing

[mark florisson]

On 28 May 2012 11:41, Nathaniel Smith <njs at pobox.com> wrote:
> On Mon, May 28, 2012 at 10:13 AM, mark florisson
> <markflorisson88 at gmail.com> wrote:
>> On 28 May 2012 09:54, mark florisson <markflorisson88 at gmail.com> wrote:
>>> On 27 May 2012 23:12, Nathaniel Smith <njs at pobox.com> wrote:
>>>> On Sun, May 27, 2012 at 10:24 PM, Dag Sverre Seljebotn
>>>> <d.s.seljebotn at astro.uio.no> wrote:
>>>>> On 05/18/2012 10:30 AM, Dag Sverre Seljebotn wrote:
>>>>>>
>>>>>> On 05/18/2012 12:57 AM, Nick Coghlan wrote:
>>>>>>>
>>>>>>> I think the main things we'd be looking for would be:
>>>>>>> - a clear explanation of why a new metaclass is considered too complex a
>>>>>>> solution
>>>>>>> - what the implications are for classes that have nothing to do with the
>>>>>>> SciPy/NumPy ecosystem
>>>>>>> - how subclassing would behave (both at the class and metaclass level)
>>>>>>>
>>>>>>> Yes, defining a new metaclass for fast signature exchange has its
>>>>>>> challenges - but it means that *our* concerns about maintaining
>>>>>>> consistent behaviour in the default object model and avoiding adverse
>>>>>>> effects on code that doesn't need the new behaviour are addressed
>>>>>>> automatically.
>>>>>>>
>>>>>>> Also, I'd consider a functioning reference implementation using a custom
>>>>>>> metaclass a requirement before we considered modifying type anyway, so I
>>>>>>> think that's the best thing to pursue next rather than a PEP. It also
>>>>>>> has the virtue of letting you choose which Python versions to target and
>>>>>>> iterating at a faster rate than CPython.
>>>>>>
>>>>>>
>>>>>> This seems right on target. I could make a utility code C header for
>>>>>> such a metaclass, and then the different libraries can all include it
>>>>>> and handshake on which implementation becomes the real one through
>>>>>> sys.modules during module initialization. That way an eventual PEP will
>>>>>> only be a natural incremental step to make things more polished, whether
>>>>>> that happens by making such a metaclass part of the standard library or
>>>>>> by extending PyTypeObject.
>>>>>
>>>>>
>>>>> So I finally got around to implementing this:
>>>>>
>>>>> https://github.com/dagss/pyextensibletype
>>>>>
>>>>> Documentation now in a draft in the NumFOCUS SEP repo, which I believe is a
>>>>> better place to store cross-project standards like this. (The NumPy
>>>>> docstring standard will be SEP 100).
>>>>>
>>>>> https://github.com/numfocus/sep/blob/master/sep200.rst
>>>>>
>>>>> Summary:
>>>>>
>>>>>  - No common runtime dependency
>>>>>
>>>>>  - 1 ns overhead per lookup (that's for the custom slot *alone*, no
>>>>> fast-callable signature matching or similar)
>>>>>
>>>>>  - Slight annoyance: Types that want to use the metaclass must be a
>>>>> PyHeapExtensibleType, to make the binary layout work with how CPython makes
>>>>> subclasses from Python scripts
>>>>>
>>>>> My conclusion: I think the metaclass approach should work really well.
>>>>
>>>> Few quick comments on skimming the code:
>>>>
>>>> The complicated nested #ifdef for __builtin_expect could be simplified to
>>>>  #if defined(__GNUC__) && (__GNUC__ > 2 || __GNUC_MINOR__ > 95)
>>>>
>>>> PyCustomSlots_Check should be called PyCustomSlots_CheckExact, surely?
>>>> And given that, how can this code work if someone does subclass this
>>>> metaclass?
>>>
>>> I think we should provide a wrapper for PyType_Ready, which just
>>> copies the pointer to the table and the count directly into the
>>> subclass. If a user then wishes to add stuff, the user can allocate a
>>> new memory region dynamically, memcpy the base class' stuff in there,
>>> and append some entries.
>>
>> Maybe we should also allow each custom type to set a deallocator,
>> since they are then heap types which can go out of scope. The
>> metaclass can then call this deallocator to deallocate the table.
>
> Custom types are plain old Python objects, they can use tp_dealloc.
>
> - N
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

If I set etp_custom_slots to something allocated on the heap, then the
(shared) metaclass would have to deallocate it. The tp_dealloc of the
type itself would be called for its instances (which can be used to
deallocate dynamically allocated memory in the objects if you use a
custom slot "pointer offset").