[Cython] [Python-Dev] C-level duck typing

[Nathaniel Smith]

On Mon, May 28, 2012 at 12:09 PM, mark florisson
<markflorisson88 at gmail.com> wrote:
> On 28 May 2012 12:01, Nathaniel Smith <njs at pobox.com> wrote:
>> On Mon, May 28, 2012 at 11:55 AM, mark florisson
>> <markflorisson88 at gmail.com> wrote:
>>> On 28 May 2012 11:41, Nathaniel Smith <njs at pobox.com> wrote:
>>>> On Mon, May 28, 2012 at 10:13 AM, mark florisson
>>>> <markflorisson88 at gmail.com> wrote:
>>>>> On 28 May 2012 09:54, mark florisson <markflorisson88 at gmail.com> wrote:
>>>>>> On 27 May 2012 23:12, Nathaniel Smith <njs at pobox.com> wrote:
>>>>>>> On Sun, May 27, 2012 at 10:24 PM, Dag Sverre Seljebotn
>>>>>>> <d.s.seljebotn at astro.uio.no> wrote:
>>>>>>>> On 05/18/2012 10:30 AM, Dag Sverre Seljebotn wrote:
>>>>>>>>>
>>>>>>>>> On 05/18/2012 12:57 AM, Nick Coghlan wrote:
>>>>>>>>>>
>>>>>>>>>> I think the main things we'd be looking for would be:
>>>>>>>>>> - a clear explanation of why a new metaclass is considered too complex a
>>>>>>>>>> solution
>>>>>>>>>> - what the implications are for classes that have nothing to do with the
>>>>>>>>>> SciPy/NumPy ecosystem
>>>>>>>>>> - how subclassing would behave (both at the class and metaclass level)
>>>>>>>>>>
>>>>>>>>>> Yes, defining a new metaclass for fast signature exchange has its
>>>>>>>>>> challenges - but it means that *our* concerns about maintaining
>>>>>>>>>> consistent behaviour in the default object model and avoiding adverse
>>>>>>>>>> effects on code that doesn't need the new behaviour are addressed
>>>>>>>>>> automatically.
>>>>>>>>>>
>>>>>>>>>> Also, I'd consider a functioning reference implementation using a custom
>>>>>>>>>> metaclass a requirement before we considered modifying type anyway, so I
>>>>>>>>>> think that's the best thing to pursue next rather than a PEP. It also
>>>>>>>>>> has the virtue of letting you choose which Python versions to target and
>>>>>>>>>> iterating at a faster rate than CPython.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This seems right on target. I could make a utility code C header for
>>>>>>>>> such a metaclass, and then the different libraries can all include it
>>>>>>>>> and handshake on which implementation becomes the real one through
>>>>>>>>> sys.modules during module initialization. That way an eventual PEP will
>>>>>>>>> only be a natural incremental step to make things more polished, whether
>>>>>>>>> that happens by making such a metaclass part of the standard library or
>>>>>>>>> by extending PyTypeObject.
>>>>>>>>
>>>>>>>>
>>>>>>>> So I finally got around to implementing this:
>>>>>>>>
>>>>>>>> https://github.com/dagss/pyextensibletype
>>>>>>>>
>>>>>>>> Documentation now in a draft in the NumFOCUS SEP repo, which I believe is a
>>>>>>>> better place to store cross-project standards like this. (The NumPy
>>>>>>>> docstring standard will be SEP 100).
>>>>>>>>
>>>>>>>> https://github.com/numfocus/sep/blob/master/sep200.rst
>>>>>>>>
>>>>>>>> Summary:
>>>>>>>>
>>>>>>>>  - No common runtime dependency
>>>>>>>>
>>>>>>>>  - 1 ns overhead per lookup (that's for the custom slot *alone*, no
>>>>>>>> fast-callable signature matching or similar)
>>>>>>>>
>>>>>>>>  - Slight annoyance: Types that want to use the metaclass must be a
>>>>>>>> PyHeapExtensibleType, to make the binary layout work with how CPython makes
>>>>>>>> subclasses from Python scripts
>>>>>>>>
>>>>>>>> My conclusion: I think the metaclass approach should work really well.
>>>>>>>
>>>>>>> Few quick comments on skimming the code:
>>>>>>>
>>>>>>> The complicated nested #ifdef for __builtin_expect could be simplified to
>>>>>>>  #if defined(__GNUC__) && (__GNUC__ > 2 || __GNUC_MINOR__ > 95)
>>>>>>>
>>>>>>> PyCustomSlots_Check should be called PyCustomSlots_CheckExact, surely?
>>>>>>> And given that, how can this code work if someone does subclass this
>>>>>>> metaclass?
>>>>>>
>>>>>> I think we should provide a wrapper for PyType_Ready, which just
>>>>>> copies the pointer to the table and the count directly into the
>>>>>> subclass. If a user then wishes to add stuff, the user can allocate a
>>>>>> new memory region dynamically, memcpy the base class' stuff in there,
>>>>>> and append some entries.
>>>>>
>>>>> Maybe we should also allow each custom type to set a deallocator,
>>>>> since they are then heap types which can go out of scope. The
>>>>> metaclass can then call this deallocator to deallocate the table.
>>>>
>>>> Custom types are plain old Python objects, they can use tp_dealloc.
>>>>
>>> If I set etp_custom_slots to something allocated on the heap, then the
>>> (shared) metaclass would have to deallocate it. The tp_dealloc of the
>>> type itself would be called for its instances (which can be used to
>>> deallocate dynamically allocated memory in the objects if you use a
>>> custom slot "pointer offset").
>>
>> Oh, I see. Right, the natural way to handle this would be have each
>> user define their own metaclass with the behavior they want. Another
>> argument for supporting multiple metaclasses simultaneously I guess...
>>
>> - N
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>
> That bludgeons your constant time type check.

Not if you steal a flag, like the interpreter already does with
Py_TPFLAGS_INT_SUBCLASS, Py_TPFLAGS_STRING_SUBCLASS, etc. I was
referring to that argument I made earlier :-)

> It's easier to just
> reserve an extra slot for a deallocator pointer :) It would probably
> be set to NULL in the common case anyway, since you allocate your
> slots statically.

-N