[Cython] CEP1000: Native dispatch through callables

[Nathaniel Smith]

On Sun, Apr 15, 2012 at 9:07 AM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 04/15/2012 09:30 AM, Stefan Behnel wrote:
>>
>> Dag Sverre Seljebotn, 15.04.2012 08:58:
>>>
>>> Ah, Cython objects. Didn't think of that. More below.
>>>
>>> On 04/14/2012 11:02 PM, Stefan Behnel wrote:
>>>>
>>>> thanks for writing this up. Comments inline as I read through it.
>>>>
>>>> Dag Sverre Seljebotn, 14.04.2012 21:08:
>>>>>
>>>>> each described by a function pointer and a signature specification
>>>>> string, such as "id)i" for {{{int f(int, double)}}}.
>>>>
>>>>
>>>> How do we deal with object argument types? Do we care on the caller
>>>> side?
>>>> Functions might have alternative signatures that differ in the type of
>>>> their object parameters. Or should we handle this inside of the caller
>>>> and
>>>> expect that it's something like a fused function with internal dispatch
>>>> in
>>>> that case?
>>>>
>>>> Personally, I think there is not enough to gain from object parameters
>>>> that
>>>> we should handle it on the caller side. The callee can dispatch those if
>>>> necessary.
>>>>
>>>> What about signatures that require an object when we have a C typed
>>>> value?
>>>>
>>>> What about signatures that require a C typed argument when we have an
>>>> arbitrary object value in our call parameters?
>>>>
>>>> We should also strip the "self" argument from the parameter list of
>>>> methods. That's handled by the attribute lookup before even getting at
>>>> the
>>>> callable.
>>>
>>>
>>> On 04/15/2012 07:59 AM, Robert Bradshaw wrote:
>>>>
>>>> It would certainly be useful to have special syntax for memory views
>>>> (after nailing down a well-defined ABI for them) and builtin types.
>>>> Being able to declare something as taking a
>>>> "sage.rings.integer.Integer" could also prove useful, but could result
>>>> in long (and prefix-sharing) signatures, favoring the
>>>> runtime-allocated ids.
>>>
>>>
>>> I do think describing Cython objects in this cross-tool CEP would work
>>> nicely, this is for standardized ABIs only (we can't do memoryviews
>>> either
>>> until their ABI is standard).
>>
>>
>> It just occurred to me that an object's type can safely be represented at
>> runtime as a pointer, i.e. an integer. Even if the type is heap allocated
>> and replaced by another one later, a signature that uses that pointer
>> value
>> in its encoding would only ever match if both sides talk about the same
>> type at call time (because at least one of them would hold a life
>> reference
>> to the type in order to actually use it).
>
>
> The missing piece here is that both me and Robert are huge fans of Go-style
> polymorphism. If you haven't read up on that I highly recommend it, basic
> idea is if you agree on method names and their signatures, you don't have to
> have access to the same interface declaration (you don't have to call the
> interface the same thing).

Go style polymorphism is certainly a neat idea, but two points:

- You can't do this kind of matching via signature comparison. If I
have a type with methods "foo", "bar" and "baz", then that should
match the interface {"foo", "bar", "baz"}, but also {"foo", "bar"},
{"foo", "baz"}, {"bar"}, {}, etc. To find the right function for such
a type, you need to decode each function signature and check them in
some structured way. Unless your plan is to precompute the hash of all
2**n interfaces that each object fulfills.

- Adding a whole new type system with polymorphic dispatch is a heck
of a thing to do in a spec for boxing and unboxing pointers. Honestly
at this level I'm even leery of describing Python objects via their
type, as opposed to just "PyObject *". Just let the callee do the type
checking if they need to, and if it later turns out that there are
actually enough cases where Cython knows the exact type at compile
time and is dispatching through a boxed pointer and the callee type
checking is significant overhead, then extend the spec then.

-- Nathaniel