[Cython] CEP1000: Native dispatch through callables

[Robert Bradshaw]

On Sat, Apr 14, 2012 at 11:58 PM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> Ah, Cython objects. Didn't think of that. More below.
>
>
> On 04/14/2012 11:02 PM, Stefan Behnel wrote:
>>
>> Hi,
>>
>> thanks for writing this up. Comments inline as I read through it.
>>
>> Dag Sverre Seljebotn, 14.04.2012 21:08:
>>>
>>> each described by a function pointer and a signature specification
>>>
>>> string, such as "id)i" for {{{int f(int, double)}}}.
>>
>>
>> How do we deal with object argument types? Do we care on the caller side?
>> Functions might have alternative signatures that differ in the type of
>> their object parameters. Or should we handle this inside of the caller and
>> expect that it's something like a fused function with internal dispatch in
>> that case?
>
>>
>> Personally, I think there is not enough to gain from object parameters
>> that
>> we should handle it on the caller side. The callee can dispatch those if
>> necessary.
>>
>> What about signatures that require an object when we have a C typed value?
>>
>> What about signatures that require a C typed argument when we have an
>> arbitrary object value in our call parameters?
>>
>> We should also strip the "self" argument from the parameter list of
>> methods. That's handled by the attribute lookup before even getting at the
>> callable.
>
> On 04/15/2012 07:59 AM, Robert Bradshaw wrote:
>> It would certainly be useful to have special syntax for memory views
>> (after nailing down a well-defined ABI for them) and builtin types.
>> Being able to declare something as taking a
>> "sage.rings.integer.Integer" could also prove useful, but could result
>> in long (and prefix-sharing) signatures, favoring the
>> runtime-allocated ids.
>
>
> I do think describing Cython objects in this cross-tool CEP would work
> nicely, this is for standardized ABIs only (we can't do memoryviews either
> until their ABI is standard).
>
> I think I prefer to a) exclude it now, and b) down the line we need another
> cross-tool ABI to communicate vtables, and then we could put that into this
> CEP now.
>
> I strongly believe we should go with the Go "duck-typing" approach for
> interfaces, i.e. it is not the declared name that should be compared but the
> method names and signatures.
>
> The only question that needs answering for CEP1000 is: Would this blow up
> the signature string enough that interning is the only viable option?

Exactly.

> Some strcmp solutions:
>
>  a) Hash each vtable descriptor to 160-bits, and assume the hash is unique.
> Still, a couple of interfaces would blow up the signature string a lot.
>
>  b) Modify approach B in CEP 1000 to this: If it is longer than 160 bits,
> take a full cryptographic hash, and just assume there won't be hash
> collisions (like git does). This still saves for short signature strings,
> and avoids interning at the cost of doing 160-bit comparisons.
>
> Both of these require other ways at getting at the actual string data. But I
> still like b) above better than interning.

Requiring an implementation (or at least access too) a cryptographic
hash greatly complicates the spec. (On another note, even a simple
hash as a prefix might be useful to prevent a lot of false partial
matches, e.g. "sage.rings...") 160 * n bits starts to get large too
(and we'd have to twiddle them to insert/avoid a "dash" ever 16
bytes).

Here's a crazy thought: we could assume signatures like this are
"application specific." We can partition up portions of the signature
space to individual projects to compute however they want. Cython can
do this via interning for those signatures containing Cython types
(which is not an undue burden for anyone attempting to interoperate
with Cython types). For (some superset of) the basic C types we agree
on a common encoding and inline it.

- Robert