[Cython] CEP1000: Native dispatch through callables

[Robert Bradshaw]

On Fri, Apr 13, 2012 at 1:27 PM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> Ah, I didn't think about 6-bit or huffman. Certainly helps.

Yeah, we don't want to complicate the ABI too much, but I think
something like 8 4-bit common chars and 32 6-bit other chars (or 128
8-bit other chars) wouldn't be outrageous. The fact that we only have
to encode into a single word makes the algorithm very simple (though
the majority of the time we'd spit out pre-encoded literals). We have
a version number to play with this as well.

> I'm almost +1 on your proposal now, but a couple of more ideas:
>
> 1) Let the key (the size_t) spill over to the next specialization entry if
> it is too large; and prepend that key with a continuation code (two size-ts
> could together say "iii)-d\0\0" on 32 bit systems with 8bit encoding, using
> - as continuation). The key-based caller will expect a continuation if it
> knows about the specialization, and the prepended char will prevent spurios
> matches against the overspilled slot.
>
> We could even use the pointers for part of the continuation...
>
> 2) Separate the char* format strings from the keys, ie this memory layout:
>
> Version,nslots,nspecs,funcptr,key,funcptr,key,...,sigcharptr,sigcharptr...
>
> Where nslots is larger than nspecs if there are continuations.
>
> OK, this is getting close to my original proposal, but the difference is the
> contiunation char, so that if you expect a short signature, you can safely
> scan every slot and branching and no null-checking necesarry.

I don't think we need nslots (though it might be interesting). My
thought is that once you start futzing with variable-length keys, you
might as well just compare char*s.

If one is concerned about memory, one could force the sigcharptr to be
aligned, and then the "keys" could be either sigcharptr or key
depending on whether the least significant bit was set. One could
easily scan for/switch on a key and scanning for a char* would be
almost as easy (just don't dereference if the lsb is set).

I don't see us being memory constrained, so

(version,nspecs,futureuse),(key,sigcharptr,funcptr)*,optionalsigchardata*

seems fine to me even if only one of key/sigchrptr is ever used per
spec. Null-terminating the specs would work fine as well (one less
thing to keep track of during iteration).

- Robert