[Cython] Fused Types

[mark florisson]

On 2 May 2011 18:24, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> On Sun, May 1, 2011 at 2:38 AM, mark florisson
> <markflorisson88 at gmail.com> wrote:
>> On 30 April 2011 09:51, Dag Sverre Seljebotn <d.s.seljebotn at astro.uio.no> wrote:
>>> On 04/30/2011 08:39 AM, Robert Bradshaw wrote:
>>>>
>>>> On Fri, Apr 29, 2011 at 3:53 AM, mark florisson
>>>> <markflorisson88 at gmail.com>  wrote:
>>>>>
>>>>> On 29 April 2011 12:28, Pauli Virtanen<pav at iki.fi>  wrote:
>>>>>>
>>>>>> No, just that real_t is specialized to float whenever struct_t is
>>>>>> specialized
>>>>>> to A and to double when B. Or a more realistic example,
>>>>>>
>>>>>>        ctypedef cython.fused_type(float, double) real_t
>>>>>>        ctypedef cython.fused_type(float complex, double complex)
>>>>>> complex_t
>>>>>>
>>>>>>        cdef real_plus_one(complex_t a):
>>>>>>            real_t b = a.real
>>>>>>            return b + 1
>>>>>>
>>>>>> which I suppose would not be a very unusual thing in numerical codes.
>>>>>> This would also allow writing the case you had earlier as
>>>>>>
>>>>>>        cdef cython.fused_type(string_t, int, paired=struct_t) attr_t
>>>>>>
>>>>>>        cdef func(struct_t mystruct, int i):
>>>>>>            cdef attr_t var
>>>>>>
>>>>>>            if typeof(mystruct) is typeof(int):
>>>>>>                var = mystruct.attrib + i
>>>>>>                ...
>>>>>>            else:
>>>>>>                var = mystruct.attrib + i
>>>>>>                ...
>>>>>>
>>>>>> Things would need to be done explicitly instead of implicitly, though,
>>>>>> but it would remove the need for any special handling of
>>>>>> the "complex" keyword.
>>>>
>>>> If we're going to introduce pairing, another option would be
>>>>
>>>>     ctypedef fused_type((double complex, double), (float complex,
>>>> float)) (complex_t, real_t)
>>>>
>>>> though I'm not sure I like that either. We're not trying to create the
>>>> all-powerful templating system here, and anything that can be done
>>>> with pairing can be done (though less elegantly) via branching on the
>>>> types, or, as Pauli mentions, using a wider type is often (but not
>>>> always) a viable option.
>>>
>>> Keeping the right balance is difficult. But, at least there's some cases of
>>> needing this in various codebases when interfacing with LAPACK.
>>>
>>> Most uses of templating with Cython code I've seen so far does a similar
>>> kind of "zip" as what you have above (as we discussed on the workshop). So
>>> at least the usage pattern you write above is very common.
>>>
>>> float32 is not about to disappear, it really is twice as fast when you're
>>> memory IO bound.
>>>
>>> Using a wider type is actually quite often not possible; any time the type
>>> is involved as the base type of an array it is not possible, and that's a
>>> pretty common case.
>>
>> Well, if the array is passed into the function directly (and not e.g.
>> as an attribute of something passed in), then you can just write
>> 'my_fused_type *array' or 'my_fused_type array[]', and the base type
>> will be available as 'my_fused_type'.
>>
>>> (With LAPACK you take the address of the variable and
>>> pass it to Fortran, so using a wider type is not possible there either,
>>> although I'll agree that's a more remote case.)
>>>
>>> My proposal: Don't support either "real_t complex" or paired fused types for
>>> the time being. Then see.
>>
>> Ok, sounds good.
>>
>>> But my vote is for paired fused types instead of "real_t complex".
>>>
>>> Dag Sverre
>>> _______________________________________________
>>> cython-devel mailing list
>>> cython-devel at python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>
>> A remaining issue which I'm not quite certain about is the
>> specialization through subscripts, e.g. func[double]. How should this
>> work from Python space (assuming cpdef functions)? Would we want to
>> pass in cython.double etc? Because it would only work for builtin
>> types, so what about types that aren't exposed to Python but can still
>> be coerced to and from Python? Perhaps it would be better to pass in
>> strings instead. I also think e.g. "int *" reads better than
>> cython.pointer(cython.int).
>
> That's whey we offer cython.p_int. On that note, we should support
> cython.astype("int *") or something like that. Generally, I don't like
> encoding semantic information in strings.
>
> OTHO, since it'll be a mapping of some sort, there's no reason we
> can't support both. Most of the time it should dispatch (at runtime or
> compile time) based on the type of the arguments.

If we have an argument type that is composed of a fused type, would be
want the indexing to specify the composed type or the fused type? e.g.

ctypedef floating *floating_p

cdef func(floating_p x):
    ...

Then do we want

    func[double](10.0)

or

    func[double_p](10.0)

to specialize func? FYI, the type checking works like 'double_p is
floating_p' and not 'double is floating_p'. But for functions this is
a little different. On the one hand specifying the full types
(double_p) makes sense as you're kind of specifying a signature, but
on the other hand you're specializing fused types and you don't care
how they are composed -- especially if they occur multiple times with
different composition. So I'm thinking we want 'func[double]'.

>> It also sounds bad to rely on objects from the Shadow module, as
>> people using Cython modules may not have Cython (or the Shadow module
>> shipped as cython) installed. And what happens if they import it under
>> a different name and we import another cython module?
>
> The (vague) idea is that one could ship as much of the cython shadow
> module with your own code as needed to run without Cython.
>
>> Perhaps the
>> specializations of the signature could also be exposed on the function
>> object, so users can see which ones are available.
>
> Sure.
>
> - Robert
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>