[Cython] [cython-users] Cython .pxd introspection: listing defined constants

[Robert Bradshaw]

On Sat, Feb 19, 2011 at 1:24 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

>>>>>>> A side effect of this cpdef change would be that now even bare .pxd
>>>>>>> files (no matching .pyx) would have a Python presence,
>>>>>>
>>>>>> Where would it live? Would we just create this module (in essence,
>>>>>> acting as if there was an empty .pyx file sitting there as well)? On
>>>>>> this note, it may be worth pursuing the idea of a "cython helper"
>>>>>> module where common code and objects could live.
>>>>>
>>>>> I'm not sure exactly what you mean by "cython helper", but this sounds
>>>>> like my 'bare .pyx can create a Python .so module idea above.
>>>>
>>>> I'm thinking of a place to put, e.g. the generator and bind-able
>>>> function classes, which are now re-implemented in every module that
>>>> uses them. I think there will be more cases like this in the future
>>>> rather than less. C-level code could be #included and linked from
>>>> "global" stores as well. However, that's somewhat tangential.
>
> If you generate more than one file from a .pyx, including files that are
> shared between compiler runs (or even readily built as .so files), you'd
> quickly end up in dependency hell. When to regenerate that file? With what
> compiler version and config? C/C++? With/out cleanup code? And where to put
> it? Are you sure that place will be writable for all compiler runs? And will
> the result be guaranteed to be reproducible, regardless of how many compiler
> runs there were in between?

I'd probably make it an option to the cythonize(...) command, so all
the regeneration/dependency information would be computed based on the
whole fileset rather than trying to figure that out for each
individual module compilation. I'm thinking this would be an
optimization, not a requirement.

>>>>>>> Unions don't really have a Python parallel,
>>>>>>
>>>>>> They can be a cdef class wrapping the union type.
>>>>>
>>>>> But I would think coercion would be difficult.  Unions are usually (in
>>>>> my limited experience) for "don't worry about the type, just make sure
>>>>> it fits in X bytes".  How would union->Python conversion work?
>>>>
>>>> There would be a wrapping type, e.g.
>>>>
>>>> cdef class MyUnion:
>>>>    cdef union_type value
>
> Wouldn't that have to be a pointer to the real thing instead?

Why? This would be a lot messier to get right, and structs and unions
already have pass-by-value semantics.

>>>> with a bunch of setters/getters for the values, just like there are
>>>> for structs. (In fact the same code would handle structs and unions).
>>>>
>>>> This is getting into the wrapper-generator territory, but I'm starting
>>>> to think for simple things that might be worth it.
>>>
>>> I think that if Cython will automatically generate a wrapper for
>>>
>>>    cdef public int x
>>>
>>> it should generate a wrapper for
>>>
>>>    cdef struct X: cdef public int x
>>
>> Or
>>
>>     cdef public struct X:
>>         int x
>>         readonly int z
>>         private int z
>>
>> I would perhaps say that non-Pythonable non-private members in public
>> structs would be a compile error.
>
> +1, keep it safe at the beginning.
>
>
>>> There really aren't that metatypes in C, so it doesn't seem like a
>>> slippery slope to me.  Maybe I'm just missing something...
>>>
>>>>> Ok, I think we're pretty much agreed ;).  I think that the next step
>>>>> is to start working on implementations of:
>>>>>
>>>>> * Stand alone .pxd ->  Python module
>>>>
>>>> I'm not sure we're agreed on this one.
>
> Same from here. To me, that doesn't make much sense for code that wraps a
> library. And if it doesn't wrap a library, there isn't much benefit in
> writing a stand-alone .pxd in the first place. A .pyx is much more explicit
> and obvious in this case. Especially having some .pxd files that generate
> .so files and others that don't will make this very ugly.
>
> I'd prefer adding support for cimporting from .pyx files instead,
> potentially with an automated caching generation of corresponding .pxd files
> (maybe as ".pxdg" files to make them easier to handle for users). However,
> cyclic dependencies would be tricky to handle automatically then.

We could put the in the new __pycache__.

>>>>> * Extending class cdef/cdpef/public/readonly handling to cover enums,
>>>>>  stucts, and possibly unions.
>>>>
>>>> This seems like the best first step.
>
> +1
>
>
>>>>> * I don't know how to handle things like dummy enums (perhaps by
>>>>>  requiring all cdef-ed enums to be named).
>>>>
>>>> All enums in C are named.
>>>
>>> But my Cython declaration (exposing a C `#define CONST_A 1`):
>>>
>>>    cdef extern from 'mylib.h':
>>>        enum: CONST_A
>>>
>>> is not a named enum.
>>
>> Ah, yes. Maybe we require a name (that would only be used in Python
>> space).
>
> ... require it for cpdef enums, you mean?
>
> OTOH, the "enum: NAME" scheme is ugly by itself. There should be a way to
> declare external constants correctly. After all, we loose all type
> information that way. I just saw that in math.pxd things like "M_PI" are
> declared as plain "enum" instead of "const double" or something. The type
> inferencer can't do anything with that. It might even draw the completely
> wrong conclusions.

+1 Const support would be really nice to have, especially given how
much more C++ uses it.

- Robert