[Cython] [cython-users] Cython .pxd introspection: listing defined constants

[W. Trevor King]

On Sat, Feb 19, 2011 at 04:41:27PM -0800, Robert Bradshaw wrote:
> On Sat, Feb 19, 2011 at 3:31 PM, W. Trevor King <wking at drexel.edu> wrote:
> > On Sat, Feb 19, 2011 at 02:04:16PM -0800, Robert Bradshaw wrote:
> >> On Sat, Feb 19, 2011 at 1:45 PM, W. Trevor King <wking at drexel.edu> wrote:
> >> > On Sat, Feb 19, 2011 at 12:47:41PM -0800, Robert Bradshaw wrote:
> >> >> On Sat, Feb 19, 2011 at 11:22 AM, W. Trevor King <wking at drexel.edu> wrote:
> >> >> > However, the filename <-> module mapping is troublesome for backing
> >> >> > externally-implemented Python modules (e.g. numpy).  If you wanted to
> >> >> > write a .pxd file backing numpy.random, how would you go about getting
> >> >> > your module installed in Cython/Includes/numpy/random.pxd or another
> >> >> > path that cython would successfully match with `cimport numpy.random`?
> >> >>
> >> >> Note that extern blocks (by definition) declare where things come from.
> >> >
> >> > They declare where the .pxd file looks for .h files, but not where
> >> > .pyx files look for the .pxd file.
> >>
> >> Sorry, I should have said extern blocks that make cdef class
> >> declarations (such as our numpy.pxd).
> >
> > It doesn't look like there are cdef class declarations in numpy.pxd:
> >
> >    cython $ grep class Cython/Includes/numpy.pxd
> >        ctypedef class numpy.dtype [object PyArray_Descr]:
> >        ctypedef extern class numpy.flatiter [object PyArrayIterObject]:
> >        ctypedef extern class numpy.broadcast [object PyArrayMultiIterObject]:
> >        ctypedef class numpy.ndarray [object PyArrayObject]:
> >        ctypedef extern class numpy.ufunc [object PyUFuncObject]:
> >
> > This still doesn't explain how .pxd files specify which external
> > implemented Python modules they correspond to.
> 
> "numpy.dtype" is the fully qualified name of the class it's
> declaring--the module is "numpy."

Hmm, that means that

    cimport numpy
    a = numpy.numpy.ndarray

also compiles.  Wouldn't it make more sense to mirror numpy's module
structure when backing it?  You do have nested .pxd modules for
cpython, libc, etc.

> >> >> >> >      cdef public struct X:
> >> >> >> >          int x
> >> >> >> >          readonly int z
> >> >> >> >          private int z
> >> >> >> >
> >> >> >> > I would perhaps say that non-Pythonable non-private members in public
> >> >> >> > structs would be a compile error.
> >> >> >>
> >> >> >> +1, keep it safe at the beginning.
> >> >> >
> >> >> > -1, keep the code clean and the interface consistent ;).  I think the
> >> >> > struct syntax should be identical to the class syntax, with the
> >> >> > exception that you can't bind methods to structs.  That's the only
> >> >> > real difference between structs and classes, isn't it?
> >> >>
> >> >> In C++, the only difference between structs and classes is that struct
> >> >> members are public by default. (Not saying that C++ is always the
> >> >> model to follow, but it gives precedent). And structs can have
> >> >> function members, that's how to do OOP in C.
> >> >
> >> > Oh.  Even more reason to have identical struct and class handling in
> >> > Cython ;).
> >> >
> >> > It is unclear to me what `cdef public struct` means.  I think it
> >> > should mean "Python bindings can alter this struct's definition",
> >> > which doesn't make sense.
> >>
> >> I think it should mean "this struct is accessible from Python (as X)"
> >
> > Wouldn't that be "cdef readonly struct X"?
> >
> >> > Shouldn't the syntax for public members be
> >> >
> >> >    cdef struct X:
> >> >        cdef public:
> >> >            int x
> >> >            readonly int y
> >> >            private int z
> >>
> >> -1 on nesting things like this. Rather than make a struct visible from
> >> Python iff any of its members are,
> >
> > A struct is visible from python iff it is declared public or readonly:
> >
> >    cdef public struct X:
> >        ...
> >
> > or
> >
> >    cdef readonly struct X:
> >        ...
> >
> > I don't think the visibility of the struct as a whole should have any
> > effect over the visibility of its members, so you should be able to
> > specify member visibility explicitly with per-member granularity (as
> > you currently can for classes).
> >
> > I was assuming that structs would be public by default (like classes),
> > but that is obviously configurable.
> >
> >> I think it makes more sense to put
> >> the declaration on the struct itself. We could support
> >>
> >> cdef public struct X:
> >>     int x # public
> >>
> >> cdef readonly struct Y:
> >>     int y # readonly
> >>
> >> cdef [private] struct Z:
> >>     int z # private, as we don't even have Z in the Python namespace,
> >> and no wrapper is created.
> >
> > The problems with this are:
> >
> > * It's differnent from how we handle the almost identical class case.
> 
> True it's different, but there are significant differences between
> classes and structs in Cython, and this is the way things are now.

Ah, well.  I suppose I'll leave the Parser pretty much alone.  Classes
and structs will end up being much closer on the Python interface
side, perhaps the syntax will grow closer in future Cythons...

> > * It makes it impossible to define, for example a public struct with
> >  C-only attributes:
> >
> >    cdef public struct X:
> >        cdef public int a
> >        cdef private void* ptr
> 
> ?
> 
> The above would work just fine. I was proposing that it would be
> semantically equivalent to
> 
>     cdef public struct X:
>         cdef int a
>         cdef private void* ptr
> 
> Perhaps that was not clear. Consider the currently valid
> 
> cdef struct X:
>     int a
>     void* ptr
> 
> We can't make everything public by default, as this would break valid
> code. (I don't think it's a good idea to make the default visibility
> of a member be a function of its type if we can help it.)

Oh, I though (for some reason) that you were ruling out member-level
visibility adjustments.  Using the struct visibility to set
member-level defaults is fine.

> BTW, the "public" keyword is the wrong thing to use here, as that
> actually controls name mangling and (c-level) symbol exporting. The
> fact that means a different thing for members than for top-level
> symbols isn't ideal, but at least it's unambiguous as members need not
> be mangled.

"public" means "read/write Python interface" for struct attributes, I
was assuming that would also apply to struct members acording to
tutorial/cdef_classes.  Should I use a different visibility name in
structs and unions?

> >> >> > If safety with a new feature is a concern, a warning like
> >> >> > "EXPERIMENTAL FEATURE" in the associated docs and compiler output
> >> >> > should be sufficient.
> >> >>
> >> >> I think the point of "safe" is to start out with a compiler error, and
> >> >> we can change our minds later, which is better than trying to make
> >> >> legal statements illegal in the future.
> >> >
> >> > Ok, but I still don't understand why the cdefs were removed from the
> >> > proposed structure members, when they are required for class
> >> > definitions.
> >>
> >> Because structs can only have c members, so the cdef was entirely
> >> redundant. They weren't really removed per say, it's just that with
> >> the exception of cdef classes, "cdef ..." meant "a c declaration
> >> follows."
> >
> > But cdef classes can also only have cdef members.
> >
> > I think it's better to keep cdef meaning "backed by C data", not
> > necessarily "written using C syntax", since you're trying to do more
> > with Cython, so it doesn't make sense to force C syntax.
> 
> It means both. Were I to start over, I would make "cdef int* a, b, c"
> declare three pointers, but we're stuck with the C syntax we have...
>
> In a struct "cdef" on members is entirely redundant

As it is in a cdef class.  If you don't want it there for struct/union
members, though, then I'll do it that way.

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20110219/5392cffb/attachment-0001.pgp>