From dw-git at d-woods.co.uk Thu Mar 12 11:11:57 2020 From: dw-git at d-woods.co.uk (da-woods) Date: Thu, 12 Mar 2020 15:11:57 +0000 Subject: [Cython] Auto-generation of wrapper types Message-ID: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> The process of wrapping a C struct or C++ class in an extension type often has the user doing a pretty mechanical duplication of attributes/functions that Cython already knows about. I'm looking at doing: cdef struct S: ??? int a ??? # etc. then `cython.autowrap[S]` would create an extension type that wraps S by value. All attributes convertible to/from a Python type gets a property (as well as any attribute that has an has an autowrap declared). For `cppclass` this would extend to member functions as well - this obviously gets more involved, but again the same basic rule applies of only including stuff with an obvious conversion. I wouldn't propose to deal with the C++ nightmare of "how to return an owned reference". The idea would be to copy by value or nothing. I'd also propose only minor customization via keyword arguments (for example the name of a cdef staticmethod constructor, the name of the "obj" field in the C++ class). The basic rule would be that if it doesn't do what you want then it's an extension type, so you can always inherit from it and define the missing bits. Obviously structs have already have an automatic conversion to/from dicts and some cppclasses have autoconversions too. This wouldn't aim to replace that - it'd just be an option that the user could explicitly ask for. I have a somewhat working prototype for the struct side. Obviously the real complications are on the C++ side, but I don't think it's hugely difficult providing you accept there's lots of stuff that can't be guessed and inheritance is the way round that. Does this sound reasonable/something that'd be accepted? Any other thoughts? David From frank.schlimbach at intel.com Fri Mar 13 05:56:23 2020 From: frank.schlimbach at intel.com (Schlimbach, Frank) Date: Fri, 13 Mar 2020 09:56:23 +0000 Subject: [Cython] Auto-generation of wrapper types In-Reply-To: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> References: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> Message-ID: I like automatism, this one looks useful. How would you allow extending the Extension type with a new attribute or method? I'd expect that in many cases you need to do more than just wrapping the C class/struct. -----Original Message----- From: cython-devel On Behalf Of da-woods Sent: Thursday, March 12, 2020 4:12 PM To: cython-devel at python.org Subject: [Cython] Auto-generation of wrapper types The process of wrapping a C struct or C++ class in an extension type often has the user doing a pretty mechanical duplication of attributes/functions that Cython already knows about. I'm looking at doing: cdef struct S: ??? int a ??? # etc. then `cython.autowrap[S]` would create an extension type that wraps S by value. All attributes convertible to/from a Python type gets a property (as well as any attribute that has an has an autowrap declared). For `cppclass` this would extend to member functions as well - this obviously gets more involved, but again the same basic rule applies of only including stuff with an obvious conversion. I wouldn't propose to deal with the C++ nightmare of "how to return an owned reference". The idea would be to copy by value or nothing. I'd also propose only minor customization via keyword arguments (for example the name of a cdef staticmethod constructor, the name of the "obj" field in the C++ class). The basic rule would be that if it doesn't do what you want then it's an extension type, so you can always inherit from it and define the missing bits. Obviously structs have already have an automatic conversion to/from dicts and some cppclasses have autoconversions too. This wouldn't aim to replace that - it'd just be an option that the user could explicitly ask for. I have a somewhat working prototype for the struct side. Obviously the real complications are on the C++ side, but I don't think it's hugely difficult providing you accept there's lots of stuff that can't be guessed and inheritance is the way round that. Does this sound reasonable/something that'd be accepted? Any other thoughts? David _______________________________________________ cython-devel mailing list cython-devel at python.org https://mail.python.org/mailman/listinfo/cython-devel Intel Deutschland GmbH Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany Tel: +49 89 99 8853-0, www.intel.de Managing Directors: Christin Eisenschmid, Gary Kershaw Chairperson of the Supervisory Board: Nicole Lau Registered Office: Munich Commercial Register: Amtsgericht Muenchen HRB 186928 From dw-git at d-woods.co.uk Fri Mar 13 07:02:12 2020 From: dw-git at d-woods.co.uk (da-woods) Date: Fri, 13 Mar 2020 11:02:12 +0000 Subject: [Cython] Auto-generation of wrapper types In-Reply-To: References: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> Message-ID: <8e0b2dc6-43ae-012e-2947-d62736824d5a@d-woods.co.uk> (Sorry, originally replied just to Frank not to the list) If I understand you correctly it'd be something like: cdef class WrappingOfS(autowrap[S]): ?? def new_method(self): ????? # implementation i.e. if you want to do something the automation can't do then you just inherit. If I don't understand you correctly then please clarify... On 13/03/2020 09:56, Schlimbach, Frank wrote: > I like automatism, this one looks useful. > > How would you allow extending the Extension type with a new attribute or method? I'd expect that in many cases you need to do more than just wrapping the C class/struct. > > -----Original Message----- > From: cython-devel On Behalf Of da-woods > Sent: Thursday, March 12, 2020 4:12 PM > To: cython-devel at python.org > Subject: [Cython] Auto-generation of wrapper types > > The process of wrapping a C struct or C++ class in an extension type often has the user doing a pretty mechanical duplication of attributes/functions that Cython already knows about. I'm looking at doing: > > > cdef struct S: > > ??? int a > > ??? # etc. > > > then `cython.autowrap[S]` would create an extension type that wraps S by value. All attributes convertible to/from a Python type gets a property (as well as any attribute that has an has an autowrap declared). For `cppclass` this would extend to member functions as well - this obviously gets more involved, but again the same basic rule applies of only including stuff with an obvious conversion. > > I wouldn't propose to deal with the C++ nightmare of "how to return an owned reference". The idea would be to copy by value or nothing. > > I'd also propose only minor customization via keyword arguments (for example the name of a cdef staticmethod constructor, the name of the "obj" field in the C++ class). The basic rule would be that if it doesn't do what you want then it's an extension type, so you can always inherit from it and define the missing bits. > > Obviously structs have already have an automatic conversion to/from dicts and some cppclasses have autoconversions too. This wouldn't aim to replace that - it'd just be an option that the user could explicitly ask for. > > I have a somewhat working prototype for the struct side. Obviously the real complications are on the C++ side, but I don't think it's hugely difficult providing you accept there's lots of stuff that can't be guessed and inheritance is the way round that. > > Does this sound reasonable/something that'd be accepted? Any other thoughts? > > > David > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel > Intel Deutschland GmbH > Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany > Tel: +49 89 99 8853-0, www.intel.de > Managing Directors: Christin Eisenschmid, Gary Kershaw > Chairperson of the Supervisory Board: Nicole Lau > Registered Office: Munich > Commercial Register: Amtsgericht Muenchen HRB 186928 > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel From frank.schlimbach at intel.com Fri Mar 13 07:29:41 2020 From: frank.schlimbach at intel.com (Schlimbach, Frank) Date: Fri, 13 Mar 2020 11:29:41 +0000 Subject: [Cython] Auto-generation of wrapper types In-Reply-To: <8e0b2dc6-43ae-012e-2947-d62736824d5a@d-woods.co.uk> References: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> <8e0b2dc6-43ae-012e-2947-d62736824d5a@d-woods.co.uk> Message-ID: Right. Sorry, not sure what I was thinking. Instead of a staticmethod constructor, maybe it's more useful to have a keyword arg for the name of the generated extension class? -----Original Message----- From: cython-devel On Behalf Of da-woods Sent: Friday, March 13, 2020 12:02 PM To: cython-devel at python.org Subject: Re: [Cython] Auto-generation of wrapper types (Sorry, originally replied just to Frank not to the list) If I understand you correctly it'd be something like: cdef class WrappingOfS(autowrap[S]): ?? def new_method(self): ????? # implementation i.e. if you want to do something the automation can't do then you just inherit. If I don't understand you correctly then please clarify... On 13/03/2020 09:56, Schlimbach, Frank wrote: > I like automatism, this one looks useful. > > How would you allow extending the Extension type with a new attribute or method? I'd expect that in many cases you need to do more than just wrapping the C class/struct. > > -----Original Message----- > From: cython-devel > On Behalf > Of da-woods > Sent: Thursday, March 12, 2020 4:12 PM > To: cython-devel at python.org > Subject: [Cython] Auto-generation of wrapper types > > The process of wrapping a C struct or C++ class in an extension type often has the user doing a pretty mechanical duplication of attributes/functions that Cython already knows about. I'm looking at doing: > > > cdef struct S: > > ??? int a > > ??? # etc. > > > then `cython.autowrap[S]` would create an extension type that wraps S by value. All attributes convertible to/from a Python type gets a property (as well as any attribute that has an has an autowrap declared). For `cppclass` this would extend to member functions as well - this obviously gets more involved, but again the same basic rule applies of only including stuff with an obvious conversion. > > I wouldn't propose to deal with the C++ nightmare of "how to return an owned reference". The idea would be to copy by value or nothing. > > I'd also propose only minor customization via keyword arguments (for example the name of a cdef staticmethod constructor, the name of the "obj" field in the C++ class). The basic rule would be that if it doesn't do what you want then it's an extension type, so you can always inherit from it and define the missing bits. > > Obviously structs have already have an automatic conversion to/from dicts and some cppclasses have autoconversions too. This wouldn't aim to replace that - it'd just be an option that the user could explicitly ask for. > > I have a somewhat working prototype for the struct side. Obviously the real complications are on the C++ side, but I don't think it's hugely difficult providing you accept there's lots of stuff that can't be guessed and inheritance is the way round that. > > Does this sound reasonable/something that'd be accepted? Any other thoughts? > > > David > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel > Intel Deutschland GmbH > Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany > Tel: +49 89 99 8853-0, www.intel.de > Managing Directors: Christin Eisenschmid, Gary Kershaw Chairperson of > the Supervisory Board: Nicole Lau Registered Office: Munich Commercial > Register: Amtsgericht Muenchen HRB 186928 > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel _______________________________________________ cython-devel mailing list cython-devel at python.org https://mail.python.org/mailman/listinfo/cython-devel Intel Deutschland GmbH Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany Tel: +49 89 99 8853-0, www.intel.de Managing Directors: Christin Eisenschmid, Gary Kershaw Chairperson of the Supervisory Board: Nicole Lau Registered Office: Munich Commercial Register: Amtsgericht Muenchen HRB 186928 From dw-git at d-woods.co.uk Fri Mar 13 07:42:57 2020 From: dw-git at d-woods.co.uk (da-woods) Date: Fri, 13 Mar 2020 11:42:57 +0000 Subject: [Cython] Auto-generation of wrapper types In-Reply-To: References: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> <8e0b2dc6-43ae-012e-2947-d62736824d5a@d-woods.co.uk> Message-ID: The staticmethod constructor is useful I think - regular constructors can't take C/C++ types so the static method constructor provides a means of doing something like: ??? cdef mycppclass inst = # some call to cpp ??? return wrapperclass.create(inst) I definitely agree you want some means of naming the generated class. At the minute I'm my thoughts are to do two things: 1. Having a cache of the generated classes (so autowrap[S] always returns the exact same class if you've already created one). That'd mean there's no penalty for just always referring to it as autowrap[S], so you don't /really/ need to name it. 2. Allow something like `ctypedef autowrap[S] whatever_name` (which doesn't currently look to work...) With that said, adding an option to override the name would be pretty easy, and probably useful. As always, it's a fine balance is always between making sure you have the useful options and ending up in a huge option overload so no-one can work out where to start. On 13/03/2020 11:29, Schlimbach, Frank wrote: > Right. Sorry, not sure what I was thinking. > > Instead of a staticmethod constructor, maybe it's more useful to have a keyword arg for the name of the generated extension class? > > -----Original Message----- > From: cython-devel On Behalf Of da-woods > Sent: Friday, March 13, 2020 12:02 PM > To: cython-devel at python.org > Subject: Re: [Cython] Auto-generation of wrapper types > > (Sorry, originally replied just to Frank not to the list) > > If I understand you correctly it'd be something like: > > cdef class WrappingOfS(autowrap[S]): > ?? def new_method(self): > ????? # implementation > > i.e. if you want to do something the automation can't do then you just inherit. > > If I don't understand you correctly then please clarify... > > On 13/03/2020 09:56, Schlimbach, Frank wrote: >> I like automatism, this one looks useful. >> >> How would you allow extending the Extension type with a new attribute or method? I'd expect that in many cases you need to do more than just wrapping the C class/struct. >> >> -----Original Message----- >> From: cython-devel >> On Behalf >> Of da-woods >> Sent: Thursday, March 12, 2020 4:12 PM >> To: cython-devel at python.org >> Subject: [Cython] Auto-generation of wrapper types >> >> The process of wrapping a C struct or C++ class in an extension type often has the user doing a pretty mechanical duplication of attributes/functions that Cython already knows about. I'm looking at doing: >> >> >> cdef struct S: >> >> ??? int a >> >> ??? # etc. >> >> >> then `cython.autowrap[S]` would create an extension type that wraps S by value. All attributes convertible to/from a Python type gets a property (as well as any attribute that has an has an autowrap declared). For `cppclass` this would extend to member functions as well - this obviously gets more involved, but again the same basic rule applies of only including stuff with an obvious conversion. >> >> I wouldn't propose to deal with the C++ nightmare of "how to return an owned reference". The idea would be to copy by value or nothing. >> >> I'd also propose only minor customization via keyword arguments (for example the name of a cdef staticmethod constructor, the name of the "obj" field in the C++ class). The basic rule would be that if it doesn't do what you want then it's an extension type, so you can always inherit from it and define the missing bits. >> >> Obviously structs have already have an automatic conversion to/from dicts and some cppclasses have autoconversions too. This wouldn't aim to replace that - it'd just be an option that the user could explicitly ask for. >> >> I have a somewhat working prototype for the struct side. Obviously the real complications are on the C++ side, but I don't think it's hugely difficult providing you accept there's lots of stuff that can't be guessed and inheritance is the way round that. >> >> Does this sound reasonable/something that'd be accepted? Any other thoughts? >> >> >> David >> >> _______________________________________________ >> cython-devel mailing list >> cython-devel at python.org >> https://mail.python.org/mailman/listinfo/cython-devel >> Intel Deutschland GmbH >> Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany >> Tel: +49 89 99 8853-0, www.intel.de >> Managing Directors: Christin Eisenschmid, Gary Kershaw Chairperson of >> the Supervisory Board: Nicole Lau Registered Office: Munich Commercial >> Register: Amtsgericht Muenchen HRB 186928 >> _______________________________________________ >> cython-devel mailing list >> cython-devel at python.org >> https://mail.python.org/mailman/listinfo/cython-devel > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel > Intel Deutschland GmbH > Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany > Tel: +49 89 99 8853-0, www.intel.de > Managing Directors: Christin Eisenschmid, Gary Kershaw > Chairperson of the Supervisory Board: Nicole Lau > Registered Office: Munich > Commercial Register: Amtsgericht Muenchen HRB 186928 > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan_ml at behnel.de Sun Mar 15 04:14:08 2020 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 15 Mar 2020 09:14:08 +0100 Subject: [Cython] Auto-generation of wrapper types In-Reply-To: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> References: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> Message-ID: <1d7f1e0e-db3e-77d2-a1ae-239ac9c4fd6a@behnel.de> da-woods schrieb am 12.03.20 um 16:11: > The process of wrapping a C struct or C++ class in an extension type often > has the user doing a pretty mechanical duplication of attributes/functions > that Cython already knows about. I'm looking at doing: > > cdef struct S: > ??? int a > ??? # etc. > > then `cython.autowrap[S]` would create an extension type that wraps S by > value. Sounds like a good idea in general. Questions: What makes C structs and C++ classes special enough to make this a separate language feature? That should be part of a motivation somewhere (probably in a ticket). How does this relate to the support for @dataclass in GH-2903? Is the only difference that you would write "x.member" instead of "x.structattr.member"? https://github.com/cython/cython/issues/2903 Why "autowrap[S]" and not "autowrap(S)"? I'm asking also because there should probably be a way for this to work in Python files, for which a function call seems more natural. It also feels like a @decorator would somehow fit quite well here. Even when defined in a .pxd file, the struct would still only be exported in the corresponding module (with the same name). Something like this: @cclass cdef struct S: int x Not sure if reusing @cclass is a good idea, but it doesn't seem wrong. Defining this in a .pxd file could then even allow cimporting modules to understand the extension type wrapper as well, by simply reconstructing the type representation on their side. Stefan From dw-git at d-woods.co.uk Sun Mar 15 16:34:29 2020 From: dw-git at d-woods.co.uk (da-woods) Date: Sun, 15 Mar 2020 20:34:29 +0000 Subject: [Cython] Auto-generation of wrapper types In-Reply-To: <1d7f1e0e-db3e-77d2-a1ae-239ac9c4fd6a@behnel.de> References: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> <1d7f1e0e-db3e-77d2-a1ae-239ac9c4fd6a@behnel.de> Message-ID: <4d33989d-1c44-fe73-fa47-c5fc87ac99a8@d-woods.co.uk> On 15/03/2020 08:14, Stefan Behnel wrote: > What makes C structs and C++ classes special enough to make this a separate > language feature? That should be part of a motivation somewhere (probably > in a ticket Most of the other major things you could want to wrap are covered through other mechanisms - you can create a wrapper round a `cdef extern` function or enum with `cpdef`. The basic C types (int, float) etc largely have a direct Python equivalent that converts freely. This is mainly targetted at C++ classes which seem to be the last major feature that doesn't have this kind of wrapper. C structs just seemed easy to deal with by the same mechanism - although they obviously have the conversion to/from dict, which is fine but does involve a conversion, and so doesn't propagate modifications back. I'll create a ticket fairly shortly. > How does this relate to the support for @dataclass in GH-2903? Is the only > difference that you would write "x.member" instead of "x.structattr.member"? It's slightly difficult to answer that because we don't know exactly what @dataclass will look like in Cython. In Python it's mostly about generating `__init__`, `__repr__`, `__eq__` etc from a "struct-like" description. The Cython version I've submitted basically just follows the Python scheme, but it isn't obvious how Python-level access to attributes should work. "x.structattr" would probably end up doing the usual Cython conversion to/from a dict. This feature is mostly for creating a quick wrapper for some external C/C++ thing that already exists. For a struct it probably isn't too different from a dataclass. For a C++ class it should hopefully be able to fill in a lot of the functions too. > Why "autowrap[S]" and not "autowrap(S)"? I'm asking also because there > should probably be a way for this to work in Python files, for which a > function call seems more natural. > > It also feels like a @decorator would somehow fit quite well here. Even > when defined in a .pxd file, the struct would still only be exported in the > corresponding module (with the same name). Something like this: > > @cclass > cdef struct S: > int x > > Not sure if reusing @cclass is a good idea, but it doesn't seem wrong. Definitely open to different syntax - now you point it out I think a function call might be better than an index (especially with optional keyword arguments to control it). One other option (for consistency with how functions/enums are already handled) would be `cpdef struct S/cpdef cppclass ClassName`. The difficulty here is that it overrides the name which I suspect could be a challenge (in most cases you'd probably want to access the original definition and the wrapper extension type separately in Cython). This might be an issue with a decorator too. > Defining this in a .pxd file could then even allow cimporting modules to > understand the extension type wrapper as well, by simply reconstructing the > type representation on their side. I hadn't thought of this, but yes. This should be pretty simple - the type representation is really just: cdef class SomeName: ??? cdef c_type* obj ??? @staticmethod ??? cdef factor_func(obj x) # possibly reference or r-value here reference.... (Almost) everything else it defines is a Python interface of def functions and properties so a cimporting module would have no special knowledge - for fast access the user would go through obj. David > > Stefan > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel From robertwb at gmail.com Sat Mar 21 14:47:49 2020 From: robertwb at gmail.com (Robert Bradshaw) Date: Sat, 21 Mar 2020 11:47:49 -0700 Subject: [Cython] Auto-generation of wrapper types In-Reply-To: <4d33989d-1c44-fe73-fa47-c5fc87ac99a8@d-woods.co.uk> References: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> <1d7f1e0e-db3e-77d2-a1ae-239ac9c4fd6a@behnel.de> <4d33989d-1c44-fe73-fa47-c5fc87ac99a8@d-woods.co.uk> Message-ID: I am in general of more automated wrapping. Two things that make C++ classes more difficulty are. (1) How to handle inheritance (including multiple inheritance)? Would the wrapped types mirror the inheritance? (2) We can do auto-conversion of most types because they are passed by value. This breaks down a bit with char*, but generally we assume we can take a copy. Similarly with structs. However C++ classes are (in practice) much more stateful and this gets into all sorts of thorny issues with what the conventions should be about ownership and common references. There's also a fair number of existing libraries out there that specifically target wrapping C++ libraries (often directly from the source files). As well as making it easier to wrap from C++ classes from Cython, it'd be good to have more seamless (and efficient) integration with those. On Sun, Mar 15, 2020 at 1:34 PM da-woods wrote: > > On 15/03/2020 08:14, Stefan Behnel wrote: > > What makes C structs and C++ classes special enough to make this a separate > > language feature? That should be part of a motivation somewhere (probably > > in a ticket > > Most of the other major things you could want to wrap are covered > through other mechanisms - you can create a wrapper round a `cdef > extern` function or enum with `cpdef`. The basic C types (int, float) > etc largely have a direct Python equivalent that converts freely. > > This is mainly targetted at C++ classes which seem to be the last major > feature that doesn't have this kind of wrapper. C structs just seemed > easy to deal with by the same mechanism - although they obviously have > the conversion to/from dict, which is fine but does involve a > conversion, and so doesn't propagate modifications back. > > I'll create a ticket fairly shortly. > > > How does this relate to the support for @dataclass in GH-2903? Is the only > > difference that you would write "x.member" instead of "x.structattr.member"? > > It's slightly difficult to answer that because we don't know exactly > what @dataclass will look like in Cython. In Python it's mostly about > generating `__init__`, `__repr__`, `__eq__` etc from a "struct-like" > description. The Cython version I've submitted basically just follows > the Python scheme, but it isn't obvious how Python-level access to > attributes should work. "x.structattr" would probably end up doing the > usual Cython conversion to/from a dict. > > This feature is mostly for creating a quick wrapper for some external > C/C++ thing that already exists. For a struct it probably isn't too > different from a dataclass. For a C++ class it should hopefully be able > to fill in a lot of the functions too. > > > Why "autowrap[S]" and not "autowrap(S)"? I'm asking also because there > > should probably be a way for this to work in Python files, for which a > > function call seems more natural. > > > > It also feels like a @decorator would somehow fit quite well here. Even > > when defined in a .pxd file, the struct would still only be exported in the > > corresponding module (with the same name). Something like this: > > > > @cclass > > cdef struct S: > > int x > > > > Not sure if reusing @cclass is a good idea, but it doesn't seem wrong. > > Definitely open to different syntax - now you point it out I think a > function call might be better than an index (especially with optional > keyword arguments to control it). > > One other option (for consistency with how functions/enums are already > handled) would be `cpdef struct S/cpdef cppclass ClassName`. The > difficulty here is that it overrides the name which I suspect could be a > challenge (in most cases you'd probably want to access the original > definition and the wrapper extension type separately in Cython). This > might be an issue with a decorator too. > > > Defining this in a .pxd file could then even allow cimporting modules to > > understand the extension type wrapper as well, by simply reconstructing the > > type representation on their side. > > I hadn't thought of this, but yes. This should be pretty simple - the > type representation is really just: > > cdef class SomeName: > cdef c_type* obj > @staticmethod > cdef factor_func(obj x) # possibly reference or r-value here > reference.... > > (Almost) everything else it defines is a Python interface of def > functions and properties so a cimporting module would have no special > knowledge - for fast access the user would go through obj. > > David > > > > > > Stefan > > _______________________________________________ > > cython-devel mailing list > > cython-devel at python.org > > https://mail.python.org/mailman/listinfo/cython-devel > > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel From stefan_ml at behnel.de Wed Mar 25 04:10:15 2020 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 25 Mar 2020 09:10:15 +0100 Subject: [Cython] Preparing the first alpha release of Cython 3.0 Message-ID: Hi all, I'm curently preparing a first alpha release for Cython 3.0 (alpha 1). Please ping me on any open pull requests that you think are ready and should be included. Note that this is not the end of the journey ? anything that's not included right now can still come in later, before the first beta. (That's later this year, before you ask.) There will definitely be at least one more alpha release before we get even there. So, please concentrate on changes for this release that have a potentially disruptive user impact, so that our users can start preparing their code and/or complain early in order to keep us from making The One Big Mistake of breaking their code. Thanks, Stefan From dw-git at d-woods.co.uk Wed Mar 25 07:39:16 2020 From: dw-git at d-woods.co.uk (da-woods) Date: Wed, 25 Mar 2020 11:39:16 +0000 Subject: [Cython] Preparing the first alpha release of Cython 3.0 In-Reply-To: References: Message-ID: (I've not got the hang of the difference between "reply" and "reply-to-list" so initially sent this to the wrong place...) I've pinged a couple that would be nice to have because they both target Python compatibility: https://github.com/cython/cython/pull/3323/ https://github.com/cython/cython/pull/3383 They're both fairly large changes though and haven't been through any real review yet so it's possible they're exactly the opposite of what you're looking for. Feel free to ignore if that's the case.... David On 25/03/2020 08:10, Stefan Behnel wrote: > Hi all, > > I'm curently preparing a first alpha release for Cython 3.0 (alpha 1). > > Please ping me on any open pull requests that you think are ready and > should be included. > > Note that this is not the end of the journey ? anything that's not included > right now can still come in later, before the first beta. (That's later > this year, before you ask.) There will definitely be at least one more > alpha release before we get even there. > > So, please concentrate on changes for this release that have a potentially > disruptive user impact, so that our users can start preparing their code > and/or complain early in order to keep us from making The One Big Mistake > of breaking their code. > > Thanks, > Stefan > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel From dw-git at d-woods.co.uk Wed Mar 25 17:22:24 2020 From: dw-git at d-woods.co.uk (da-woods) Date: Wed, 25 Mar 2020 21:22:24 +0000 Subject: [Cython] Auto-generation of wrapper types In-Reply-To: References: <3795f79b-8915-a97a-3ed0-64e7a14fb94a@d-woods.co.uk> <1d7f1e0e-db3e-77d2-a1ae-239ac9c4fd6a@behnel.de> <4d33989d-1c44-fe73-fa47-c5fc87ac99a8@d-woods.co.uk> Message-ID: <74c82f30-7550-9667-e5a2-a3c30f23832a@d-woods.co.uk> 1) I hadn't give too much thought to inheritance. I think trying to mirror it in the wrapped types might be more trouble than it's worth - duck typing will probably do for most cases. 2) I'm definitely aware of the ownership and common references issue. I was going to assume that arguments passed in by (maybe const?)-reference could be treated as "by-value", but I guess that'll have to be a heavily flagged caveat in the documentation. Agree that integration with other wrappers could be useful. A very basic version is probably surprisingly easy in a lot of cases - it'd just be a case of exposing where they store the underlying C++ object. On 21/03/2020 18:47, Robert Bradshaw wrote: > I am in general of more automated wrapping. Two things that make C++ > classes more difficulty are. > > (1) How to handle inheritance (including multiple inheritance)? Would > the wrapped types mirror the inheritance? > (2) We can do auto-conversion of most types because they are passed by > value. This breaks down a bit with char*, but generally we assume we > can take a copy. Similarly with structs. However C++ classes are (in > practice) much more stateful and this gets into all sorts of thorny > issues with what the conventions should be about ownership and common > references. > > There's also a fair number of existing libraries out there that > specifically target wrapping C++ libraries (often directly from the > source files). As well as making it easier to wrap from C++ classes > from Cython, it'd be good to have more seamless (and efficient) > integration with those. > > On Sun, Mar 15, 2020 at 1:34 PM da-woods wrote: >> On 15/03/2020 08:14, Stefan Behnel wrote: >>> What makes C structs and C++ classes special enough to make this a separate >>> language feature? That should be part of a motivation somewhere (probably >>> in a ticket >> Most of the other major things you could want to wrap are covered >> through other mechanisms - you can create a wrapper round a `cdef >> extern` function or enum with `cpdef`. The basic C types (int, float) >> etc largely have a direct Python equivalent that converts freely. >> >> This is mainly targetted at C++ classes which seem to be the last major >> feature that doesn't have this kind of wrapper. C structs just seemed >> easy to deal with by the same mechanism - although they obviously have >> the conversion to/from dict, which is fine but does involve a >> conversion, and so doesn't propagate modifications back. >> >> I'll create a ticket fairly shortly. >> >>> How does this relate to the support for @dataclass in GH-2903? Is the only >>> difference that you would write "x.member" instead of "x.structattr.member"? >> It's slightly difficult to answer that because we don't know exactly >> what @dataclass will look like in Cython. In Python it's mostly about >> generating `__init__`, `__repr__`, `__eq__` etc from a "struct-like" >> description. The Cython version I've submitted basically just follows >> the Python scheme, but it isn't obvious how Python-level access to >> attributes should work. "x.structattr" would probably end up doing the >> usual Cython conversion to/from a dict. >> >> This feature is mostly for creating a quick wrapper for some external >> C/C++ thing that already exists. For a struct it probably isn't too >> different from a dataclass. For a C++ class it should hopefully be able >> to fill in a lot of the functions too. >> >>> Why "autowrap[S]" and not "autowrap(S)"? I'm asking also because there >>> should probably be a way for this to work in Python files, for which a >>> function call seems more natural. >>> >>> It also feels like a @decorator would somehow fit quite well here. Even >>> when defined in a .pxd file, the struct would still only be exported in the >>> corresponding module (with the same name). Something like this: >>> >>> @cclass >>> cdef struct S: >>> int x >>> >>> Not sure if reusing @cclass is a good idea, but it doesn't seem wrong. >> Definitely open to different syntax - now you point it out I think a >> function call might be better than an index (especially with optional >> keyword arguments to control it). >> >> One other option (for consistency with how functions/enums are already >> handled) would be `cpdef struct S/cpdef cppclass ClassName`. The >> difficulty here is that it overrides the name which I suspect could be a >> challenge (in most cases you'd probably want to access the original >> definition and the wrapper extension type separately in Cython). This >> might be an issue with a decorator too. >> >>> Defining this in a .pxd file could then even allow cimporting modules to >>> understand the extension type wrapper as well, by simply reconstructing the >>> type representation on their side. >> I hadn't thought of this, but yes. This should be pretty simple - the >> type representation is really just: >> >> cdef class SomeName: >> cdef c_type* obj >> @staticmethod >> cdef factor_func(obj x) # possibly reference or r-value here >> reference.... >> >> (Almost) everything else it defines is a Python interface of def >> functions and properties so a cimporting module would have no special >> knowledge - for fast access the user would go through obj. >> >> David >> >> >>> Stefan >>> _______________________________________________ >>> cython-devel mailing list >>> cython-devel at python.org >>> https://mail.python.org/mailman/listinfo/cython-devel >> >> _______________________________________________ >> cython-devel mailing list >> cython-devel at python.org >> https://mail.python.org/mailman/listinfo/cython-devel > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel