From stefan_ml at behnel.de Sun Feb 1 09:25:05 2015 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 01 Feb 2015 09:25:05 +0100 Subject: [Cython] IPython with Cython support on Android In-Reply-To: References: Message-ID: <54CDE2E1.7060308@behnel.de> Apps Embedded schrieb am 27.01.2015 um 15:12: > We have developped an android app which makes it possible to use the > IPython shell through Android. > > https://play.google.com/store/apps/details?id=com.appsopensource.labpy > https://play.google.com/store/apps/details?id=com.appsopensource.labpypremium > > We decided then to add the Cython extension to speed up script execution > through the notebook for instance. Interesting. Thanks for sharing. (I don't understand why it requests the right to track where I am, though, so I didn't try it out.) Stefan From apps.embedded at gmail.com Sun Feb 1 18:04:10 2015 From: apps.embedded at gmail.com (Apps Embedded) Date: Sun, 1 Feb 2015 18:04:10 +0100 Subject: [Cython] IPython with Cython support on Android In-Reply-To: <54CDE2E1.7060308@behnel.de> References: <54CDE2E1.7060308@behnel.de> Message-ID: Hi, The Free app has an advertising banner based on the Mobfox library. When integrating this library, it is written in their integration guide to add this permission. If you want to get rid of this permission, try the premium version. Since there is no advertising banner in this version, this permission is useless and thus has been removed. Best regards. Apps Embedded team. 2015-02-01 9:25 GMT+01:00 Stefan Behnel : > Apps Embedded schrieb am 27.01.2015 um 15:12: > > We have developped an android app which makes it possible to use the > > IPython shell through Android. > > > > https://play.google.com/store/apps/details?id=com.appsopensource.labpy > > > https://play.google.com/store/apps/details?id=com.appsopensource.labpypremium > > > > We decided then to add the Cython extension to speed up script execution > > through the notebook for instance. > > Interesting. Thanks for sharing. > > (I don't understand why it requests the right to track where I am, though, > so I didn't try it out.) > > Stefan > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sun Feb 1 22:41:34 2015 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 02 Feb 2015 10:41:34 +1300 Subject: [Cython] IPython with Cython support on Android In-Reply-To: References: <54CDE2E1.7060308@behnel.de> Message-ID: <54CE9D8E.5000506@canterbury.ac.nz> Apps Embedded wrote: > 2015-02-01 9:25 GMT+01:00 Stefan Behnel >: > > (I don't understand why it requests the right to track where I am, > though, > so I didn't try it out.) > > The Free app has an advertising banner based on the Mobfox library. > When integrating this library, it is written in their integration guide > to add this permission. So the price of using a "free" app is getting subjected to advertising *and* being spied upon? Seems a tad obnoxious to me. -- Greg From rhansen at bbn.com Mon Feb 2 01:52:34 2015 From: rhansen at bbn.com (Richard Hansen) Date: Sun, 1 Feb 2015 19:52:34 -0500 Subject: [Cython] [RFC PATCH] add read-only buffer support to typed memoryviews Message-ID: <1422838354-16180-1-git-send-email-rhansen@bbn.com> This patch is a very rough attempt at adding support for read-only buffer objects to typed memoryviews. It is incomplete, has bugs, and is very dirty. I'm positing it anyway to solicit help in figuring out the right way to add read-only buffer support. The 'XXX' comments mark issues I'm not sure about. The problem: The following example function raises an exception when passed a bytes object or any other read-only buffer object: cpdef object printbuf(unsigned char[:] buf): chars = [chr(x) for x in buf] print(repr(''.join(chars))) This is what happens: $ python -c 'import test; test.printbuf("test\0ing")' Traceback (most recent call last): File "", line 1, in File "test.pyx", line 1, in test.printbuf (test.c:1417) File "stringsource", line 614, in View.MemoryView.memoryview_cwrapper (test.c:6795) File "stringsource", line 321, in View.MemoryView.memoryview.__cinit__ (test.c:3341) BufferError: Object is not writable. The exception is raised because the code generated for typed memoryviews always passes the PyBUF_WRITABLE flag to PyObject_GetBuffer(), and "test\0ing" is a read-only bytes object. The bytes class raises an exception when it sees the PyBUF_WRITABLE flag. In this example case, there's no need to pass PyBUF_WRITABLE, and if it wasn't passed then the code would work as expected. The approach this patch takes is: * Never pass PyBUF_WRITABLE to PyObject_GetBuffer() just in case the underlying object is read-only. * If it turns out that we do need write access to the buffer (e.g., to handle 'buf[0] = x'), then check to see if the Py_buffer structure's readonly member is false. If so, it's safe to write. Otherwise, raise a BufferError exception ("read-only object"). This approach is not ideal because it doesn't support copy-on-write objects or similarly complicated buffer objects, but a better approach would be a more invasive change. See the comments I added in check_buffer_writable(). Feedback would be appreciated. Thanks, Richard --- Cython/Compiler/ExprNodes.py | 100 ++++++++++++++++++++++++++++++++++++++++++ Cython/Compiler/MemoryView.py | 12 ++--- Cython/Utility/MemoryView.pyx | 14 +++++- tests/memoryview/memslice.pyx | 12 ++--- 4 files changed, 124 insertions(+), 14 deletions(-) diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py index ac28d03..13017a9 100644 --- a/Cython/Compiler/ExprNodes.py +++ b/Cython/Compiler/ExprNodes.py @@ -3642,8 +3642,106 @@ class IndexNode(ExprNode): self.extra_index_params(code), code.error_goto(self.pos))) + def check_buffer_writable(self, code): + """Raise BufferError if the Py_buffer struct's readonly flag is set""" + + # XXX This is called from: + # * generate_buffer_setitem_code() + # * generate_memoryviewslice_setslice_code() + # * generate_memoryviewslice_assign_scalar_code() + # Are these sufficient? Is there another function called to + # generate code to directly modify a buffer via Cython syntax? + + # XXX is it safe to call self.buffer_entry() multiple times? + buffer_entry = self.buffer_entry() + + # XXX is this the right way to check to see if the object + # being modified is a memoryview slice object and not some + # other object type? + if not buffer_entry.type.is_buffer: + + # XXX Currently the PyBUF_WRITABLE flag is never passed + # when calling PyObject_GetBuffer() on the underlying + # buffer object, so we never asked the buffer object to + # provide us with a writable chunk of memory. Even if the + # Py_buffer struct's readonly flag is unset, might it be + # inappropriate to write to the buffer since we never + # passed PyBUF_WRITABLE? The Python 3 documentation seems + # to imply that this is OK, but it's not entirely clear. + + # XXX This code simply checks to see whether the Py_buffer + # view's readonly flag is unset. This design does not + # support copy-on-write or other similarly complicated + # buffer objects. + # + # For example, imagine a class that does copy-on-write. + # Suppose there is an instance of that type, and a copy of + # it is made. The copy hasn't been modified yet, so it + # shares a buffer with the original object. If + # PyBUF_WRITABLE is not passed to PyObject_GetBuffer(), + # the returned Py_buffer simply points to the shared + # memory and the readonly flag is set to avoid corrupting + # the original object. If PyBUF_WRITABLE is passed to + # PyObject_GetBuffer(), the object makes its own copy of + # the memory and the returned Py_buffer points to the + # private copy with the readonly flag unset. + # + # With the current design, writing to the copy in this + # scenario would raise a BufferError ("read-only buffer") + # even though it is possible to get a writable buffer by + # re-calling PyObject_GetBuffer() with PyBUF_WRITABLE. + # + # An improved design would look like this: + # + # * When creating the typed memoryview: + # 1. Call PyObject_GetBuffer() to fetch a Py_buffer + # view into the object, but do not pass + # PyBUF_WRITABLE. + # 2. Save the Py_buffer view somewhere. + # 3. Save the flags that were passed to + # PyObject_GetBuffer(). + # + # * Whenever write access is required: + # 1. If the saved flags do not include + # PyBUF_WRITABLE: + # a. Call PyOjbect_GetBuffer() again, but this + # time include the PyBUF_WRITABLE flag. If + # this raises a BufferError exception, let + # it propagate. + # b. Call PyBuffer_Release() on the Py_buffer + # view originally saved when the typed + # memoryview was created. + # c. Save the new Py_buffer view. + # d. Save the new flags. + # 2. Assert that the Py_buffer view's readonly flag + # is not set + + # XXX is there a better way to get the cname of the + # __Pyx_memviewslice variable? + if self.is_temp: + # buffer_entry.cname is None in this case, so we have + # to get the name of the variable some other way + cname = self.temp_code + else: + cname = buffer_entry.cname + assert cname is not None + + # XXX is this the right way to test if the Py_buffer's + # readonly flag is set? + code.putln('if (unlikely(%s.memview->view.readonly)) {' % (cname,)) + + # XXX is this the right way to raise an exception? + code.putln('PyErr_Format(PyExc_BufferError, "read-only buffer");') + code.putln(code.error_goto(self.pos)) + code.putln('}') + else: + # XXX what do we do here? + raise NotImplementedError() + def generate_buffer_setitem_code(self, rhs, code, op=""): # Used from generate_assignment_code and InPlaceAssignmentNode + self.check_buffer_writable(code) + buffer_entry, ptrexpr = self.buffer_lookup_code(code) if self.buffer_type.dtype.is_pyobject: @@ -3834,11 +3932,13 @@ class IndexNode(ExprNode): def generate_memoryviewslice_setslice_code(self, rhs, code): "memslice1[...] = memslice2 or memslice1[:] = memslice2" + self.check_buffer_writable(code) from . import MemoryView MemoryView.copy_broadcast_memview_src_to_dst(rhs, self, code) def generate_memoryviewslice_assign_scalar_code(self, rhs, code): "memslice1[...] = 0.0 or memslice1[:] = 0.0" + self.check_buffer_writable(code) from . import MemoryView MemoryView.assign_scalar(self, rhs, code) diff --git a/Cython/Compiler/MemoryView.py b/Cython/Compiler/MemoryView.py index 8e2f58b..9e75267 100644 --- a/Cython/Compiler/MemoryView.py +++ b/Cython/Compiler/MemoryView.py @@ -32,12 +32,12 @@ def concat_flags(*flags): format_flag = "PyBUF_FORMAT" -memview_c_contiguous = "(PyBUF_C_CONTIGUOUS | PyBUF_FORMAT | PyBUF_WRITABLE)" -memview_f_contiguous = "(PyBUF_F_CONTIGUOUS | PyBUF_FORMAT | PyBUF_WRITABLE)" -memview_any_contiguous = "(PyBUF_ANY_CONTIGUOUS | PyBUF_FORMAT | PyBUF_WRITABLE)" -memview_full_access = "PyBUF_FULL" -#memview_strided_access = "PyBUF_STRIDED" -memview_strided_access = "PyBUF_RECORDS" +memview_c_contiguous = "(PyBUF_C_CONTIGUOUS | PyBUF_FORMAT)" +memview_f_contiguous = "(PyBUF_F_CONTIGUOUS | PyBUF_FORMAT)" +memview_any_contiguous = "(PyBUF_ANY_CONTIGUOUS | PyBUF_FORMAT)" +memview_full_access = "PyBUF_FULL_RO" +#memview_strided_access = "PyBUF_STRIDED_RO" +memview_strided_access = "PyBUF_RECORDS_RO" MEMVIEW_DIRECT = '__Pyx_MEMVIEW_DIRECT' MEMVIEW_PTR = '__Pyx_MEMVIEW_PTR' diff --git a/Cython/Utility/MemoryView.pyx b/Cython/Utility/MemoryView.pyx index f7ecd58..4344978 100644 --- a/Cython/Utility/MemoryView.pyx +++ b/Cython/Utility/MemoryView.pyx @@ -61,6 +61,7 @@ cdef extern from *: PyBUF_STRIDES PyBUF_INDIRECT PyBUF_RECORDS + PyBUF_RECORDS_RO ctypedef struct __Pyx_TypeInfo: pass @@ -367,6 +368,9 @@ cdef class memoryview(object): return self.convert_item_to_object(itemp) def __setitem__(memoryview self, object index, object value): + if self.view.readonly: + raise BufferError('read-only buffer') + have_slices, index = _unellipsify(index, self.view.ndim) if have_slices: @@ -466,6 +470,9 @@ cdef class memoryview(object): @cname('getbuffer') def __getbuffer__(self, Py_buffer *info, int flags): + if (flags & PyBUF_WRITABLE) and self.view.readonly: + raise BufferError("read-only buffer") + if flags & PyBUF_STRIDES: info.shape = self.view.shape else: @@ -490,7 +497,7 @@ cdef class memoryview(object): info.ndim = self.view.ndim info.itemsize = self.view.itemsize info.len = self.view.len - info.readonly = 0 + info.readonly = self.view.readonly info.obj = self __pyx_getbuffer = capsule( &__pyx_memoryview_getbuffer, "getbuffer(obj, view, flags)") @@ -981,7 +988,10 @@ cdef memoryview_fromslice({{memviewslice_name}} memviewslice, (<__pyx_buffer *> &result.view).obj = Py_None Py_INCREF(Py_None) - result.flags = PyBUF_RECORDS + if (result.from_slice.memview).flags & PyBUF_WRITABLE: + result.flags = PyBUF_RECORDS + else: + result.flags = PyBUF_RECORDS_RO result.view.shape = result.from_slice.shape result.view.strides = result.from_slice.strides diff --git a/tests/memoryview/memslice.pyx b/tests/memoryview/memslice.pyx index 417bcbc..62f8782 100644 --- a/tests/memoryview/memslice.pyx +++ b/tests/memoryview/memslice.pyx @@ -480,7 +480,7 @@ def strided(int[:] buf): released A 2 >>> [str(x) for x in A.recieved_flags] # Py2/3 - ['FORMAT', 'ND', 'STRIDES', 'WRITABLE'] + ['FORMAT', 'ND', 'STRIDES'] Check that the suboffsets were patched back prior to release. >>> A.release_ok @@ -495,7 +495,7 @@ def c_contig(int[::1] buf): >>> c_contig(A) 2 >>> [str(x) for x in A.recieved_flags] - ['FORMAT', 'ND', 'STRIDES', 'C_CONTIGUOUS', 'WRITABLE'] + ['FORMAT', 'ND', 'STRIDES', 'C_CONTIGUOUS'] """ return buf[2] @@ -508,7 +508,7 @@ def c_contig_2d(int[:, ::1] buf): >>> c_contig_2d(A) 7 >>> [str(x) for x in A.recieved_flags] - ['FORMAT', 'ND', 'STRIDES', 'C_CONTIGUOUS', 'WRITABLE'] + ['FORMAT', 'ND', 'STRIDES', 'C_CONTIGUOUS'] """ return buf[1, 3] @@ -519,7 +519,7 @@ def f_contig(int[::1, :] buf): >>> f_contig(A) 2 >>> [str(x) for x in A.recieved_flags] - ['FORMAT', 'ND', 'STRIDES', 'F_CONTIGUOUS', 'WRITABLE'] + ['FORMAT', 'ND', 'STRIDES', 'F_CONTIGUOUS'] """ return buf[0, 1] @@ -532,7 +532,7 @@ def f_contig_2d(int[::1, :] buf): >>> f_contig_2d(A) 7 >>> [str(x) for x in A.recieved_flags] - ['FORMAT', 'ND', 'STRIDES', 'F_CONTIGUOUS', 'WRITABLE'] + ['FORMAT', 'ND', 'STRIDES', 'F_CONTIGUOUS'] """ return buf[3, 1] @@ -990,7 +990,7 @@ def bufdefaults1(int[:] buf): acquired A released A >>> [str(x) for x in A.recieved_flags] - ['FORMAT', 'ND', 'STRIDES', 'WRITABLE'] + ['FORMAT', 'ND', 'STRIDES'] """ pass -- 2.2.2 From apps.embedded at gmail.com Tue Feb 3 08:13:12 2015 From: apps.embedded at gmail.com (Apps Embedded) Date: Tue, 3 Feb 2015 08:13:12 +0100 Subject: [Cython] IPython with Cython support on Android In-Reply-To: <54CE9D8E.5000506@canterbury.ac.nz> References: <54CDE2E1.7060308@behnel.de> <54CE9D8E.5000506@canterbury.ac.nz> Message-ID: Hi, We are going to suppress the advertising banner in the Free version. Have a look to the next free coming version. No banner any more and no more localisation permission needed. Finding a business model for free software is not that easy. But we try improving our concept. Best regards. Apps Embedded team. 2015-02-01 22:41 GMT+01:00 Greg Ewing : > Apps Embedded wrote: > > > 2015-02-01 9:25 GMT+01:00 Stefan Behnel > >: > > > > (I don't understand why it requests the right to track where I am, > > though, > > so I didn't try it out.) > > > >> The Free app has an advertising banner based on the Mobfox library. >> When integrating this library, it is written in their integration guide >> to add this permission. >> > > So the price of using a "free" app is getting subjected to > advertising *and* being spied upon? Seems a tad obnoxious > to me. > > -- > Greg > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From supermihi at posteo.de Thu Feb 5 09:52:08 2015 From: supermihi at posteo.de (Michael) Date: Thu, 05 Feb 2015 09:52:08 +0100 Subject: [Cython] Compilation failes if a class member is named "INFINITY" In-Reply-To: References: <54CB53BE.3060404@posteo.de> Message-ID: <54D32F38.30605@posteo.de> Am 31.01.2015 um 19:48 schrieb Matthew Brett: > Hi, > > On Fri, Jan 30, 2015 at 1:49 AM, Michael wrote: >> Hello, >> >> if I try to compile the following minimal example: >> >> cdef class Test: >> >> cdef readonly int INFINITY >> >> cython does not complain but gcc refuses with an error message: >> In file included from /usr/include/math.h:38:0, >> from /usr/include/python2.7/pyport.h:325, >> from /usr/include/python2.7/Python.h:58, >> from testinf.c:16: >> testinf.c:433:7: error: field '__builtin_inff' declared as a function >> int INFINITY; >> ^ >> testinf.c: In function '__pyx_pf_7testinf_4Test_8INFINITY___get__': >> testinf.c:569:50: error: expected identifier before '(' token >> __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->INFINITY); if >> (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 10; >> __pyx_clineno = __LINE__; goto __pyx_L1_error;} >> ^ >> Apparently the name "INFINITY" is handled wrongly; any other variable >> name seems to be fine. > > Maybe you hit the INFINITY gcc macro? > > http://www.gnu.org/software/libc/manual/html_node/Infinity-and-NaN.html Yes, that's probably the explanation. So should Cython do anything about the problem, i.e. raise an exception if struct members are named like a macro? I don't know about the Cython internals, but I wonder why the name not altered in the c file: struct __pyx_obj_7testinf_Test { PyObject_HEAD int INFINITY; }; I would have expected something like "int __pyx_member_INFINITY"? Michael From stefan_ml at behnel.de Thu Feb 5 10:44:15 2015 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Feb 2015 10:44:15 +0100 Subject: [Cython] Compilation failes if a class member is named "INFINITY" In-Reply-To: <54D32F38.30605@posteo.de> References: <54CB53BE.3060404@posteo.de> <54D32F38.30605@posteo.de> Message-ID: <54D33B6F.6050300@behnel.de> Michael schrieb am 05.02.2015 um 09:52: > Am 31.01.2015 um 19:48 schrieb Matthew Brett: >> On Fri, Jan 30, 2015 at 1:49 AM, Michael wrote: >>> if I try to compile the following minimal example: >>> >>> cdef class Test: >>> >>> cdef readonly int INFINITY >>> >>> cython does not complain but gcc refuses with an error message: >>> In file included from /usr/include/math.h:38:0, >>> from /usr/include/python2.7/pyport.h:325, >>> from /usr/include/python2.7/Python.h:58, >>> from testinf.c:16: >>> testinf.c:433:7: error: field '__builtin_inff' declared as a function >>> int INFINITY; >>> ^ >>> testinf.c: In function '__pyx_pf_7testinf_4Test_8INFINITY___get__': >>> testinf.c:569:50: error: expected identifier before '(' token >>> __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->INFINITY); if >>> (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 10; >>> __pyx_clineno = __LINE__; goto __pyx_L1_error;} >>> ^ >>> Apparently the name "INFINITY" is handled wrongly; any other variable >>> name seems to be fine. >> >> Maybe you hit the INFINITY gcc macro? >> >> http://www.gnu.org/software/libc/manual/html_node/Infinity-and-NaN.html > > Yes, that's probably the explanation. So should Cython do anything about > the problem, i.e. raise an exception if struct members are named like a > macro? Cython cannot know about everything that the C compiler (or its preprocessor) will eventually see in its flat namespace. So, no, Cython can't generally solve this problem for you. > I don't know about the Cython internals, but I wonder why the name not > altered in the c file: > struct __pyx_obj_7testinf_Test { > PyObject_HEAD > int INFINITY; > }; > I would have expected something like "int __pyx_member_INFINITY"? Python extension types are just structs at the C level, and struct member names cannot be mangled. Plus, when types are external or public (or even just redeclared somewhere else by someone else), foreign code may depend on struct fields of extension types as well. Builtin types are an example. Stefan From supermihi at posteo.de Thu Feb 5 11:40:43 2015 From: supermihi at posteo.de (Michael) Date: Thu, 05 Feb 2015 11:40:43 +0100 Subject: [Cython] Compilation failes if a class member is named "INFINITY" In-Reply-To: <54D33B6F.6050300@behnel.de> References: <54CB53BE.3060404@posteo.de> <54D32F38.30605@posteo.de> <54D33B6F.6050300@behnel.de> Message-ID: <54D348AB.8070207@posteo.de> Am 05.02.2015 um 10:44 schrieb Stefan Behnel: > Michael schrieb am 05.02.2015 um 09:52: >> Am 31.01.2015 um 19:48 schrieb Matthew Brett: >>> On Fri, Jan 30, 2015 at 1:49 AM, Michael wrote: >>>> if I try to compile the following minimal example: >>>> >>>> cdef class Test: >>>> >>>> cdef readonly int INFINITY >>>> >>>> cython does not complain but gcc refuses with an error message: >>>> In file included from /usr/include/math.h:38:0, >>>> from /usr/include/python2.7/pyport.h:325, >>>> from /usr/include/python2.7/Python.h:58, >>>> from testinf.c:16: >>>> testinf.c:433:7: error: field '__builtin_inff' declared as a function >>>> int INFINITY; >>>> ^ >>>> testinf.c: In function '__pyx_pf_7testinf_4Test_8INFINITY___get__': >>>> testinf.c:569:50: error: expected identifier before '(' token >>>> __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->INFINITY); if >>>> (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 10; >>>> __pyx_clineno = __LINE__; goto __pyx_L1_error;} >>>> ^ >>>> Apparently the name "INFINITY" is handled wrongly; any other variable >>>> name seems to be fine. >>> >>> Maybe you hit the INFINITY gcc macro? >>> >>> http://www.gnu.org/software/libc/manual/html_node/Infinity-and-NaN.html >> >> Yes, that's probably the explanation. So should Cython do anything about >> the problem, i.e. raise an exception if struct members are named like a >> macro? > > Cython cannot know about everything that the C compiler (or its > preprocessor) will eventually see in its flat namespace. So, no, Cython > can't generally solve this problem for you. > > >> I don't know about the Cython internals, but I wonder why the name not >> altered in the c file: >> struct __pyx_obj_7testinf_Test { >> PyObject_HEAD >> int INFINITY; >> }; >> I would have expected something like "int __pyx_member_INFINITY"? > > Python extension types are just structs at the C level, and struct member > names cannot be mangled. Plus, when types are external or public (or even > just redeclared somewhere else by someone else), foreign code may depend on > struct fields of extension types as well. Builtin types are an example. > Ok, that makes sense. I'll just choose a different name then. :-) Michael > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel > From greg.ewing at canterbury.ac.nz Thu Feb 5 12:30:54 2015 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 06 Feb 2015 00:30:54 +1300 Subject: [Cython] Compilation failes if a class member is named "INFINITY" In-Reply-To: <54D33B6F.6050300@behnel.de> References: <54CB53BE.3060404@posteo.de> <54D32F38.30605@posteo.de> <54D33B6F.6050300@behnel.de> Message-ID: <54D3546E.4040806@canterbury.ac.nz> Stefan Behnel wrote: > Python extension types are just structs at the C level, and struct member > names cannot be mangled. Well, in principle it *could* mangle the names in structs that aren't declared "cdef extern". I didn't do that in Pyrex because it didn't seem necessary; I hadn't anticipated problems like collision with macros. It's probably too late to change it now, since it would break existing code that interoperates with foreign code. -- Greg From supermihi at posteo.de Thu Feb 5 14:26:11 2015 From: supermihi at posteo.de (Michael) Date: Thu, 05 Feb 2015 14:26:11 +0100 Subject: [Cython] Compilation failes if a class member is named "INFINITY" In-Reply-To: <54D3546E.4040806@canterbury.ac.nz> References: <54CB53BE.3060404@posteo.de> <54D32F38.30605@posteo.de> <54D33B6F.6050300@behnel.de> <54D3546E.4040806@canterbury.ac.nz> Message-ID: <54D36F73.6050201@posteo.de> Am 05.02.2015 um 12:30 schrieb Greg Ewing: > Stefan Behnel wrote: >> Python extension types are just structs at the C level, and struct member >> names cannot be mangled. > > Well, in principle it *could* mangle the names in > structs that aren't declared "cdef extern". I didn't > do that in Pyrex because it didn't seem necessary; > I hadn't anticipated problems like collision with > macros. > > It's probably too late to change it now, since it > would break existing code that interoperates with > foreign code. > To me, that sounds like a too complicated solution for a marginal problem. It's a bit odd that you can't use special names like INFINITY for cdef class members, but on the other hand, I think that's a reasonable restriction since you are generating C code, and in C you can't name a struct member INFINITY either (at least with gcc). From uli-do at gmx.at Wed Feb 11 09:56:35 2015 From: uli-do at gmx.at (Ulrich Dobramysl) Date: Wed, 11 Feb 2015 08:56:35 +0000 Subject: [Cython] operator() bug in cython Message-ID: Dear all, I tried to declare an external c++ class that includes an operator() function in a pxd file. When I then call a class instance, cython generates faulty c++ code. It includes a call to "operator()()", and not a call to the instance object. Here is a minimum working example: call_operator.pxd: ---- cdef extern from "call_operator.hpp" nogil: cdef cppclass OperatorTest: int operator()() ---- test_call_operator.pyx: ---- from call_operator cimport OperatorTest def test(): cdef OperatorTest t t() ---- Running "cython --cplus test_call_operator.pyx" generates the following code for the test() function: ---- (relevant part only) /* "test_call_operator.pyx":4 * def test(): * cdef OperatorTest t * t() # <<<<<<<<<<<<<< */ operator()(); ---- As you can see, the code that is generated is a call to "operator()()" and not "t()" as it should be. >From what I've been able to work out, the problem seems to be that the call to "t()" is treated as a NameNode in ExprNodes.py and not an AttributeNode. However, I don't know enough about Cython's internals to track where exactly this decision is made. Curiously, this bug isn't always triggered in more complex situations. I had a larger pxd file with multiple external declarations where one class operator() was treated correctly, while others weren't. I haven't been able to find out why this was the case. Ulrich -------------- next part -------------- An HTML attachment was scrubbed... URL: From robertwb at gmail.com Thu Feb 12 09:33:51 2015 From: robertwb at gmail.com (Robert Bradshaw) Date: Thu, 12 Feb 2015 00:33:51 -0800 Subject: [Cython] Cython 0.22 released Message-ID: I'm pleased to announce the release of Cython 0.22. This release has numerous bug fixes and several new features. It is is available in the usual locations: http://cython.org/release/Cython-0.22.tar.gz https://pypi.python.org/pypi/Cython/0.22 Happy Coding! Robert Features added -------------- * C functions can coerce to Python functions, which allows passing them around as callable objects. * C arrays can be assigned by value and auto-coerce from Python iterables and to Python lists (and tuples). * Extern C functions can now be declared as cpdef to export them to the module's Python namespace. Extern C functions in pxd files export their values to their own module, iff it exists. * Anonymous C tuple types can be declared as (ctype1, ctype2, ...). * PEP 479: turn accidental StopIteration exceptions that exit generators into a RuntimeError, activated with future import "generator_stop". See http://legacy.python.org/dev/peps/pep-0479/ * Looping over ``reversed(range())`` is optimised in the same way as ``range()``. Patch by Favian Contreras. Bugs fixed ---------- * Mismatching 'except' declarations on signatures in .pxd and .pyx files failed to produce a compile error. * Failure to find any files for the path pattern(s) passed into ``cythonize()`` is now an error to more easily detect accidental typos. * In Py2.6/7 and Py3.2, simple Cython memory views could accidentally be interpreted as non-contiguous by CPython, which could trigger a CPython bug when copying data from them, thus leading to data corruption. See CPython issues 12834 and 23349. Other changes ------------- * Preliminary support for defining the Cython language with a formal grammar. To try parsing your files against this grammar, use the --formal_grammar directive. Experimental. * ``_`` is no longer considered a cacheable builtin as it could interfere with gettext. * Cythonize-computed metadata now cached in the generate C files. * Several corrections and extensions in numpy, cpython, and libcpp pxd files. From stefan_ml at behnel.de Fri Feb 13 15:05:09 2015 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 13 Feb 2015 15:05:09 +0100 Subject: [Cython] operator() bug in cython In-Reply-To: References: Message-ID: <54DE0495.4040408@behnel.de> Ulrich Dobramysl schrieb am 11.02.2015 um 09:56: > I tried to declare an external c++ class that includes an operator() > function in a pxd file. When I then call a class instance, cython generates > faulty c++ code. It includes a call to "operator()()", and not a call to > the instance object. Here is a minimum working example: > > call_operator.pxd: > ---- > cdef extern from "call_operator.hpp" nogil: > cdef cppclass OperatorTest: > int operator()() > ---- > > test_call_operator.pyx: > ---- > from call_operator cimport OperatorTest > def test(): > cdef OperatorTest t > t() > ---- > > Running "cython --cplus test_call_operator.pyx" generates the following > code for the test() function: > > ---- (relevant part only) > /* "test_call_operator.pyx":4 > * def test(): > * cdef OperatorTest t > * t() # <<<<<<<<<<<<<< > */ > operator()(); > ---- > As you can see, the code that is generated is a call to "operator()()" and > not "t()" as it should be. Thanks for the report, I can confirm this. > From what I've been able to work out, the problem seems to be that the call > to "t()" is treated as a NameNode in ExprNodes.py and not an AttributeNode. > However, I don't know enough about Cython's internals to track where > exactly this decision is made. > > Curiously, this bug isn't always triggered in more complex situations. I > had a larger pxd file with multiple external declarations where one class > operator() was treated correctly, while others weren't. I haven't been able > to find out why this was the case. It happens only with stack allocated C++ objects, not with heap objects (i.e. pointers). That can be used as a work-around, I guess. Stefan From uli-do at gmx.at Fri Feb 13 16:14:56 2015 From: uli-do at gmx.at (Ulrich Dobramysl) Date: Fri, 13 Feb 2015 15:14:56 +0000 Subject: [Cython] operator() bug in cython References: <54DE0495.4040408@behnel.de> Message-ID: On Fri Feb 13 2015 at 2:23:45 PM Stefan Behnel wrote: > Ulrich Dobramysl schrieb am 11.02.2015 um 09:56: > > From what I've been able to work out, the problem seems to be that the > call > > to "t()" is treated as a NameNode in ExprNodes.py and not an > AttributeNode. > > However, I don't know enough about Cython's internals to track where > > exactly this decision is made. > > > > Curiously, this bug isn't always triggered in more complex situations. I > > had a larger pxd file with multiple external declarations where one class > > operator() was treated correctly, while others weren't. I haven't been > able > > to find out why this was the case. > > It happens only with stack allocated C++ objects, not with heap objects > (i.e. pointers). That can be used as a work-around, I guess. > > Thanks! I haven't figured out how to call heap allocated objects, as the code --- cdef OperatorTest *t = new OperatorTest() t() --- is not translatable by Cython. Is there a special syntax for calling cython "function pointers"? The trick dereference(t)() doesn't work either. However, one thing that works is if the (stack-allocated) called object is not a local variable (a NameNode), but an attribute of some object (an AttributeNode). A quick and dirty fix for this would be this patch for NameNode.calculate_result_code: --- diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py index f99ec6e..f894a64 100644 --- a/Cython/Compiler/ExprNodes.py +++ b/Cython/Compiler/ExprNodes.py @@ -1904,6 +1904,8 @@ class NameNode(AtomicExprNode): entry = self.entry if not entry: return "" # There was an error earlier + if entry.cname=='operator()': + return self.name return entry.cname def generate_result_code(self, code): --- But I have no idea what other side effects that might have. Ulrich -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sat Feb 14 00:07:13 2015 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 14 Feb 2015 12:07:13 +1300 Subject: [Cython] operator() bug in cython In-Reply-To: References: <54DE0495.4040408@behnel.de> Message-ID: <54DE83A1.6010903@canterbury.ac.nz> Ulrich Dobramysl wrote: > Thanks! I haven't figured out how to call heap allocated objects, as the > code > --- > cdef OperatorTest *t = new OperatorTest() > t() > --- > is not translatable by Cython. Have you tried: t[0]() ? > A quick and dirty fix for this would be this patch > for NameNode.calculate_result_code: > --- > diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py > index f99ec6e..f894a64 100644 > --- a/Cython/Compiler/ExprNodes.py > +++ b/Cython/Compiler/ExprNodes.py > @@ -1904,6 +1904,8 @@ class NameNode(AtomicExprNode): > entry = self.entry > if not entry: > return "" # There was an error earlier > + if entry.cname=='operator()': > + return self.name > return entry.cname I haven't been following the development of Cython's internals, so I may be speaking naively here, but it looks wrong to me that the cname of that entry should be 'operator()' rather than the c-level name of the variable that the NameNode refers to. -- Greg From uli-do at gmx.at Tue Feb 17 10:46:14 2015 From: uli-do at gmx.at (Ulrich Dobramysl) Date: Tue, 17 Feb 2015 09:46:14 +0000 Subject: [Cython] operator() bug in cython Message-ID: On Fri Feb 13 2015 at 11:07:39 PM Greg Ewing wrote: > Ulrich Dobramysl wrote: > > > Thanks! I haven't figured out how to call heap allocated objects, as the > > code > > --- > > cdef OperatorTest *t = new OperatorTest() > > t() > > --- > > is not translatable by Cython. > > Have you tried: > > t[0]() > > ? > Thanks, I didn't think about the t[0] trick! > I haven't been following the development of Cython's internals, > so I may be speaking naively here, but it looks wrong to me that > the cname of that entry should be 'operator()' rather than the > c-level name of the variable that the NameNode refers to. > > The cname of the entry is correct IMO, because it cannot have any knowledge about the cname of the object it is being called as later. The logic dealing with the special case of an operator() member function of a c++ class in SimpleCallNode.analyse_c_function_call() needs to be changed slightly to not treat operator() as an overloaded entry. Rather, the function type of the SimpleCallNode needs to be set to the type of the looked-up operator() entry. Then, at the code generation stage, the correct cname of the SimpleCallNode entry is used and not the entry of operator(). With the patch below my test case compiles and runs without problems: ---- diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py index f99ec6e..88522c9 100644 --- a/Cython/Compiler/ExprNodes.py +++ b/Cython/Compiler/ExprNodes.py @@ -4569,11 +4569,13 @@ class SimpleCallNode(CallNode): args = self.args if func_type.is_cpp_class: - overloaded_entry = self.function.type.scope.lookup("operator()") - if overloaded_entry is None: + call_operator = self.function.type.scope.lookup("operator()") + if call_operator is None: self.type = PyrexTypes.error_type self.result_code = "" return + self.function.type = call_operator.type + overloaded_entry = None elif hasattr(self.function, 'entry'): overloaded_entry = self.function.entry elif (isinstance(self.function, IndexNode) and @@ -4603,7 +4605,7 @@ class SimpleCallNode(CallNode): else: entry = None func_type = self.function_type() - if not func_type.is_cfunction: + if not (func_type.is_cfunction or func_type.is_cpp_class): error(self.pos, "Calling non-function type '%s'" % func_type) self.type = PyrexTypes.error_type self.result_code = "" ---- What do you think? I don't know a lot about cython's internals, so there might be use cases which break this code. Ulrich -------------- next part -------------- An HTML attachment was scrubbed... URL: From raniere at ime.unicamp.br Thu Feb 19 13:00:49 2015 From: raniere at ime.unicamp.br (Raniere Silva) Date: Thu, 19 Feb 2015 10:00:49 -0200 Subject: [Cython] Google Summer of Code and NumFOCUS Message-ID: <20150219120049.GD15475@pupunha> Hi, NumFOCUS has promotes and supports the ongoing research and development of open-source computing tools including Cython. This year NumFOCUS want to try be a Google Summer of Code "umbrella" mentoring organization, Umbrella organizations are mentoring organizations accepted into the Google Summer of Code program that have other open source organizations working "under" them. Sometime organizations that work very closely or have very similar goals or communities may get put together under an "umbrella." Google stills expects all organizations under the umbrella, whether accepted into the program under their title or not, to adhere to all the rules and regulations of the program. From https://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2015/help_page#umbrella_organization To help promote and support Cython. We encourage Cython to apply to Google Summer of Code under your own title and will be very happy if you can also do with us. If you are interested, please check https://github.com/swcarpentry/gsoc2015 and https://github.com/swcarpentry/gsoc2015/blob/master/CONTRIBUTING.md. If you have any question, please email me directly. Thanks in advance, Raniere -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From raniere at ime.unicamp.br Fri Feb 20 21:31:57 2015 From: raniere at ime.unicamp.br (Raniere Silva) Date: Fri, 20 Feb 2015 18:31:57 -0200 Subject: [Cython] =?utf-8?q?ANN=3A_SciPy_Latin_Am=C3=A9rica_2015_-_Call_fo?= =?utf-8?q?r_Proposals?= Message-ID: <20150220203157.GM12853@pupunha> *Call for Proposals* *SciPy Latin Am?rica 2015*, the third annual Scientific Computing with Python Conference, will be held this *May 20-22* in *Posadas, Misiones, Argentina*. SciPy is a community dedicated to the advancement of scientific computing through open source Python software for mathematics, science, and engineering. The annual SciPy Conferences allows participants from academic, commercial, and governmental organizations to showcase their latest projects, learn from skilled users and developers, and collaborate on code development. *Proposals are now being accepted for SciPy Latin Am?rica 2015*. Presentation content can be at a novice, intermediate or advanced level. Talks will run 30-40 min and hands-on tutorials will run 100-120 min. We also receive proposal for posters. For more information about the different types of proposal, see below the "*Different types of Communication*" section. *How to Submit?* 1. Register for an account on http://conf.scipyla.org/user/register 2. Submit your proposal at http://conf.scipyla.org/activity/propose *Important Dates* - *April 6th*: Talks, poster, tutorial submission deadline. - *April 20th*: Notification Talks / Posters / Tutorial accepted. - *May 20th-22nd*: SciPy Latin Am?rica 2015. *Different types of Communication* *Talks*: These are the traditional talk sessions given during the main conference days. They're mostly 30 minutes long with 5 min for questions. If you think you have a topic but aren't sure how to propose it, contact our program committee and we'll work with you. We'd love to help you come up with a great proposal. *Tutorials*: We are looking for tutorials that can grow this community at any level. We aim for tutorials that will advance Scientific Python, advance this community, and shape the future. They're are 100-120 minutes long, but if you think you need more than one slot, you can split the content and submit two self-contained proposals. *Posters*: The poster session provides a more interactive, attendee-driven presentation than the speaker-driven conference talks. Poster presentations have fostered extensive discussions on the topics, with many that have gone on much longer than the actual "session" called for. The idea is to present your topic on poster board and as attendees mingle through the rows, they find your topic, read through what you've written, then strike up a discussion on it. It's as simple as that. You could be doing Q&A in the first minute of the session with a group of 10 people. *Lightning Talks*: Want to give a talk, but do not have enough material for a full talk? These talks are, at max, 5 minute talks done in quick succession in the main hall. No need to fill the whole slot, though! -- *The SciPy LA 2015 Program **Committee* -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From stefan_ml at behnel.de Fri Feb 27 10:59:36 2015 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 27 Feb 2015 10:59:36 +0100 Subject: [Cython] PEP 448: Additional Unpacking Generalizations Message-ID: <54F04008.1010004@behnel.de> Looks like a nice goody for Cython users, too: https://www.python.org/dev/peps/pep-0448/ Requires grammar changes, but otherwise could be done with a transform, I guess, by mapping the unpacking to list/tuple concatenations and dict updates. Stefan