From stefan_ml at behnel.de Fri Oct 3 09:27:48 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 03 Oct 2014 09:27:48 +0200 Subject: [Cython] setting C++ mode in cythonize() Message-ID: <542E4FF4.9090401@behnel.de> Hi, the documentation suggests to pass language='c++' into cythonize() to globally enable C++ mode, but AFAICT, this never actually worked. Cython previously ignored this option, Cython 0.21 rejects it, and we didn't update the docs, so the current situation is outright confusing. http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html#specify-c-language-in-setup-py To me, this option seems the one obvious way to do it, as it mimics the distutils Extension API most closely. So, I now implemented support for this, but only for modules found by file name, not for user provided Extension objects. I think that's a reasonable restriction. If users construct their own Extension instances anyway, they can just as well set the 'language' option on them directly (and probably already do so). https://github.com/cython/cython/commit/88df97f250073d8a06f965d6a7e09d8fe6f4ca63 Stefan From honnibal at gmail.com Sat Oct 4 15:37:23 2014 From: honnibal at gmail.com (Matthew Honnibal) Date: Sat, 4 Oct 2014 06:37:23 -0700 (PDT) Subject: [Cython] setting C++ mode in cythonize() In-Reply-To: <542E4FF4.9090401@behnel.de> References: <542E4FF4.9090401@behnel.de> Message-ID: <63173819-e6d6-4ef1-882f-f6d208b6f9c5@googlegroups.com> On Friday, 3 October 2014 17:27:54 UTC+10, Stefan Behnel wrote: > > Hi, > > the documentation suggests to pass language='c++' into cythonize() to > globally enable C++ mode, but AFAICT, this never actually worked. Cython > previously ignored this option, Cython 0.21 rejects it, and we didn't > update the docs, so the current situation is outright confusing. > > > http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html#specify-c-language-in-setup-py > > Thanks for this. Cython's a big project that's been changing fast, and as a user I've often found the docs a bit lacking. For instance, I wanted to use language="c++" for pyximport, and had some trouble doing so. I found this answer on StackOverflow: https://stackoverflow.com/questions/7620003/how-do-you-tell-pyximport-to-use-the-cython-cplus-option It recommends the use of a "foo.pyxbld" file, in the same directory as "foo.pyx", which pyximport will read for options. I tried it, and it does indeed work. This blew me away --- I'm fairly sure it's never mentioned in the docs, and it left me with many questions... My main question is: is this actually the recommended solution? Or is it there for historical reasons, e.g. via Pyrex? > To me, this option seems the one obvious way to do it, as it mimics the > distutils Extension API most closely. So, I now implemented support for > this, but only for modules found by file name, not for user provided > Extension objects. I think that's a reasonable restriction. If users > construct their own Extension instances anyway, they can just as well set > the 'language' option on them directly (and probably already do so). > > > https://github.com/cython/cython/commit/88df97f250073d8a06f965d6a7e09d8fe6f4ca63 > > Stefan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at onerussian.com Wed Oct 8 16:23:07 2014 From: lists at onerussian.com (Yaroslav Halchenko) Date: Wed, 8 Oct 2014 10:23:07 -0400 Subject: [Cython] Cython 0.21 released In-Reply-To: <54107F05.7080105@behnel.de> References: <54107F05.7080105@behnel.de> Message-ID: <20141008142307.GJ18756@onerussian.com> congrats! while preparing build for Debian I ran into good old (never got feedback to my analysis done in Jul/Aug, thread "Cython bugfix release") ====================================================================== ERROR: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/yoh/deb/gits/build-area/cython-0.21/Cython/Debugger/Tests/TestLibCython.py", line 280, in test_all sys.stderr.write(errmsg) UnicodeEncodeError: 'ascii' codec can't encode characters in position 23604-23607: ordinal not in range(128) and few other failures: ====================================================================== FAIL: test_typing_function_int_loop (Cython.Tests.TestJediTyper.TestJediTyper) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/yoh/deb/gits/build-area/cython-0.21/Cython/Tests/TestJediTyper.py", line 85, in test_typing_function_int_loop self.assertIn(('func', (1, 0)), types) AssertionError: ('func', (1, 0)) not found in {} ====================================================================== FAIL: test_typing_function_int_loop (Cython.Tests.TestJediTyper.TestTypeInjection) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/yoh/deb/gits/build-area/cython-0.21/Cython/Tests/TestJediTyper.py", line 85, in test_typing_function_int_loop self.assertIn(('func', (1, 0)), types) AssertionError: ('func', (1, 0)) not found in {} ---------------------------------------------------------------------- Ran 8775 tests in 2249.908s FAILED (failures=2, errors=1) are they known/fixed? On Wed, 10 Sep 2014, Stefan Behnel wrote: > Hi all, > on behalf of the Cython dev team, I'm pleased to announce the release of > Cython 0.21, a major feature release. Thanks everyone who contributed code, > documentation improvements, test feedback, bug reports and/or otherwise > helpful insights for this release. > I copied the complete changelog of this release below, but here are the > major highlights, in addition to the many bug fixes and general > improvements in this release. > * Legacy code and support for old CPython versions was removed, baselines > are now Python 2.6 and 3.2. > * cdef functions finally support inner Python functions with closures. > * C enums can now be declared as "cpdef" to publish them to Python space. > * Python method calls, boolean "and/or" operators and cascaded assignments > were optimised to streamline calls and type coercions. > * The annotated HTML code output is syntax highlighted and beautified. > Have fun, > Stefan > Downloads: > https://pypi.python.org/pypi/Cython/0.21 > http://cython.org/release/Cython-0.21.tar.gz > http://cython.org/release/Cython-0.21.zip > Signatures: > http://cython.org/release/Cython-0.21.tar.gz.asc > http://cython.org/release/Cython-0.21.zip.asc > Changelog: > https://github.com/cython/cython/blob/c67e895414aac90dfe9f789530143cff5b2ec7ad/CHANGES.rst > Documentation: > http://docs.cython.org/ > 0.21 (2014-09-10) > ================= > Features added > -------------- > * C (cdef) functions allow inner Python functions. > * Enums can now be declared as cpdef to export their values to > the module's Python namespace. Cpdef enums in pxd files export > their values to their own module, iff it exists. > * Allow @staticmethod decorator to declare static cdef methods. > This is especially useful for declaring "constructors" for > cdef classes that can take non-Python arguments. > * Taking a ``char*`` from a temporary Python string object is safer > in more cases and can be done inside of non-trivial expressions, > including arguments of a function call. A compile time error > is raised only when such a pointer is assigned to a variable and > would thus exceed the lifetime of the string itself. > * Generators have new properties ``__name__`` and ``__qualname__`` > that provide the plain/qualified name of the generator function > (following CPython 3.5). See http://bugs.python.org/issue21205 > * The ``inline`` function modifier is available as a decorator > ``@cython.inline`` in pure mode. > * When cygdb is run in a virtualenv, it enables the same virtualenv > inside of the debugger. Patch by Marc Abramowitz. > * PEP 465: dedicated infix operator for matrix multiplication (A @ B). > * HTML output of annotated code uses Pygments for code highlighting > and generally received a major overhaul by Matthias Bussonier. > * IPython magic support is now available directly from Cython with > the command "%load_ext cython". Cython code can directly be > executed in a cell when marked with "%%cython". Code analysis > is available with "%%cython -a". Patch by Mart?n Gait?n. > * Simple support for declaring Python object types in Python signature > annotations. Currently requires setting the compiler directive > ``annotation_typing=True``. > * New directive ``use_switch`` (defaults to True) to optionally disable > the optimization of chained if statement to C switch statements. > * Defines dynamic_cast et al. in ``libcpp.cast`` and C++ heap data > structure operations in ``libcpp.algorithm``. > * Shipped header declarations in ``posix.*`` were extended to cover > more of the POSIX API. Patches by Lars Buitinck and Mark Peek. > Optimizations > ------------- > * Simple calls to C implemented Python functions/methods are faster. > This also speeds up many operations on builtins that Cython cannot > otherwise optimise. > * The "and"/"or" operators try to avoid unnecessary coercions of their > arguments. They now evaluate the truth value of each argument > independently and only coerce the final result of the whole expression > to the target type (e.g. the type on the left side of an assignment). > This also avoids reference counting overhead for Python values during > evaluation and generally improves the code flow in the generated C code. > * The Python expression "2 ** N" is optimised into bit shifting. > See http://bugs.python.org/issue21420 > * Cascaded assignments (a = b = ...) try to minimise the number of > type coercions. > * Calls to ``slice()`` are translated to a straight C-API call. > Bugs fixed > ---------- > * Crash when assigning memory views from ternary conditional expressions. > * Nested C++ templates could lead to unseparated ">>" characters being > generated into the C++ declarations, which older C++ compilers could > not parse. > * Sending SIGINT (Ctrl-C) during parallel cythonize() builds could > hang the child processes. > * No longer ignore local setup.cfg files for distutils in pyximport. > Patch by Martin Teichmann. > * Taking a ``char*`` from an indexed Python string generated unsafe > reference counting code. > * Set literals now create all of their items before trying to add them > to the set, following the behaviour in CPython. This makes a > difference in the rare case that the item creation has side effects > and some items are not hashable (or if hashing them has side effects, > too). > * Cython no longer generates the cross product of C functions for code > that uses memory views of fused types in function signatures (e.g. > ``cdef func(floating[:] a, floating[:] b)``). This is considered the > expected behaviour by most users and was previously inconsistent with > other structured types like C arrays. Code that really wants all type > combinations can create the same fused memoryview type under different > names and use those in the signature to make it clear which types are > independent. > * Names that were unknown at compile time were looked up as builtins at > runtime but not as global module names. Trying both lookups helps with > globals() manipulation. > * Fixed stl container conversion for typedef element types. > * ``obj.pop(x)`` truncated large C integer values of x to ``Py_ssize_t``. > * ``__init__.pyc`` is recognised as marking a package directory > (in addition to .py, .pyx and .pxd). > * Syntax highlighting in ``cython-mode.el`` for Emacs no longer > incorrectly highlights keywords found as part of longer names. > * Correctly handle ``from cython.submodule cimport name``. > * Fix infinite recursion when using super with cpdef methods. > * No-args ``dir()`` was not guaranteed to return a sorted list. > Other changes > ------------- > * The header line in the generated C files no longer contains the > timestamp but only the Cython version that wrote it. This was > changed to make builds more reproducible. > * Removed support for CPython 2.4, 2.5 and 3.1. > * The licensing implications on the generated code were clarified > to avoid legal constraints for users. > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Research Scientist, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik From stefan_ml at behnel.de Fri Oct 10 08:03:20 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 10 Oct 2014 08:03:20 +0200 Subject: [Cython] Cython 0.21 released In-Reply-To: <20141008142307.GJ18756@onerussian.com> References: <54107F05.7080105@behnel.de> <20141008142307.GJ18756@onerussian.com> Message-ID: <543776A8.2090303@behnel.de> Hi, thanks for the feedback. Yaroslav Halchenko schrieb am 08.10.2014 um 16:23: > while preparing build for Debian I ran into good old (never got feedback > to my analysis done in Jul/Aug, thread "Cython bugfix release") > > ====================================================================== > ERROR: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/yoh/deb/gits/build-area/cython-0.21/Cython/Debugger/Tests/TestLibCython.py", line 280, in test_all > sys.stderr.write(errmsg) > UnicodeEncodeError: 'ascii' codec can't encode characters in position 23604-23607: ordinal not in range(128) The debugger tests are generally broken at the moment. > and few other failures: > > ====================================================================== > FAIL: test_typing_function_int_loop (Cython.Tests.TestJediTyper.TestJediTyper) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/yoh/deb/gits/build-area/cython-0.21/Cython/Tests/TestJediTyper.py", line 85, in test_typing_function_int_loop > self.assertIn(('func', (1, 0)), types) > AssertionError: ('func', (1, 0)) not found in {} > > ====================================================================== > FAIL: test_typing_function_int_loop (Cython.Tests.TestJediTyper.TestTypeInjection) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/yoh/deb/gits/build-area/cython-0.21/Cython/Tests/TestJediTyper.py", line 85, in test_typing_function_int_loop > self.assertIn(('func', (1, 0)), types) > AssertionError: ('func', (1, 0)) not found in {} > > ---------------------------------------------------------------------- I faintly recall that there's a version dependency here, so I restricted the test runs to Jedi 0.8.1 and later. Stefan From lists at onerussian.com Fri Oct 10 20:41:27 2014 From: lists at onerussian.com (Yaroslav Halchenko) Date: Fri, 10 Oct 2014 14:41:27 -0400 Subject: [Cython] Cython 0.21 released In-Reply-To: <543776A8.2090303@behnel.de> References: <54107F05.7080105@behnel.de> <20141008142307.GJ18756@onerussian.com> <543776A8.2090303@behnel.de> Message-ID: <20141010184127.GK18756@onerussian.com> On Fri, 10 Oct 2014, Stefan Behnel wrote: > Hi, > thanks for the feedback. > Yaroslav Halchenko schrieb am 08.10.2014 um 16:23: > > while preparing build for Debian I ran into good old (never got feedback > > to my analysis done in Jul/Aug, thread "Cython bugfix release") > > ====================================================================== > > ERROR: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/home/yoh/deb/gits/build-area/cython-0.21/Cython/Debugger/Tests/TestLibCython.py", line 280, in test_all > > sys.stderr.write(errmsg) > > UnicodeEncodeError: 'ascii' codec can't encode characters in position 23604-23607: ordinal not in range(128) > The debugger tests are generally broken at the moment. thanks! I will try to exclude them then > > and few other failures: > > ====================================================================== > > FAIL: test_typing_function_int_loop (Cython.Tests.TestJediTyper.TestJediTyper) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/home/yoh/deb/gits/build-area/cython-0.21/Cython/Tests/TestJediTyper.py", line 85, in test_typing_function_int_loop > > self.assertIn(('func', (1, 0)), types) > > AssertionError: ('func', (1, 0)) not found in {} > > ====================================================================== > > FAIL: test_typing_function_int_loop (Cython.Tests.TestJediTyper.TestTypeInjection) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/home/yoh/deb/gits/build-area/cython-0.21/Cython/Tests/TestJediTyper.py", line 85, in test_typing_function_int_loop > > self.assertIn(('func', (1, 0)), types) > > AssertionError: ('func', (1, 0)) not found in {} > > ---------------------------------------------------------------------- > I faintly recall that there's a version dependency here, so I restricted > the test runs to Jedi 0.8.1 and later. cool -- cherry picked that one for the package -- thanks! -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Research Scientist, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik From niki.spahiev at gmail.com Thu Oct 9 17:57:07 2014 From: niki.spahiev at gmail.com (Niki Spahiev) Date: Thu, 09 Oct 2014 18:57:07 +0300 Subject: [Cython] Cython 0.21 released In-Reply-To: <54107F05.7080105@behnel.de> References: <54107F05.7080105@behnel.de> Message-ID: Hello, While testing on windows without setuptools, i found that cythonize.py is missing from Cython-0.21.tar.gz Niki From niki.spahiev at gmail.com Fri Oct 10 15:04:19 2014 From: niki.spahiev at gmail.com (=?UTF-8?B?0J3QuNC60L7Qu9Cw0Lk=?=) Date: Fri, 10 Oct 2014 16:04:19 +0300 Subject: [Cython] compiler bug in 0.21 Message-ID: I found a bug in cython v0.21 Test source is attached. regards, Niki -- | (\_/) This is Bunny. Copy and paste | (='.'=) Bunny into your signature to help | (")_(") him gain world domination -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cython_err.pyx Type: text/x-python Size: 134 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cy.trace Type: application/octet-stream Size: 4543 bytes Desc: not available URL: From lists at onerussian.com Sat Oct 11 16:18:41 2014 From: lists at onerussian.com (Yaroslav Halchenko) Date: Sat, 11 Oct 2014 10:18:41 -0400 Subject: [Cython] Cython 0.21 released In-Reply-To: <54107F05.7080105@behnel.de> References: <54107F05.7080105@behnel.de> Message-ID: <20141011141841.GL18756@onerussian.com> On Wed, 10 Sep 2014, Stefan Behnel wrote: > Hi all, > on behalf of the Cython dev team, I'm pleased to announce the release of > Cython 0.21, a major feature release. Thanks everyone who contributed code, > documentation improvements, test feedback, bug reports and/or otherwise > helpful insights for this release. a little change detected while down-stream testing builds in Debian (previous version was 0.20.2 and it built fine) ------------------------------------------------------------ ... vertex_format.last_shader = self for i in xrange(vertex_format.vattr_count): attr = &vertex_format.vattr[i] if attr.per_vertex == 0: continue attr.index = glGetAttribLocation(self.program, attr.name) ^ ------------------------------------------------------------ kivy/graphics/shader.pyx:448:63: Casting temporary Python object to non-numeric non-Python type I wondered if that is an intentional restriction now to restrict such casting only to numeric (and exclude the simplest form -- bytes/chars) or a regression? Cheers! -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Research Scientist, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik From stefan_ml at behnel.de Sat Oct 11 16:53:05 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 11 Oct 2014 16:53:05 +0200 Subject: [Cython] Cython 0.21 released In-Reply-To: <20141011141841.GL18756@onerussian.com> References: <54107F05.7080105@behnel.de> <20141011141841.GL18756@onerussian.com> Message-ID: <54394451.8080504@behnel.de> Yaroslav Halchenko schrieb am 11.10.2014 um 16:18: > On Wed, 10 Sep 2014, Stefan Behnel wrote: >> on behalf of the Cython dev team, I'm pleased to announce the release of >> Cython 0.21, a major feature release. Thanks everyone who contributed code, >> documentation improvements, test feedback, bug reports and/or otherwise >> helpful insights for this release. > > a little change detected while down-stream testing builds in Debian > (previous version was 0.20.2 and it built fine) > > ------------------------------------------------------------ > ... > vertex_format.last_shader = self > for i in xrange(vertex_format.vattr_count): > attr = &vertex_format.vattr[i] > if attr.per_vertex == 0: > continue > attr.index = glGetAttribLocation(self.program, attr.name) > ^ > ------------------------------------------------------------ > > kivy/graphics/shader.pyx:448:63: Casting temporary Python object to non-numeric non-Python type Wow, interesting piece of code. What's that even supposed to do? Looking up their code, I find that "attr.name" is a char*: https://github.com/kivy/kivy/blob/master/kivy/graphics/vertex.pxd So the above code creates a temporary Python bytes object by copying data from a char*, then gets the char* to the internal object buffer and throws the object away, thus deleting its buffer. Then it passes that invalidated char* into a function. I can't see how this makes any sense. And I'm happy to see that Cython catches this kind of bug now. > I wondered if that is an intentional restriction now to restrict such casting > only to numeric (and exclude the simplest form -- bytes/chars) or a > regression? It seems they fixed their code already: https://github.com/kivy/kivy/commit/827bd6c7b7d04ec72cb3bdbf0ffcd90630d90008 Stefan From lists at onerussian.com Sat Oct 11 17:13:23 2014 From: lists at onerussian.com (Yaroslav Halchenko) Date: Sat, 11 Oct 2014 11:13:23 -0400 Subject: [Cython] kivy patch for: Cython 0.21 released In-Reply-To: <54394451.8080504@behnel.de> References: <54107F05.7080105@behnel.de> <20141011141841.GL18756@onerussian.com> <54394451.8080504@behnel.de> Message-ID: <20141011151322.GM18756@onerussian.com> On Sat, 11 Oct 2014, Stefan Behnel wrote: > Yaroslav Halchenko schrieb am 11.10.2014 um 16:18: > > On Wed, 10 Sep 2014, Stefan Behnel wrote: > >> on behalf of the Cython dev team, I'm pleased to announce the release of > >> Cython 0.21, a major feature release. Thanks everyone who contributed code, > >> documentation improvements, test feedback, bug reports and/or otherwise > >> helpful insights for this release. > > a little change detected while down-stream testing builds in Debian > > (previous version was 0.20.2 and it built fine) > > ------------------------------------------------------------ > > ... > > vertex_format.last_shader = self > > for i in xrange(vertex_format.vattr_count): > > attr = &vertex_format.vattr[i] > > if attr.per_vertex == 0: > > continue > > attr.index = glGetAttribLocation(self.program, attr.name) > > ^ > > ------------------------------------------------------------ > > kivy/graphics/shader.pyx:448:63: Casting temporary Python object to non-numeric non-Python type > Wow, interesting piece of code. What's that even supposed to do? > Looking up their code, I find that "attr.name" is a char*: > https://github.com/kivy/kivy/blob/master/kivy/graphics/vertex.pxd > So the above code creates a temporary Python bytes object by copying data > from a char*, then gets the char* to the internal object buffer and throws > the object away, thus deleting its buffer. Then it passes that invalidated > char* into a function. I can't see how this makes any sense. And I'm happy > to see that Cython catches this kind of bug now. > > I wondered if that is an intentional restriction now to restrict such casting > > only to numeric (and exclude the simplest form -- bytes/chars) or a > > regression? > It seems they fixed their code already: > https://github.com/kivy/kivy/commit/827bd6c7b7d04ec72cb3bdbf0ffcd90630d90008 Gotcha -- THANKS a bunch for a detailed response! CCing kivy maintainers in Debian -- get ready for upcoming cython 0.21 upload - a little patch to pick up! ;) -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Research Scientist, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik From lists at onerussian.com Sun Oct 12 05:04:22 2014 From: lists at onerussian.com (Yaroslav Halchenko) Date: Sat, 11 Oct 2014 23:04:22 -0400 Subject: [Cython] s3ql and llfuse FTBFS Re: Cython 0.21 released In-Reply-To: <54107F05.7080105@behnel.de> References: <54107F05.7080105@behnel.de> Message-ID: <20141012030422.GN18756@onerussian.com> On Wed, 10 Sep 2014, Stefan Behnel wrote: > on behalf of the Cython dev team, I'm pleased to announce the release of > Cython 0.21, a major feature release. Thanks everyone who contributed code, > documentation improvements, test feedback, bug reports and/or otherwise > helpful insights for this release. in two packages (s3ql, python-llfuse) so far I have ran into Traceback (most recent call last): File "setup.py", line 304, in main() File "setup.py", line 182, in main command_options={ 'sdist': { 'formats': ('setup.py', 'bztar') } }, File "/usr/lib/python3.4/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands self.run_command(cmd) File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command cmd_obj.run() File "setup.py", line 228, in run **options) File "/usr/lib/python3/dist-packages/Cython/Compiler/Main.py", line 620, in compile options = CompilationOptions(defaults = options, **kwds) File "/usr/lib/python3/dist-packages/Cython/Compiler/Main.py", line 501, in __init__ ', '.join(unknown_options))) ValueError: got unexpected compilation options: warning_errors, recursive looking at s3ql those are provided to cython_compile call and packages built successfully before. I see that recursive option was removed in 0.20b1~505 so not sure how it built before with 0.20.2 (probably providing "bogus" options just didn't trigger this ValueError). Would you advise on the ideal course of patching? (CCing maintainers of those packages) P.S. besides those few of manageable failures I have reported, haven't found any other new hiccups (there also was a failing unittest in bzr but not even yet sure if cython related or just a fluke), so will shortly upload 0.21 to Debian sid and backports to -devel repository of NeuroDebian. Cheers! -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Research Scientist, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik From stefan_ml at behnel.de Sun Oct 12 06:52:55 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 12 Oct 2014 06:52:55 +0200 Subject: [Cython] s3ql and llfuse FTBFS Re: Cython 0.21 released In-Reply-To: <20141012030422.GN18756@onerussian.com> References: <54107F05.7080105@behnel.de> <20141012030422.GN18756@onerussian.com> Message-ID: <543A0927.5060001@behnel.de> Hi, thanks for bringing this up. Yaroslav Halchenko schrieb am 12.10.2014 um 05:04: > On Wed, 10 Sep 2014, Stefan Behnel wrote: >> on behalf of the Cython dev team, I'm pleased to announce the release of >> Cython 0.21, a major feature release. Thanks everyone who contributed code, >> documentation improvements, test feedback, bug reports and/or otherwise >> helpful insights for this release. > > in two packages (s3ql, python-llfuse) so far I have ran into > > Traceback (most recent call last): > File "setup.py", line 304, in > main() > File "setup.py", line 182, in main > command_options={ 'sdist': { 'formats': ('setup.py', 'bztar') } }, > File "/usr/lib/python3.4/distutils/core.py", line 148, in setup > dist.run_commands() > File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands > self.run_command(cmd) > File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command > cmd_obj.run() > File "setup.py", line 228, in run > **options) > File "/usr/lib/python3/dist-packages/Cython/Compiler/Main.py", line 620, in compile > options = CompilationOptions(defaults = options, **kwds) > File "/usr/lib/python3/dist-packages/Cython/Compiler/Main.py", line 501, in __init__ > ', '.join(unknown_options))) > ValueError: got unexpected compilation options: warning_errors, recursive > > looking at s3ql those are provided to cython_compile call and packages built > successfully before. > > I see that recursive option was removed in 0.20b1~505 so not sure how it built > before with 0.20.2 (probably providing "bogus" options just didn't trigger this > ValueError). Would you advise on the ideal course of patching? (CCing > maintainers of those packages) Those options were previously ignored, so they can safely be removed. I actually think it would have been nicer (blaming myself here) to start by raising a visible warning first rather than a hard error. I wasn't aware of the impact that silently ignoring these options had in the past. I'll relax it for 0.21.1, which is close anyway. > P.S. besides those few of manageable failures I have reported, haven't found > any other new hiccups (there also was a failing unittest in bzr but not even > yet sure if cython related or just a fluke), so will shortly upload 0.21 to > Debian sid and backports to -devel repository of NeuroDebian. Thanks! Stefan From stefan_ml at behnel.de Sun Oct 12 06:52:45 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 12 Oct 2014 06:52:45 +0200 Subject: [Cython] accidental breakage of 0.20.x branch Message-ID: <543A091D.1040502@behnel.de> Hi, I accidentally updated the 0.20.x branch to recent 0.21.x (typo), and then had to force push it over to fix it. Sorry for that, I hope it doesn't cause too much trouble. Stefan From stefan_ml at behnel.de Wed Oct 15 07:58:37 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 15 Oct 2014 07:58:37 +0200 Subject: [Cython] Release candidate for Cython 0.21.1 Message-ID: <543E0D0D.3040009@behnel.de> Hi everyone, I uploaded a candidate for a 0.21.1 bug fix release. http://cython.org/release/Cython-0.21.1pre.tar.gz It adds a few minor new features and some bug fixes and corrections. Please give it a quick try, especially if you reported problems that this release should fix. I had to manually select changes from the master branch, so I hope I didn't forget anything. Stefan Features added -------------- * New ``cythonize`` option ``-a`` to generate the annotated HTML source view. * Missing C-API declarations in ``cpython.unicode`` were added. * Passing ``language='c++'`` into cythonize() globally enables C++ mode for all modules that were not passed as Extension objects (i.e. only source files and file patterns). * ``Py_hash_t`` is a known type (used in CPython for hash values). * ``PySlice_*()`` C-API functions are available from the ``cpython.slice`` module. * Allow arrays of C++ classes. Bugs fixed ---------- * Reference leak for non-simple Python expressions in boolean and/or expressions. * To fix a name collision and to reflect availability on host platforms, standard C declarations [ clock(), time(), struct tm and tm* functions ] were moved from posix/time.pxd to a new libc/time.pxd. Patch by Charles Blake. * Rerunning unmodified modules in IPython's cython support failed. Patch by Matthias Bussonier. * Casting C++ ``std::string`` to Python byte strings failed when auto-decoding was enabled. * Fatal exceptions in global module init code could lead to crashes if the already created module was used later on (e.g. through a stale reference in sys.modules or elsewhere). Other changes ------------- * Compilation no longer fails hard when unknown compilation options are passed. Instead, it raises a warning and ignores them (as it did silently before 0.21). This will be changed back to an error in a future release. From patricksnape at gmail.com Wed Oct 15 20:27:46 2014 From: patricksnape at gmail.com (Patrick Snape) Date: Wed, 15 Oct 2014 19:27:46 +0100 Subject: [Cython] Strange bug? with np.int64_t Message-ID: System information: Ubuntu 14.04 x64 gcc: Version 4.8.2 (Ubuntu 4.8.2-19ubuntu1), Target: x86_64-linux-gnu To reproduce: # distutils: language = c++ import numpy as np cimport numpy as np cimport cython cdef test_np_int64(const np.int64_t test_var): cdef np.int64_t error = test_var / 2 This fails to compile with the error: In function ?const __pyx_t_5numpy_int64_t __Pyx_div___pyx_t_5numpy_int64_t__const__(__pyx_t_5numpy_int64_t, __pyx_t_5numpy_int64_t)?: error: assignment of read-only variable ?q? q -= ((r != 0) & ((r ^ b) < 0)); Using just a regular int type: # distutils: language = c++ import numpy as np cimport numpy as np cimport cython cdef test_int(const int test_var): cdef int no_error = test_var / 2 Works as expected. Fixing this is as simple as removing the const that is being complained about. I assume this is a bug? I can't see why I wouldn't be allowed to perform that operation? Regards, Patrick -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Thu Oct 16 22:46:31 2014 From: matthew.brett at gmail.com (Matthew Brett) Date: Thu, 16 Oct 2014 13:46:31 -0700 Subject: [Cython] Travis-ci builds of OSX wheels In-Reply-To: References: Message-ID: Hi, On Sat, Jul 5, 2014 at 6:37 PM, Robert Bradshaw wrote: > On Sat, Jul 5, 2014 at 6:27 PM, Matthew Brett wrote: >> Hi, >> >> On Sun, Jul 6, 2014 at 1:31 AM, Robert Bradshaw wrote: >>> On Fri, Jul 4, 2014 at 3:04 PM, Matthew Brett wrote: >>>> On Fri, Jul 4, 2014 at 5:51 AM, Robert Bradshaw wrote: >>>>> Sure! >>>>> >>>>> On Thu, Jul 3, 2014 at 3:49 AM, Matthew Brett wrote: >>>>>> Hi, >>>>>> >>>>>> We just got scikit-image OSX wheel builds more or less fully automated >>>>>> on travis-ci : >>>>>> >>>>>> https://travis-ci.org/scikit-image/scikit-image-wheels >>>>>> https://github.com/scikit-image/scikit-image-wheels >>>>>> >>>>>> Any interest in getting the same thing working for Cython? >>>>>> >>>>>> Cheers, >>>>>> >>>>>> Matthew >>>> >>>> Done: >>>> >>>> https://github.com/matthew-brett/cython-wheels >>>> https://travis-ci.org/matthew-brett/cython-wheels >>>> http://wheels.scikit-image.org >>>> >>>> When y'all have permission, you should be able to click on the refresh >>>> button on the travis page and get built wheels in the http directory, >>>> after 20 minutes or so. It's set up to checkout and build the latest >>>> tag'ed commit. >>>> >>>> Feel free to just clone this repo and push to the Cython organization. >>>> I can then send you a PR updating the rackspace credentials to match >>>> the new repo. >>> >>> Forked at https://github.com/cython/cython-wheels >> >> OK - then I will try deleting my repo and see what happens. >> >> Can you activate travis testing for that repo? Then I'll set the credentials. Actually, I've since found (I believe) a better way of doing this, which allows me to keep updating the build system as travis changes, and that is to house the wheel building at the MacPython organization, and give y'all admin access to the repo so you can push and trigger builds as necessary. New version of repo here: https://github.com/MacPython/cython-wheels New builds of Cython 0.21 here: http://wheels.scikit-image.org/ I've sent admin invites for this repo to you (Robert) and Stefan (I think) - y'all can invite more people if you like. As usual, pressing the rebuild button on the travis page: https://travis-ci.org/MacPython/cython-wheels will (by default) trigger a build of the latest tagged version, Cheers, Matthew From robertwb at gmail.com Fri Oct 17 00:03:44 2014 From: robertwb at gmail.com (Robert Bradshaw) Date: Thu, 16 Oct 2014 15:03:44 -0700 Subject: [Cython] Travis-ci builds of OSX wheels In-Reply-To: References: Message-ID: Thanks. On Thu, Oct 16, 2014 at 1:46 PM, Matthew Brett wrote: > Hi, > > On Sat, Jul 5, 2014 at 6:37 PM, Robert Bradshaw > wrote: > > On Sat, Jul 5, 2014 at 6:27 PM, Matthew Brett > wrote: > >> Hi, > >> > >> On Sun, Jul 6, 2014 at 1:31 AM, Robert Bradshaw > wrote: > >>> On Fri, Jul 4, 2014 at 3:04 PM, Matthew Brett > wrote: > >>>> On Fri, Jul 4, 2014 at 5:51 AM, Robert Bradshaw > wrote: > >>>>> Sure! > >>>>> > >>>>> On Thu, Jul 3, 2014 at 3:49 AM, Matthew Brett < > matthew.brett at gmail.com> wrote: > >>>>>> Hi, > >>>>>> > >>>>>> We just got scikit-image OSX wheel builds more or less fully > automated > >>>>>> on travis-ci : > >>>>>> > >>>>>> https://travis-ci.org/scikit-image/scikit-image-wheels > >>>>>> https://github.com/scikit-image/scikit-image-wheels > >>>>>> > >>>>>> Any interest in getting the same thing working for Cython? > >>>>>> > >>>>>> Cheers, > >>>>>> > >>>>>> Matthew > >>>> > >>>> Done: > >>>> > >>>> https://github.com/matthew-brett/cython-wheels > >>>> https://travis-ci.org/matthew-brett/cython-wheels > >>>> http://wheels.scikit-image.org > >>>> > >>>> When y'all have permission, you should be able to click on the refresh > >>>> button on the travis page and get built wheels in the http directory, > >>>> after 20 minutes or so. It's set up to checkout and build the latest > >>>> tag'ed commit. > >>>> > >>>> Feel free to just clone this repo and push to the Cython organization. > >>>> I can then send you a PR updating the rackspace credentials to match > >>>> the new repo. > >>> > >>> Forked at https://github.com/cython/cython-wheels > >> > >> OK - then I will try deleting my repo and see what happens. > >> > >> Can you activate travis testing for that repo? Then I'll set the > credentials. > > Actually, I've since found (I believe) a better way of doing this, > which allows me to keep updating the build system as travis changes, > and that is to house the wheel building at the MacPython organization, > and give y'all admin access to the repo so you can push and trigger > builds as necessary. > > New version of repo here: > > https://github.com/MacPython/cython-wheels > > New builds of Cython 0.21 here: > > http://wheels.scikit-image.org/ > > I've sent admin invites for this repo to you (Robert) and Stefan (I > think) - y'all can invite more people if you like. > > As usual, pressing the rebuild button on the travis page: > > https://travis-ci.org/MacPython/cython-wheels > > will (by default) trigger a build of the latest tagged version, > > Cheers, > > Matthew > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Fri Oct 17 00:41:13 2014 From: matthew.brett at gmail.com (Matthew Brett) Date: Thu, 16 Oct 2014 15:41:13 -0700 Subject: [Cython] Travis-ci builds of OSX wheels In-Reply-To: References: Message-ID: On Thu, Oct 16, 2014 at 3:03 PM, Robert Bradshaw wrote: > Thanks. > > On Thu, Oct 16, 2014 at 1:46 PM, Matthew Brett > wrote: >> >> Hi, >> >> On Sat, Jul 5, 2014 at 6:37 PM, Robert Bradshaw >> wrote: >> > On Sat, Jul 5, 2014 at 6:27 PM, Matthew Brett >> > wrote: >> >> Hi, >> >> >> >> On Sun, Jul 6, 2014 at 1:31 AM, Robert Bradshaw >> >> wrote: >> >>> On Fri, Jul 4, 2014 at 3:04 PM, Matthew Brett >> >>> wrote: >> >>>> On Fri, Jul 4, 2014 at 5:51 AM, Robert Bradshaw >> >>>> wrote: >> >>>>> Sure! >> >>>>> >> >>>>> On Thu, Jul 3, 2014 at 3:49 AM, Matthew Brett >> >>>>> wrote: >> >>>>>> Hi, >> >>>>>> >> >>>>>> We just got scikit-image OSX wheel builds more or less fully >> >>>>>> automated >> >>>>>> on travis-ci : >> >>>>>> >> >>>>>> https://travis-ci.org/scikit-image/scikit-image-wheels >> >>>>>> https://github.com/scikit-image/scikit-image-wheels >> >>>>>> >> >>>>>> Any interest in getting the same thing working for Cython? >> >>>>>> >> >>>>>> Cheers, >> >>>>>> >> >>>>>> Matthew >> >>>> >> >>>> Done: >> >>>> >> >>>> https://github.com/matthew-brett/cython-wheels >> >>>> https://travis-ci.org/matthew-brett/cython-wheels >> >>>> http://wheels.scikit-image.org >> >>>> >> >>>> When y'all have permission, you should be able to click on the >> >>>> refresh >> >>>> button on the travis page and get built wheels in the http directory, >> >>>> after 20 minutes or so. It's set up to checkout and build the latest >> >>>> tag'ed commit. >> >>>> >> >>>> Feel free to just clone this repo and push to the Cython >> >>>> organization. >> >>>> I can then send you a PR updating the rackspace credentials to match >> >>>> the new repo. >> >>> >> >>> Forked at https://github.com/cython/cython-wheels >> >> >> >> OK - then I will try deleting my repo and see what happens. >> >> >> >> Can you activate travis testing for that repo? Then I'll set the >> >> credentials. >> >> Actually, I've since found (I believe) a better way of doing this, >> which allows me to keep updating the build system as travis changes, >> and that is to house the wheel building at the MacPython organization, >> and give y'all admin access to the repo so you can push and trigger >> builds as necessary. >> >> New version of repo here: >> >> https://github.com/MacPython/cython-wheels >> >> New builds of Cython 0.21 here: >> >> http://wheels.scikit-image.org/ >> >> I've sent admin invites for this repo to you (Robert) and Stefan (I >> think) - y'all can invite more people if you like. >> >> As usual, pressing the rebuild button on the travis page: >> >> https://travis-ci.org/MacPython/cython-wheels >> >> will (by default) trigger a build of the latest tagged version, By the way - if you would consider giving me access to Cython pypi, I am very happy to upload these wheels, Cheers, Matthew From lists at onerussian.com Fri Oct 17 07:18:00 2014 From: lists at onerussian.com (Yaroslav Halchenko) Date: Fri, 17 Oct 2014 01:18:00 -0400 Subject: [Cython] Release candidate for Cython 0.21.1 In-Reply-To: <543E0D0D.3040009@behnel.de> References: <543E0D0D.3040009@behnel.de> Message-ID: <20141017051800.GP18756@onerussian.com> FWIW -- uploaded to Debian sid. 0.21.0 experienced some issues during tests on some exotic platforms but probably not due to cython's issues (tried manually on sparc -- built fine): https://buildd.debian.org/status/package.php?p=cython&suite=unstable Cheers and thanks for keeping Cython going strong! On Wed, 15 Oct 2014, Stefan Behnel wrote: > Hi everyone, > I uploaded a candidate for a 0.21.1 bug fix release. > http://cython.org/release/Cython-0.21.1pre.tar.gz > It adds a few minor new features and some bug fixes and corrections. Please > give it a quick try, especially if you reported problems that this release > should fix. I had to manually select changes from the master branch, so I > hope I didn't forget anything. > Stefan > Features added > -------------- > * New ``cythonize`` option ``-a`` to generate the annotated HTML source > view. > * Missing C-API declarations in ``cpython.unicode`` were added. > * Passing ``language='c++'`` into cythonize() globally enables C++ mode for > all modules that were not passed as Extension objects (i.e. only source > files and file patterns). > * ``Py_hash_t`` is a known type (used in CPython for hash values). > * ``PySlice_*()`` C-API functions are available from the ``cpython.slice`` > module. > * Allow arrays of C++ classes. > Bugs fixed > ---------- > * Reference leak for non-simple Python expressions in boolean and/or > expressions. > * To fix a name collision and to reflect availability on host platforms, > standard C declarations [ clock(), time(), struct tm and tm* functions ] > were moved from posix/time.pxd to a new libc/time.pxd. Patch by Charles > Blake. > * Rerunning unmodified modules in IPython's cython support failed. > Patch by Matthias Bussonier. > * Casting C++ ``std::string`` to Python byte strings failed when > auto-decoding was enabled. > * Fatal exceptions in global module init code could lead to crashes > if the already created module was used later on (e.g. through a > stale reference in sys.modules or elsewhere). > Other changes > ------------- > * Compilation no longer fails hard when unknown compilation options are > passed. Instead, it raises a warning and ignores them (as it did > silently before 0.21). This will be changed back to an error in a > future release. > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Research Scientist, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik From stefan_ml at behnel.de Sat Oct 18 16:10:56 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 18 Oct 2014 16:10:56 +0200 Subject: [Cython] Cython 0.21.1 released Message-ID: <544274F0.4040902@behnel.de> Hi everyone, I uploaded Cython 0.21.1. It adds a few minor new features and some bug fixes and corrections. Downloads: https://pypi.python.org/pypi/Cython/0.21.1 http://cython.org/release/Cython-0.21.1.tar.gz http://cython.org/release/Cython-0.21.1.zip Changelog: https://github.com/cython/cython/blob/862b7869ac95e786475fcf8ac216a7c0c3378dc2/CHANGES.rst Documentation: http://docs.cython.org/ Stefan Features added -------------- * New ``cythonize`` option ``-a`` to generate the annotated HTML source view. * Missing C-API declarations in ``cpython.unicode`` were added. * Passing ``language='c++'`` into cythonize() globally enables C++ mode for all modules that were not passed as Extension objects (i.e. only source files and file patterns). * ``Py_hash_t`` is a known type (used in CPython for hash values). * ``PySlice_*()`` C-API functions are available from the ``cpython.slice`` module. * Allow arrays of C++ classes. Bugs fixed ---------- * Reference leak for non-simple Python expressions in boolean and/or expressions. * To fix a name collision and to reflect availability on host platforms, standard C declarations [ clock(), time(), struct tm and tm* functions ] were moved from posix/time.pxd to a new libc/time.pxd. Patch by Charles Blake. * Rerunning unmodified modules in IPython's cython support failed. Patch by Matthias Bussonier. * Casting C++ ``std::string`` to Python byte strings failed when auto-decoding was enabled. * Fatal exceptions in global module init code could lead to crashes if the already created module was used later on (e.g. through a stale reference in sys.modules or elsewhere). * ``cythonize.py`` script was not installed on MS-Windows. Other changes ------------- * Compilation no longer fails hard when unknown compilation options are passed. Instead, it raises a warning and ignores them (as it did silently before 0.21). This will be changed back to an error in a future release. From yusunn at gmail.com Sun Oct 19 16:10:08 2014 From: yusunn at gmail.com (Shang Yu) Date: Sun, 19 Oct 2014 22:10:08 +0800 Subject: [Cython] wrong c file Message-ID: Hi dear all, I'm trying cyhotn(0.21) with following code (hello.pyx) cdef void cfun(): print "hello world" if __name__ == "__main__": cfun() the generated c file has following line static void "__hello" __pyx_f_5hello_cfun(void) { which can't compiled by c compiler , what's wrong with this ? Many thanks !!! From yusunn at gmail.com Mon Oct 20 00:54:40 2014 From: yusunn at gmail.com (Shang Yu) Date: Mon, 20 Oct 2014 06:54:40 +0800 Subject: [Cython] Fwd: wrong c file In-Reply-To: References: Message-ID: Sorry . I've found it's because of the following line #cython:callspec="__hello",language_level=2 ---------- Forwarded message ---------- From: Shang Yu Date: 2014-10-19 22:10 GMT+08:00 Subject: wrong c file To: cython-devel at python.org Hi dear all, I'm trying cyhotn(0.21) with following code (hello.pyx) cdef void cfun(): print "hello world" if __name__ == "__main__": cfun() the generated c file has following line static void "__hello" __pyx_f_5hello_cfun(void) { which can't compiled by c compiler , what's wrong with this ? Many thanks !!! From stefan_ml at behnel.de Fri Oct 24 10:36:56 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 24 Oct 2014 10:36:56 +0200 Subject: [Cython] aritmetic with arrays in Cython In-Reply-To: References: <53E45A84.2070308@behnel.de> <5698943.UtWdqSqk1C@sleeky> <53E4A639.4000507@behnel.de> <53E50A9A.5050805@behnel.de> <53E510C7.4090903@googlemail.com> Message-ID: <544A0FA8.8050506@behnel.de> Ian Henriksen schrieb am 09.08.2014 um 00:15: > On Fri, Aug 8, 2014 at 3:09 PM, Ian Henriksen wrote: >> On Fri, Aug 8, 2014 at 12:02 PM, Julian Taylor wrote: >> Stefan, thanks for the direction. I think I know how to proceed now. It'll >> probably take me some time to learn how to do all this, but I'll start >> reading/working on it now. >> >> Julian, from a dependency standpoint I'd really prefer to use numpy. As >> near as I can tell, the people most likely to use this feature already use >> numpy. That would also remove the C++ dependency for this. I think the main >> reason for using eigen is that it will be easy to interface with and will >> eliminate temporaries automatically. In the long term, it could be good to >> offer multiple backends for these sorts of operations. Numpy would be a >> primary candidate for something like that. I'll spend some time working >> with numpy's ufunc api to see if I can figure out a good way to use that >> instead. > > Maybe I should clarify a little about why eigen is a good place to start. > According to http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html it > already takes care of things like the elimination of temporary variables > and common subexpression reduction at compile time. This should make it > possible to compile array expressions in Cython without having to > re-implement those sorts of optimizations. Ideally we would just have to > map memory view operations to corresponding equivalents from eigen. It's > not yet clear to me how to do things with arbitrary-dimensional arrays or > broadcasting, but, given some more time, a solution may present itself. Did anything come out of this yet? Stefan From insertinterestingnamehere at gmail.com Fri Oct 24 15:18:19 2014 From: insertinterestingnamehere at gmail.com (Ian Henriksen) Date: Fri, 24 Oct 2014 07:18:19 -0600 Subject: [Cython] aritmetic with arrays in Cython In-Reply-To: <544A0FA8.8050506@behnel.de> References: <53E45A84.2070308@behnel.de> <5698943.UtWdqSqk1C@sleeky> <53E4A639.4000507@behnel.de> <53E50A9A.5050805@behnel.de> <53E510C7.4090903@googlemail.com> <544A0FA8.8050506@behnel.de> Message-ID: On Fri, Oct 24, 2014 at 2:36 AM, Stefan Behnel wrote: > Ian Henriksen schrieb am 09.08.2014 um 00:15: > > On Fri, Aug 8, 2014 at 3:09 PM, Ian Henriksen wrote: > >> On Fri, Aug 8, 2014 at 12:02 PM, Julian Taylor wrote: > >> Stefan, thanks for the direction. I think I know how to proceed now. > It'll > >> probably take me some time to learn how to do all this, but I'll start > >> reading/working on it now. > >> > >> Julian, from a dependency standpoint I'd really prefer to use numpy. As > >> near as I can tell, the people most likely to use this feature already > use > >> numpy. That would also remove the C++ dependency for this. I think the > main > >> reason for using eigen is that it will be easy to interface with and > will > >> eliminate temporaries automatically. In the long term, it could be good > to > >> offer multiple backends for these sorts of operations. Numpy would be a > >> primary candidate for something like that. I'll spend some time working > >> with numpy's ufunc api to see if I can figure out a good way to use that > >> instead. > > > > Maybe I should clarify a little about why eigen is a good place to start. > > According to http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html it > > already takes care of things like the elimination of temporary variables > > and common subexpression reduction at compile time. This should make it > > possible to compile array expressions in Cython without having to > > re-implement those sorts of optimizations. Ideally we would just have to > > map memory view operations to corresponding equivalents from eigen. It's > > not yet clear to me how to do things with arbitrary-dimensional arrays or > > broadcasting, but, given some more time, a solution may present itself. > > Did anything come out of this yet? > > Stefan > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel > Unfortunately, nothing directly on an syntax tree transformer. I've been working on a related scipy pull request to expose their blas and lapack wrappers to cython. There are a few things left to add before it is merged there, but it should be a helpful start to making linear algebra operations more available in cython. See https://github.com/scipy/scipy/pull/4021. I'm a little overwhelmed this semester, so that is coming slower than I'd like, but I'm hoping to finish in the next two weeks or so. >From there, I've done a bit more reading, and it seems the best approach is to work on the wrappers for continuum's dynd array library. It already has a design similar to numpy and it's bindings are already written in cython. See https://github.com/ContinuumIO/libdynd and https://github.com/ContinuumIO/dynd-python, particularly https://github.com/ContinuumIO/libdynd/blob/master/docs/DESIGN.txt Getting n-dimensional array arithmetic should just be a matter of exposing their cython bindings to external packages properly. They don't have the buffer interface from pep 3118 set up yet though, so there is more work to be done there. I'd definitely welcome your input on that plan, or on anything I've done thus far. Thank you for your ideas and feedback. -Ian Henriksen -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan_ml at behnel.de Sat Oct 25 19:07:23 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 25 Oct 2014 19:07:23 +0200 Subject: [Cython] Automatic conversion with fixed-size C arrays In-Reply-To: References: Message-ID: <544BD8CB.5060606@behnel.de> Kurt Smith schrieb am 16.07.2014 um 19:01: > Being able to convert between simple C structs and Python dictionaries is > great, but it doesn't work if there is a fixed-size array field in the > struct. It seems like the following struct should be convertible safely: > > cdef struct state_t: > int i, j, k > float x[3] > float v[3] > > Along with that, I'd expect that converting between fixed-sized C arrays > and Python iterables should work as well: > > def auto_convert(list ll): > cdef int arr[10] = ll > # ... > return arr I implemented this, it's in master. There were multiple places in the code where array assignments were either blocked or special cased (or not special cased yet), so many that I'm sure I missed some. Please give it a try and report any problems. I also had to dig through some custom type conversion code and convert it from C to Cython to make it support more complex type conversions automatically. What should work now is: - assigning C arrays by value (which came more as a side effect) - copying Python iterables to C arrays on assignment (also recursively) - coercing C arrays to Python lists (and tuples if you cast) Pull requests that add more tests are *very* welcome. See carray_coercion.pyx for stuff that works and e_ass.pyx for some error tests. Stefan From daniele at grinta.net Sat Oct 25 03:36:50 2014 From: daniele at grinta.net (Daniele Nicolodi) Date: Sat, 25 Oct 2014 03:36:50 +0200 Subject: [Cython] [PATCH] explain how to compile C++ extensions up to Cython 0.21 Message-ID: <1414201010-32178-1-git-send-email-daniele@grinta.net> --- docs/src/userguide/wrapping_CPlusPlus.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/src/userguide/wrapping_CPlusPlus.rst b/docs/src/userguide/wrapping_CPlusPlus.rst index 59c89a1..5119b63 100644 --- a/docs/src/userguide/wrapping_CPlusPlus.rst +++ b/docs/src/userguide/wrapping_CPlusPlus.rst @@ -141,6 +141,21 @@ Note that the ``language`` option has no effect on user provided Extension objects that are passed into ``cythonize()``. It is only used for modules found by file name (as in the example above). +The ``cythonize()`` function in Cython versions up to 0.21 does not +recognize the ``language`` option and it needs to be specified as an +option to an :class:`Extension` that describes your extension and that +is then handled by ``cythonize()`` as follows:: + + from distutils.core import setup, Extension + from Cython.Build import cythonize + + setup(ext_modules = cythonize(Extension( + "rect", # the extesion name + sources=["rect.pyx", "Rectangle.cpp"], # the Cython source and + # additional C++ source files + language="c++", # generate and compile C++ code + ))) + The options can also be passed directly from the source file, which is often preferable (and overrides any global option). Starting with version 0.17, Cython also allows to pass external source files into the -- 2.1.1 From stefan_ml at behnel.de Thu Oct 30 19:32:15 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 30 Oct 2014 19:32:15 +0100 Subject: [Cython] [PATCH] explain how to compile C++ extensions up to Cython 0.21 In-Reply-To: <1414201010-32178-1-git-send-email-daniele@grinta.net> References: <1414201010-32178-1-git-send-email-daniele@grinta.net> Message-ID: <5452842F.6020204@behnel.de> Daniele Nicolodi schrieb am 25.10.2014 um 03:36: > --- > docs/src/userguide/wrapping_CPlusPlus.rst | 15 +++++++++++++++ > 1 file changed, 15 insertions(+) > > diff --git a/docs/src/userguide/wrapping_CPlusPlus.rst b/docs/src/userguide/wrapping_CPlusPlus.rst > index 59c89a1..5119b63 100644 > --- a/docs/src/userguide/wrapping_CPlusPlus.rst > +++ b/docs/src/userguide/wrapping_CPlusPlus.rst > @@ -141,6 +141,21 @@ Note that the ``language`` option has no effect on user provided Extension > objects that are passed into ``cythonize()``. It is only used for modules > found by file name (as in the example above). > > +The ``cythonize()`` function in Cython versions up to 0.21 does not > +recognize the ``language`` option and it needs to be specified as an > +option to an :class:`Extension` that describes your extension and that > +is then handled by ``cythonize()`` as follows:: > + > + from distutils.core import setup, Extension > + from Cython.Build import cythonize > + > + setup(ext_modules = cythonize(Extension( > + "rect", # the extesion name > + sources=["rect.pyx", "Rectangle.cpp"], # the Cython source and > + # additional C++ source files > + language="c++", # generate and compile C++ code > + ))) > + > The options can also be passed directly from the source file, which is > often preferable (and overrides any global option). Starting with > version 0.17, Cython also allows to pass external source files into the Thanks. https://github.com/cython/cython/commit/dba0a5f0e80c676367a2a137ac88d33c5ee0119a Stefan From daniele at grinta.net Fri Oct 31 13:07:11 2014 From: daniele at grinta.net (Daniele Nicolodi) Date: Fri, 31 Oct 2014 13:07:11 +0100 Subject: [Cython] [PATCH] explain how to compile C++ extensions up to Cython 0.21 In-Reply-To: <5452842F.6020204@behnel.de> References: <1414201010-32178-1-git-send-email-daniele@grinta.net> <5452842F.6020204@behnel.de> Message-ID: <54537B6F.3020604@grinta.net> On 30/10/14 19:32, Stefan Behnel wrote: > Thanks. > > https://github.com/cython/cython/commit/dba0a5f0e80c676367a2a137ac88d33c5ee0119a Hello Stefan, Thanks for applying the patch. I see that you applied it manually and not through git am or similar, otherwise my identity would have been recorded in the git metadata and you would not have to add my name to the commit log. I don't mind, but I'm wondering what is the best way to contribute patches, to make your work easier. Should I go through a pull request on github? I don't quite like this solution for trivial patches like that, because it often ends up requiring a merge, and all the merge commits make browsing the source code history harder. However, I can adapt to the project best practices. Cheers, Daniele From stefan_ml at behnel.de Fri Oct 31 13:44:29 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 31 Oct 2014 13:44:29 +0100 Subject: [Cython] [PATCH] explain how to compile C++ extensions up to Cython 0.21 In-Reply-To: <54537B6F.3020604@grinta.net> References: <1414201010-32178-1-git-send-email-daniele@grinta.net> <5452842F.6020204@behnel.de> <54537B6F.3020604@grinta.net> Message-ID: <5453842D.9090901@behnel.de> Daniele Nicolodi schrieb am 31.10.2014 um 13:07: > Thanks for applying the patch. > I see that you applied it manually and not through git am or similar, > otherwise my identity would have been recorded in the git metadata and > you would not have to add my name to the commit log. If I get a patch by email, I usually interpret it as: "here's a proposal, please apply or change as you like, I don't care about being named", unless stated otherwise. Sorry for the misunderstanding. > I don't mind, but I'm wondering what is the best way to contribute > patches, to make your work easier. Should I go through a pull request on > github? If you want to appear in the history, then that's the way to go. > I don't quite like this solution for trivial patches like that, > because it often ends up requiring a merge, and all the merge commits > make browsing the source code history harder. Merges are so common in DVCSs that most of the time I don't even notice them in the history. Except when someone else does the merge, as for pull requests. Then the distinction between committing and merging is actually relevant, as it shows which core developer takes responsibility for accepting a foreign change to the main repo. Stefan From daniele at grinta.net Fri Oct 31 13:54:37 2014 From: daniele at grinta.net (Daniele Nicolodi) Date: Fri, 31 Oct 2014 13:54:37 +0100 Subject: [Cython] [PATCH] explain how to compile C++ extensions up to Cython 0.21 In-Reply-To: <5453842D.9090901@behnel.de> References: <1414201010-32178-1-git-send-email-daniele@grinta.net> <5452842F.6020204@behnel.de> <54537B6F.3020604@grinta.net> <5453842D.9090901@behnel.de> Message-ID: <5453868D.9080402@grinta.net> On 31/10/14 13:44, Stefan Behnel wrote: > Daniele Nicolodi schrieb am 31.10.2014 um 13:07: >> Thanks for applying the patch. >> I see that you applied it manually and not through git am or similar, >> otherwise my identity would have been recorded in the git metadata and >> you would not have to add my name to the commit log. > > If I get a patch by email, I usually interpret it as: "here's a proposal, > please apply or change as you like, I don't care about being named", unless > stated otherwise. As I tried to convey in my previous email, I don't care about being credited, as long as the improvements makes into the project. The question is rather: are patches by email ok for your workflow, or do you prefer other means of sending contributions (pull requests or whatever)? > Sorry for the misunderstanding. There is no misunderstanding :) >> I don't mind, but I'm wondering what is the best way to contribute >> patches, to make your work easier. Should I go through a pull request on >> github? > > If you want to appear in the history, then that's the way to go. I don't care much about that, I just want to make your job easier. >> I don't quite like this solution for trivial patches like that, >> because it often ends up requiring a merge, and all the merge commits >> make browsing the source code history harder. > > Merges are so common in DVCSs that most of the time I don't even notice > them in the history. Except when someone else does the merge, as for pull > requests. Then the distinction between committing and merging is actually > relevant, as it shows which core developer takes responsibility for > accepting a foreign change to the main repo. If you apply a patch (in git patch format) from someone else, that someone else ends up being the author, you are recorded as the committer of the patch, so the information is there anyway. But it is just personal preference. I prefer to avoid merges when the repository history is linear, and I use rebase when my changes would require a merge because of interleaved commits. Cheers, Daniele From robertwb at gmail.com Fri Oct 31 15:59:56 2014 From: robertwb at gmail.com (Robert Bradshaw) Date: Fri, 31 Oct 2014 07:59:56 -0700 Subject: [Cython] [PATCH] explain how to compile C++ extensions up to Cython 0.21 In-Reply-To: <5453868D.9080402@grinta.net> References: <1414201010-32178-1-git-send-email-daniele@grinta.net> <5452842F.6020204@behnel.de> <54537B6F.3020604@grinta.net> <5453842D.9090901@behnel.de> <5453868D.9080402@grinta.net> Message-ID: On Oct 31, 2014 5:54 AM, "Daniele Nicolodi" wrote: > > On 31/10/14 13:44, Stefan Behnel wrote: > > Daniele Nicolodi schrieb am 31.10.2014 um 13:07: > >> Thanks for applying the patch. > >> I see that you applied it manually and not through git am or similar, > >> otherwise my identity would have been recorded in the git metadata and > >> you would not have to add my name to the commit log. > > > > If I get a patch by email, I usually interpret it as: "here's a proposal, > > please apply or change as you like, I don't care about being named", unless > > stated otherwise. > > As I tried to convey in my previous email, I don't care about being > credited, as long as the improvements makes into the project. The > question is rather: are patches by email ok for your workflow, or do you > prefer other means of sending contributions (pull requests or whatever)? > > > Sorry for the misunderstanding. > > There is no misunderstanding :) > > >> I don't mind, but I'm wondering what is the best way to contribute > >> patches, to make your work easier. Should I go through a pull request on > >> github? > > > > If you want to appear in the history, then that's the way to go. > > I don't care much about that, I just want to make your job easier. Pull requests are best, but patches are just fine as well. > >> I don't quite like this solution for trivial patches like that, > >> because it often ends up requiring a merge, and all the merge commits > >> make browsing the source code history harder. > > > > Merges are so common in DVCSs that most of the time I don't even notice > > them in the history. Except when someone else does the merge, as for pull > > requests. Then the distinction between committing and merging is actually > > relevant, as it shows which core developer takes responsibility for > > accepting a foreign change to the main repo. > > If you apply a patch (in git patch format) from someone else, that > someone else ends up being the author, you are recorded as the committer > of the patch, so the information is there anyway. > > But it is just personal preference. I prefer to avoid merges when the > repository history is linear, and I use rebase when my changes would > require a merge because of interleaved commits. > > Cheers, > Daniele > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: