From stefan_ml at behnel.de Sun Oct 14 12:20:44 2018 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 14 Oct 2018 18:20:44 +0200 Subject: [Cython] Cython 0.29 final released Message-ID: <54b97eb3-6a7c-c1bd-3f40-2e5917aee7bf@behnel.de> Dear Cythonistas, after half a year of development, many community pull requests and a lot of feedback and good ideas in online discussions and at conferences, I'm proud to release Cython 0.29. This is a major feature release that comes with many great improvements and several important bug fixes. See the long list of changes below. Download: https://pypi.org/project/Cython/0.29/ https://github.com/cython/cython/releases/tag/0.29 Changelog: https://github.com/cython/cython/blob/0.29/CHANGES.rst Foresight: Given that Cython has been in critical production use all over the world for several years, but never found the perfect time for a 1.0 version bump, we designate this to become the last 0.x release of the project and decided to skip the 1.0 release which the 0.x series has long represented anyway. Planning has already started [1] for the next major release, titled "3.0". It will finally switch the default Cython language level from Py2 to Py3, to match what users expect from a Python compiler these days without additional options or configuration. Cython 2 code will continue to be supported as before with the directive "language_level=2", although the future default (called "3str", Py3 with str literals) is already available in the 0.29 release, so users might prefer switching to that. We're happy to hear your feedback on the proposed changes. [1] https://github.com/cython/cython/milestone/58 Have fun, Stefan 0.29 (2018-10-14) ================= Features added -------------- * PEP-489 multi-phase module initialisation has been enabled again. Module reloads in other subinterpreters raise an exception to prevent corruption of the static module state. * A set of ``mypy`` compatible PEP-484 declarations were added for Cython's C data types to integrate with static analysers in typed Python code. They are available in the ``Cython/Shadow.pyi`` module and describe the types in the special ``cython`` module that can be used for typing in Python code. Original patch by Julian Gethmann. (Github issue #1965) * Memoryviews are supported in PEP-484/526 style type declarations. (Github issue #2529) * ``@cython.nogil`` is supported as a C-function decorator in Python code. (Github issue #2557) * Raising exceptions from nogil code will automatically acquire the GIL, instead of requiring an explicit ``with gil`` block. * C++ functions can now be declared as potentially raising both C++ and Python exceptions, so that Cython can handle both correctly. (Github issue #2615) * ``cython.inline()`` supports a direct ``language_level`` keyword argument that was previously only available via a directive. * A new language level name ``3str`` was added that mostly corresponds to language level 3, but keeps unprefixed string literals as type 'str' in both Py2 and Py3, and the builtin 'str' type unchanged. This will become the default in the next Cython release and is meant to help user code a) transition more easily to this new default and b) migrate to Python 3 source code semantics without making support for Python 2.x difficult. * In CPython 3.6 and later, looking up globals in the module dict is almost as fast as looking up C globals. (Github issue #2313) * For a Python subclass of an extension type, repeated method calls to non-overridden cpdef methods can avoid the attribute lookup in Py3.6+, which makes them 4x faster. (Github issue #2313) * (In-)equality comparisons of objects to integer literals are faster. (Github issue #2188) * Some internal and 1-argument method calls are faster. * Modules that cimport many external extension types from other Cython modules execute less import requests during module initialisation. * Constant tuples and slices are deduplicated and only created once per module. (Github issue #2292) * The coverage plugin considers more C file extensions such as ``.cc`` and ``.cxx``. (Github issue #2266) * The ``cythonize`` command accepts compile time variable values (as set by ``DEF``) through the new ``-E`` option. Patch by Jerome Kieffer. (Github issue #2315) * ``pyximport`` can import from namespace packages. Patch by Prakhar Goel. (Github issue #2294) * Some missing numpy and CPython C-API declarations were added. Patch by John Kirkham. (Github issues #2523, #2520, #2537) * Declarations for the ``pylifecycle`` C-API functions were added in a new .pxd file ``cpython.pylifecycle``. * The Pythran support was updated to work with the latest Pythran 0.8.7. Original patch by Adrien Guinet. (Github issue #2600) * ``%a`` is included in the string formatting types that are optimised into f-strings. In this case, it is also automatically mapped to ``%r`` in Python 2.x. * New C macro ``CYTHON_HEX_VERSION`` to access Cython's version in the same style as ``PY_HEX_VERSION``. * Constants in ``libc.math`` are now declared as ``const`` to simplify their handling. * An additional ``check_size`` clause was added to the ``ctypedef class`` name specification to allow suppressing warnings when importing modules with backwards-compatible ``PyTypeObject`` size changes. Patch by Matti Picus. (Github issue #2627) Bugs fixed ---------- * The exception handling in generators and coroutines under CPython 3.7 was adapted to the newly introduced exception stack. Users of Cython 0.28 who want to support Python 3.7 are encouraged to upgrade to 0.29 to avoid potentially incorrect error reporting and tracebacks. (Github issue #1958) * Crash when importing a module under Stackless Python that was built for CPython. Patch by Anselm Kruis. (Github issue #2534) * 2-value slicing of typed sequences failed if the start or stop index was None. Patch by Christian Gibson. (Github issue #2508) * Multiplied string literals lost their factor when they are part of another constant expression (e.g. 'x' * 10 + 'y' => 'xy'). * String formatting with the '%' operator didn't call the special ``__rmod__()`` method if the right side is a string subclass that implements it. (Python issue 28598) * The directive ``language_level=3`` did not apply to the first token in the source file. (Github issue #2230) * Overriding cpdef methods did not work in Python subclasses with slots. Note that this can have a performance impact on calls from Cython code. (Github issue #1771) * Fix declarations of builtin or C types using strings in pure python mode. (Github issue #2046) * Generator expressions and lambdas failed to compile in ``@cfunc`` functions. (Github issue #459) * Global names with ``const`` types were not excluded from star-import assignments which could lead to invalid C code. (Github issue #2621) * Several internal function signatures were fixed that lead to warnings in gcc-8. (Github issue #2363) * The numpy helper functions ``set_array_base()`` and ``get_array_base()`` were adapted to the current numpy C-API recommendations. Patch by Matti Picus. (Github issue #2528) * Some NumPy related code was updated to avoid deprecated API usage. Original patch by jbrockmendel. (Github issue #2559) * Several C++ STL declarations were extended and corrected. Patch by Valentin Valls. (Github issue #2207) * C lines of the module init function were unconditionally not reported in exception stack traces. Patch by Jeroen Demeyer. (Github issue #2492) * When PEP-489 support is enabled, reloading the module overwrote any static module state. It now raises an exception instead, given that reloading is not actually supported. * Object-returning, C++ exception throwing functions were not checking that the return value was non-null. Original patch by Matt Wozniski (Github Issue #2603) * The source file encoding detection could get confused if the ``c_string_encoding`` directive appeared within the first two lines. (Github issue #2632) * Cython generated modules no longer emit a warning during import when the size of the NumPy array type is larger than what was found at compile. Instead, this is assumed to be a backwards compatible change on NumPy side. Other changes ------------- * Cython now emits a warning when no ``language_level`` (2, 3 or '3str') is set explicitly, neither as a ``cythonize()`` option nor as a compiler directive. This is meant to prepare the transition of the default language level from currently Py2 to Py3, since that is what most new users will expect these days. The future default will, however, not enforce unicode literals, because this has proven a major obstacle in the support for both Python 2.x and 3.x. The next major release is intended to make this change, so that it will parse all code that does not request a specific language level as Python 3 code, but with ``str`` literals. The language level 2 will continue to be supported for an indefinite time. * The documentation was restructured, cleaned up and examples are now tested. The NumPy tutorial was also rewritten to simplify the running example. Contributed by Gabriel de Marmiesse. (Github issue #2245) * Cython compiles less of its own modules at build time to reduce the installed package size to about half of its previous size. This makes the compiler slightly slower, by about 5-7%. From J.Demeyer at UGent.be Tue Oct 16 06:42:58 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Tue, 16 Oct 2018 12:42:58 +0200 Subject: [Cython] Failing test reimport_from_subinterpreter Message-ID: <5BC5C0B2.2080202@UGent.be> Hello, I'm getting various reports of the test reimport_from_subinterpreter failing (within Sage, Cython 0.29, Python 2.7.15). Annoyingly, the problem can only be reproduced when running the full Cython testsuite, not when running the test in isolation. I'll continue to investigate, but I'm already posting it here in case somebody has an idea. The error is not very enlightening (is there a way to get the exception from the subinterpreter?): End-to-end reimport_from_subinterpreter ... /home/jdemeyer/sage-test/local/bin/python2 setup.py build_ext --inplace Compiling package/subtest.pyx because it changed. Compiling subtest.pyx because it changed. [1/2] Cythonizing package/subtest.pyx [2/2] Cythonizing subtest.pyx running build_ext building 'package.subtest' extension creating build creating build/temp.linux-ppc64le-2.7 creating build/temp.linux-ppc64le-2.7/package gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -fPIC -I/home/jdemeyer/sage-test/local/include/python2.7 -c package/subtest.c -o build/temp.linux-ppc64le-2.7/package/subtest.o gcc -pthread -shared -L/home/jdemeyer/sage-test/local/lib -Wl,-rpath,/home/jdemeyer/sage-test/local/lib -L/home/jdemeyer/sage-test/local/lib -Wl,-rpath,/home/jdemeyer/sage-test/local/lib build/temp.linux-ppc64le-2.7/package/subtest.o -L/home/jdemeyer/sage-test/local/lib -lpython2.7 -o /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/run/reimport_from_subinterpreter/package/subtest.so building 'subtest' extension gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -fPIC -I/home/jdemeyer/sage-test/local/include/python2.7 -c subtest.c -o build/temp.linux-ppc64le-2.7/subtest.o gcc -pthread -shared -L/home/jdemeyer/sage-test/local/lib -Wl,-rpath,/home/jdemeyer/sage-test/local/lib -L/home/jdemeyer/sage-test/local/lib -Wl,-rpath,/home/jdemeyer/sage-test/local/lib build/temp.linux-ppc64le-2.7/subtest.o -L/home/jdemeyer/sage-test/local/lib -lpython2.7 -o /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/run/reimport_from_subinterpreter/subtest.so /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/run/reimport_from_subinterpreter/package/subtest.pyx tree = Parsing.p_module(s, pxd, full_module_name) /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/run/reimport_from_subinterpreter/subtest.pyx tree = Parsing.p_module(s, pxd, full_module_name) /home/jdemeyer/sage-test/local/bin/python2 -c "import subtest; subtest.run_main()" Module loaded: package.subtest /home/jdemeyer/sage-test/local/bin/python2 -c "import subtest; subtest.run_sub()" Traceback (most recent call last): File "", line 1, in ImportError: No module named package Traceback (most recent call last): File "", line 1, in File "subtest.pyx", line 44, in subtest.run_sub assert 0 == run_in_subinterpreter(b'import package') AssertionError FAIL From stefan_ml at behnel.de Tue Oct 16 14:30:25 2018 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 16 Oct 2018 20:30:25 +0200 Subject: [Cython] Failing test reimport_from_subinterpreter In-Reply-To: <5BC5C0B2.2080202@UGent.be> References: <5BC5C0B2.2080202@UGent.be> Message-ID: Jeroen Demeyer schrieb am 16.10.2018 um 12:42: > I'm getting various reports of the test reimport_from_subinterpreter > failing (within Sage, Cython 0.29, Python 2.7.15). > > Annoyingly, the problem can only be reproduced when running the full Cython > testsuite, not when running the test in isolation. I'll continue to > investigate, but I'm already posting it here in case somebody has an idea. > The error is not very enlightening (is there a way to get the exception > from the subinterpreter?): > > End-to-end reimport_from_subinterpreter ... > /home/jdemeyer/sage-test/local/bin/python2 setup.py build_ext --inplace > Compiling package/subtest.pyx because it changed. > Compiling subtest.pyx because it changed. > [1/2] Cythonizing package/subtest.pyx > [2/2] Cythonizing subtest.pyx > running build_ext > building 'package.subtest' extension > creating build > creating build/temp.linux-ppc64le-2.7 > creating build/temp.linux-ppc64le-2.7/package > gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused > -fPIC -I/home/jdemeyer/sage-test/local/include/python2.7 -c > package/subtest.c -o build/temp.linux-ppc64le-2.7/package/subtest.o > gcc -pthread -shared -L/home/jdemeyer/sage-test/local/lib > -Wl,-rpath,/home/jdemeyer/sage-test/local/lib > -L/home/jdemeyer/sage-test/local/lib > -Wl,-rpath,/home/jdemeyer/sage-test/local/lib > build/temp.linux-ppc64le-2.7/package/subtest.o > -L/home/jdemeyer/sage-test/local/lib -lpython2.7 -o > /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/run/reimport_from_subinterpreter/package/subtest.so > > building 'subtest' extension > gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused > -fPIC -I/home/jdemeyer/sage-test/local/include/python2.7 -c subtest.c -o > build/temp.linux-ppc64le-2.7/subtest.o > gcc -pthread -shared -L/home/jdemeyer/sage-test/local/lib > -Wl,-rpath,/home/jdemeyer/sage-test/local/lib > -L/home/jdemeyer/sage-test/local/lib > -Wl,-rpath,/home/jdemeyer/sage-test/local/lib > build/temp.linux-ppc64le-2.7/subtest.o -L/home/jdemeyer/sage-test/local/lib > -lpython2.7 -o > /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/run/reimport_from_subinterpreter/subtest.so > > > /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/Cython/Compiler/Main.py:367: > FutureWarning: Cython directive 'language_level' not set, using 2 for now > (Py2). This will change in a later release! File: > /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/run/reimport_from_subinterpreter/package/subtest.pyx > > ? tree = Parsing.p_module(s, pxd, full_module_name) > /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/Cython/Compiler/Main.py:367: > FutureWarning: Cython directive 'language_level' not set, using 2 for now > (Py2). This will change in a later release! File: > /home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/run/reimport_from_subinterpreter/subtest.pyx > > ? tree = Parsing.p_module(s, pxd, full_module_name) > > > /home/jdemeyer/sage-test/local/bin/python2 -c "import subtest; > subtest.run_main()" > Module loaded: package.subtest > > > > /home/jdemeyer/sage-test/local/bin/python2 -c "import subtest; > subtest.run_sub()" > > Traceback (most recent call last): > ? File "", line 1, in > ImportError: No module named package The error message is right above. ^^^ > Traceback (most recent call last): > ? File "", line 1, in > ? File "subtest.pyx", line 44, in subtest.run_sub > ??? assert 0 == run_in_subinterpreter(b'import package') > AssertionError Difficult to say why this would fail to find the package. Could it be an import path problem? Current directory missing from the PYTHONPATH or something like that? Stefan From J.Demeyer at UGent.be Sun Oct 21 06:47:16 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Sun, 21 Oct 2018 12:47:16 +0200 Subject: [Cython] Failing test reimport_from_subinterpreter In-Reply-To: References: <5BC5C0B2.2080202@UGent.be> Message-ID: <5BCC5934.4030803@UGent.be> On 2018-10-16 20:30, Stefan Behnel wrote: > Difficult to say why this would fail to find the package. Could it be an > import path problem? Current directory missing from the PYTHONPATH or > something like that? The problem is that the error depends on the proverbial "phase of the moon". When running the full testsuite, I always get the error. But any attempt at debugging it makes the error go away. So it must be something subtle, like a race condition or bad interaction with another test. From stefan_ml at behnel.de Sun Oct 21 09:44:11 2018 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 21 Oct 2018 15:44:11 +0200 Subject: [Cython] Failing test reimport_from_subinterpreter In-Reply-To: <5BCC5934.4030803@UGent.be> References: <5BC5C0B2.2080202@UGent.be> <5BCC5934.4030803@UGent.be> Message-ID: <26037f69-86b6-5669-a40e-b7e91cac53f3@behnel.de> Jeroen Demeyer schrieb am 21.10.2018 um 12:47: > On 2018-10-16 20:30, Stefan Behnel wrote: >> Difficult to say why this would fail to find the package. Could it be an >> import path problem? Current directory missing from the PYTHONPATH or >> something like that? > > The problem is that the error depends on the proverbial "phase of the > moon". When running the full testsuite, I always get the error. But any > attempt at debugging it makes the error go away. So it must be something > subtle, like a race condition or bad interaction with another test. Maybe print "sys.path" and "sys.meta_path" in a couple of places? Before and while running the test suite? There are pyximport tests, for example, and "pyxim..." < "reimp...". It does sound like something in the import system is leaking from previous tests. Stefan From robertwb at gmail.com Mon Oct 22 05:03:29 2018 From: robertwb at gmail.com (Robert Bradshaw) Date: Mon, 22 Oct 2018 11:03:29 +0200 Subject: [Cython] longintrepr.h issues with MinGW Message-ID: Given that https://bugs.python.org/issue4709 results in extension modules that seem to work, but silently produce completely incorrect answers, I'm thinking we should either disable our long-unpacking code on these platforms, or at the very least give a runtime error if we detect issues like sizeof(void*) != SIZEOF_VOID_P (or both). I'm not sure what has made this issue pop up more lately (Python 3), but perhaps we should even consider releasing bugfixes for previous Cython versions. - Robert From J.Demeyer at UGent.be Mon Oct 22 10:44:23 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Mon, 22 Oct 2018 16:44:23 +0200 Subject: [Cython] Failing test reimport_from_subinterpreter In-Reply-To: <11852e98d3f948ecb15e53fc07b639cc@xmail103.UGent.be> References: <5BC5C0B2.2080202@UGent.be> <5BCC5934.4030803@UGent.be> <11852e98d3f948ecb15e53fc07b639cc@xmail103.UGent.be> Message-ID: <5BCDE247.8050303@UGent.be> On 2018-10-21 15:44, Stefan Behnel wrote: > Maybe print "sys.path" and "sys.meta_path" in a couple of places? So I did and the problem is indeed related to sys.path. When running *only* the reimport_from_subinterpreter tests, sys.path starts with the following entries: '' '/home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/support/lib.linux-ppc64le-2.7' '/home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src' '/home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/run/reimport_from_subinterpreter' # <---- '/home/jdemeyer/sage-test/local/lib/python27.zip' '/home/jdemeyer/sage-test/local/lib/python2.7' When running all tests, sys.path starts with the following entries: '' '/home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/support/lib.linux-ppc64le-2.7' '/home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src' '/home/jdemeyer/sage-test/local' # <---- '/home/jdemeyer/sage-test/local/lib/python27.zip' '/home/jdemeyer/sage-test/local/lib/python2.7' The entries marked with # <---- are different but I don't know why. Further suggestions for debugging are more than welcome... In both cases, the initial entry ('') is dropped in the sub-interpreter. I assume that this is a feature. From J.Demeyer at UGent.be Mon Oct 22 10:55:30 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Mon, 22 Oct 2018 16:55:30 +0200 Subject: [Cython] Failing test reimport_from_subinterpreter In-Reply-To: <6a93bd74076045768b5afcd2110e8d2e@xmail103.UGent.be> References: <5BC5C0B2.2080202@UGent.be> <5BCC5934.4030803@UGent.be> <11852e98d3f948ecb15e53fc07b639cc@xmail103.UGent.be> <6a93bd74076045768b5afcd2110e8d2e@xmail103.UGent.be> Message-ID: <5BCDE4E2.3070902@UGent.be> n 2018-10-22 16:44, Jeroen Demeyer wrote: > The entries marked with # <---- are different but I don't know why. There is a difference in $PYTHONPATH. When it works, PYTHONPATH equals '/home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/support/lib.linux-ppc64le-2.7:/home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src:' Note the trailing colon, which is what makes everything work since that gets translated to the current working directory! When it fails, PYTHONPATH equals '/home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src/TEST_TMP/support/lib.linux-ppc64le-2.7:/home/jdemeyer/sage-test/local/var/tmp/sage/build/cython-0.29/src:/home/jdemeyer/sage-test/local' ...which has sys.prefix added for some reason. From J.Demeyer at UGent.be Mon Oct 22 11:18:47 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Mon, 22 Oct 2018 17:18:47 +0200 Subject: [Cython] Failing test reimport_from_subinterpreter In-Reply-To: <1f7aeb66d4bd4bc29ea79127df211d0e@xmail103.UGent.be> References: <5BC5C0B2.2080202@UGent.be> <5BCC5934.4030803@UGent.be> <11852e98d3f948ecb15e53fc07b639cc@xmail103.UGent.be> <6a93bd74076045768b5afcd2110e8d2e@xmail103.UGent.be> <1f7aeb66d4bd4bc29ea79127df211d0e@xmail103.UGent.be> Message-ID: <5BCDEA57.5030301@UGent.be> OK, I finished analyzing the problem. It's a combination of: 1. the Sage wrapper around the Cython installer setting PYTHONPATH 2. the reimport_from_subinterpreter test failing if PYTHONPATH is set I'll open a PR quick. From stefan_ml at behnel.de Tue Oct 23 16:21:38 2018 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 23 Oct 2018 22:21:38 +0200 Subject: [Cython] longintrepr.h issues with MinGW In-Reply-To: References: Message-ID: <8a1628c2-6a34-3eb0-a2bf-9f5620e904a5@behnel.de> Robert Bradshaw schrieb am 22.10.2018 um 11:03: > Given that https://bugs.python.org/issue4709 results in extension > modules that seem to work, but silently produce completely incorrect > answers, I'm thinking we should either disable our long-unpacking code > on these platforms, or at the very least give a runtime error if we > detect issues like sizeof(void*) != SIZEOF_VOID_P (or both). We could try to detect MinGW-64 and set the "MS_WIN64" macro before importing the CPython headers. That seems to be the right fix. Can't say if we also need to define the "MS_WIN32" macro, but would probably be better to have it. CPython defines both for MSVC. There is a "__MINGW64__" macro to detect MinGW-64, it seems. > I'm not sure what has made this issue pop up more lately (Python 3), My guess is that MinGW is just fairly rarely used to build CPython extensions overall. > but perhaps we should even consider releasing bugfixes for previous > Cython versions. You mean a 0.28.6, or even older than that? As I said, MinGW does not seem to be used very widely? Stefan From robertwb at gmail.com Tue Oct 23 19:44:27 2018 From: robertwb at gmail.com (Robert Bradshaw) Date: Wed, 24 Oct 2018 01:44:27 +0200 Subject: [Cython] longintrepr.h issues with MinGW In-Reply-To: <8a1628c2-6a34-3eb0-a2bf-9f5620e904a5@behnel.de> References: <8a1628c2-6a34-3eb0-a2bf-9f5620e904a5@behnel.de> Message-ID: On Tue, Oct 23, 2018 at 10:22 PM Stefan Behnel wrote: > > Robert Bradshaw schrieb am 22.10.2018 um 11:03: > > Given that https://bugs.python.org/issue4709 results in extension > > modules that seem to work, but silently produce completely incorrect > > answers, I'm thinking we should either disable our long-unpacking code > > on these platforms, or at the very least give a runtime error if we > > detect issues like sizeof(void*) != SIZEOF_VOID_P (or both). > > We could try to detect MinGW-64 and set the "MS_WIN64" macro before > importing the CPython headers. That seems to be the right fix. Can't say if > we also need to define the "MS_WIN32" macro, but would probably be better > to have it. CPython defines both for MSVC. > > There is a "__MINGW64__" macro to detect MinGW-64, it seems. I created a minimal https://github.com/cython/cython/pull/2679 > > I'm not sure what has made this issue pop up more lately (Python 3), > > My guess is that MinGW is just fairly rarely used to build CPython > extensions overall. > > > but perhaps we should even consider releasing bugfixes for previous > > Cython versions. > > You mean a 0.28.6, or even older than that? I wish there were a good way to see what users are actually using in the wild... > As I said, MinGW does not seem to be used very widely? Yes, but it is used, and this is (IMHO) a really severe bug. From stefan_ml at behnel.de Wed Oct 24 00:53:06 2018 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 24 Oct 2018 06:53:06 +0200 Subject: [Cython] longintrepr.h issues with MinGW In-Reply-To: References: <8a1628c2-6a34-3eb0-a2bf-9f5620e904a5@behnel.de> Message-ID: <078f0ad3-4d08-9537-274b-11556053ed83@behnel.de> Robert Bradshaw schrieb am 24.10.2018 um 01:44: > On Tue, Oct 23, 2018 at 10:22 PM Stefan Behnel wrote: >> My guess is that MinGW is just fairly rarely used to build CPython >> extensions overall. >> >>> but perhaps we should even consider releasing bugfixes for previous >>> Cython versions. >> >> You mean a 0.28.6, or even older than that? > > I wish there were a good way to see what users are actually using in the wild... At least from what I hear from time to time, many people tend to stick to whatever was there when they started using Cython, and then manually upgrade in irregular intervals whenever they feel like they need something from a newer version. Especially in-house, something like, say, 0.23 might still be in use. But then, we don't really need to care about in-house use of this, because people control their environment and their C compiler there. If they didn't experience this bug yet, then it's probably not a problem for them at all. Regarding more widely used libraries that are used across different environments, I'd say that they'd probably have added support for CPython 3.7 by now. Anything before Cython 0.28 didn't even compile there, and 0.29 is recommended for it, so I think we can somewhat safely assume that this bug is most pressing for those two versions. Thanks to Mark Shannon for raising the baseline here. :) >> As I said, MinGW does not seem to be used very widely? > > Yes, but it is used, and this is (IMHO) a really severe bug. Agreed. Let's release a 0.28.6 with this fix and move on to 0.29.1. Stefan From J.Demeyer at UGent.be Wed Oct 24 01:58:24 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Wed, 24 Oct 2018 07:58:24 +0200 Subject: [Cython] longintrepr.h issues with MinGW In-Reply-To: <6cfcebc86bd5479dab250c862e2c98e4@xmail103.UGent.be> References: <6cfcebc86bd5479dab250c862e2c98e4@xmail103.UGent.be> Message-ID: <5BD00A00.4090405@UGent.be> If you care about MinGW, there is also this CPython issue which makes it basically impossible to compile any Cython extension on MinGW: https://bugs.python.org/issue11566 From J.Demeyer at UGent.be Wed Oct 24 02:02:08 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Wed, 24 Oct 2018 08:02:08 +0200 Subject: [Cython] longintrepr.h issues with MinGW In-Reply-To: <7af22ba1d6a54bf9b0bff994cafc67af@xmail103.UGent.be> References: <6cfcebc86bd5479dab250c862e2c98e4@xmail103.UGent.be> <7af22ba1d6a54bf9b0bff994cafc67af@xmail103.UGent.be> Message-ID: <5BD00AE0.2060506@UGent.be> On 2018-10-24 07:58, Jeroen Demeyer wrote: > If you care about MinGW, there is also this CPython issue which makes it > basically impossible to compile any Cython extension on MinGW: > > https://bugs.python.org/issue11566 I forgot to say that this makes it impossible to compile any *C++* Cython extension out of the box on MinGW. From insertinterestingnamehere at gmail.com Wed Oct 24 09:09:34 2018 From: insertinterestingnamehere at gmail.com (Ian Henriksen) Date: Wed, 24 Oct 2018 08:09:34 -0500 Subject: [Cython] longintrepr.h issues with MinGW In-Reply-To: <5BD00AE0.2060506@UGent.be> References: <6cfcebc86bd5479dab250c862e2c98e4@xmail103.UGent.be> <7af22ba1d6a54bf9b0bff994cafc67af@xmail103.UGent.be> <5BD00AE0.2060506@UGent.be> Message-ID: It's been a while since I've used MinGW to compile any extensions. The CRT incompatibility is a hassle to worry about unless you are using a custom Python build and MSVC 2015 and 2017 are pretty good. That said, there is a workaround for this one. See https://stackoverflow.com/a/30881190/1935144. Best, Ian On Wed, Oct 24, 2018, 1:02 AM Jeroen Demeyer wrote: > On 2018-10-24 07:58, Jeroen Demeyer wrote: > > If you care about MinGW, there is also this CPython issue which makes it > > basically impossible to compile any Cython extension on MinGW: > > > > https://bugs.python.org/issue11566 > > I forgot to say that this makes it impossible to compile any *C++* > Cython extension out of the box on MinGW. > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > https://mail.python.org/mailman/listinfo/cython-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: