[Cython] Cython 0.17 beta 3 released - release candidate

[mark florisson]

On 26 August 2012 22:25, Christoph Gohlke <cgohlke at uci.edu> wrote:
> On 8/26/2012 2:09 PM, Christoph Gohlke wrote:
>>
>> On 8/26/2012 4:08 AM, mark florisson wrote:
>>>
>>> On 25 August 2012 03:07, Christoph Gohlke <cgohlke at uci.edu> wrote:
>>>>
>>>> Hi,
>>>>
>>>>
>>>> On 8/24/2012 12:43 PM, Stefan Behnel wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> thanks for testing!
>>>>>
>>>>> Christoph Gohlke, 24.08.2012 07:20:
>>>>>>
>>>>>>
>>>>>> I tested Cython-0.17b3 on Windows 7 with Visual Studio compilers.
>>>>>>
>>>>>> 32 bit Python 2.7 works well, only 4 test failures.
>>>>>
>>>>>
>>>>>
>>>>> Three of those errors are in OpenMP tests - is OpenMP supported in your
>>>>> build environment?
>>>>
>>>>
>>>>
>>>> OpenMP is available on my system, and parallel.pyd is linked to the
>>>> openmp
>>>> library. The prange tests fail only sometimes. On my system, the prange
>>>> index is sometimes left at the start (zero) of the range, while the
>>>> tests
>>>> expect the index to be left at the stop of the range. According to the
>>>> Cython prange enhancements webpage "the iterations of the loop body
>>>> can be
>>>> executed in any order" <http://wiki.cython.org/enhancements/prange>.
>>>> Where
>>>> does that leave the loop index?
>>>>
>>>
>>> I should be the value from the last iteration, but in my experience
>>> many compilers have buggy OpenMP implementations. I think your
>>> compiler doesn't correctly support the lastprivate clause in all
>>> situations. For instance, test_prange fails (which doesn't have any
>>> break/return/exceptions), which simply has a lastprivate clause and a
>>> schedule clause set to 'dynamic'. The test without the schedule clause
>>> works fine (or maybe it's just luck). Or maybe it doesn't support
>>> multiple lastprivate() clauses? I'm not entirely sure... It also seems
>>> the thread limit on your system is 1.
>>>
>>> In any case, the generated code for these tests looks correct to me,
>>> but we've had similar problems before with different compilers...
>>
>>
>> A minimal example that fails for me is:
>>
>> def test_parallel():
>>      cdef int i = 0, s = 0
>>      with nogil, cython.parallel.parallel():
>>          for i in prange(10):
>>              s += i
>>      return i
>>
>> The returned value is often 0, otherwise 9 as expected.
>>
>> In the generated C code I see
>>
>>      #pragma omp for firstprivate(__pyx_v_i) lastprivate(__pyx_v_i)
>>
>> If I change this to
>>
>>      #pragma omp parallel for firstprivate(__pyx_v_i)
>> lastprivate(__pyx_v_i)
>>
>> the function always returns the expected value. Does that make sense?
>
>
>
> Well, apparently it doesn't make sense because the value of s is not
> correct.
>
> Christoph
>

That will use nested parallelism.

>
>>
>> Thanks for your help and patience.
>>
>> Christoph
>>
>>
>>>
>>>>
>>>>>
>>>>> The other one is the new "initial_file_path" test that fails with this
>>>>> linker error:
>>>>>
>>>>> """
>>>>> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DLL
>>>>> /nologo /INCREMENTAL:NO /INCREMENTAL:YES        /DEBUG
>>>>> /LIBPATH:X:\Python27\libs
>>>>> /LIBPATH:X:\Python27\PCbuild /EXPORT:init__init__
>>>>> build\temp.win32-2.7\Release\my_test_package\__init__.obj
>>>>>
>>>>>
>>>>> /OUT:\Cython-0.17b3\BUILD\run\initial_file_path\my_test_package\__init__.pyd
>>>>>
>>>>> /IMPLIB:build\temp.win32-2.7\Release\my_test_package\__init__.lib
>>>>>
>>>>>
>>>>> /MANIFESTFILE:build\temp.win32-2.7\Release\my_test_package\__init__.pyd.manifest
>>>>>
>>>>>
>>>>> LINK : error LNK2001: unresolved external symbol init__init__
>>>>> """
>>>>>
>>>>> Maybe the Windows build of distutils is broken here - it seems to
>>>>> assume
>>>>> the wrong module name for the package module.
>>>>
>>>>
>>>>
>>>> I think this is an issue with the test. The extension does compile
>>>> and link
>>>> outside of the tests.
>>>>
>>>>
>>>>
>>>>>
>>>>> I guess compiling package modules is just an overall badly supported
>>>>> feature in CPython...
>>>>>
>>>>>
>>>>>> On 64 bit Python 2.7 and 3.2 with msvc9 compiler, python.exe crashes
>>>>>> during
>>>>>> `test_slice_assignment (memslice.__test__)`. I tested two
>>>>>> computers. The
>>>>>> Windows executive can not identify in which specific module it
>>>>>> crashes,
>>>>>> and
>>>>>> neither enabling faulthandler nor building with debug symbols gives
>>>>>> any
>>>>>> useful information. Can anyone reproduce this? It seems compiler
>>>>>> specific
>>>>>> since Python 3.3, which is using msvc10, does not crash.
>>>>>
>>>>>
>>>>>
>>>>> Hmm, yes, sounds like a problem with the compiler. Would be good to get
>>>>> this sorted out, but it's almost impossible to debug something like
>>>>> this
>>>>> from a distance.
>>>>
>>>>
>>>>
>>>>
>>>> Maybe the following simple example is related. It fails (not crash) when
>>>> compiled with 64 bit msvc9, but does work with 32 bit msvc9 and
>>>> msvc10 (32
>>>> and 64 bit):
>>>>
>>>> ```
>>>> from cython.view cimport array as cvarray
>>>> import numpy as np
>>>>
>>>> narr = np.arange(8 * 14 * 11).reshape((8, 14, 11))
>>>>
>>>> cdef int[:, :, ::1] a = narr
>>>> cdef int[:, :, :] b = a[2:8:2, -4:1:-1, 1:3]
>>>>
>>>> print narr[2:8:2, -4:1:-1, 1:3].shape
>>>> print b.shape[0], b.shape[1], b.shape[2]
>>>> ```
>>>>
>>>> On win-amd64-py2.x the shape of b is (3, 9, 3) but it should be (3,
>>>> 9, 2)
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>> When disabling
>>>>>> test_slice_assignment, runtests.py completes with many failures.
>>>>>>
>>>>>> The results of `runtests.py -v -v` are at
>>>>>> <http://www.lfd.uci.edu/~gohlke/pythonlibs/tests/cython/>.
>>>>>
>>>>>
>>>>>
>>>>> The 64bit output looks so broken that I wonder what went wrong here. I
>>>>> mean, most of the problems look like this:
>>>>>
>>>>> """
>>>>> Expected:
>>>>>       Traceback (most recent call last):
>>>>>       TypeError: m() takes at most 2 positional arguments (3 given)
>>>>> Got:
>>>>>       Traceback (most recent call last):
>>>>>       TypeError: m() takes at most %Id positional argument%s (%Id
>>>>> given)
>>>>> """
>>>>>
>>>>> I have no idea how that can happen.
>>>>>
>>>>> I can see two other problems, one is the linker warning about the
>>>>> module
>>>>> init function in Py3:
>>>>>
>>>>> """
>>>>> bufaccess.obj : warning LNK4197: export 'PyInit_bufaccess' specified
>>>>> multiple times; using first specification
>>>>> """
>>>>
>>>>
>>>>
>>>> This is "normal".
>>>>
>>>>
>>>>
>>>>>
>>>>> The other one is about mixing Py_ssize_t and int (and some other) types
>>>>> all
>>>>> over the place:
>>>>>
>>>>> """
>>>>> bufaccess.c(3033) : warning C4244: '=' : conversion from
>>>>> 'Py_ssize_t' to
>>>>> 'int', possible loss of data
>>>>> """
>>>>>
>>>>> Some of them look like we'd need an explicit cast in the C code
>>>>> somewhere,
>>>>> others might hint at lax type usage in tests.
>>>>>
>>>>> There's also this:
>>>>>
>>>>> """
>>>>> C:\Program Files (x86)\Microsoft Visual Studio
>>>>> 10.0\VC\INCLUDE\xlocale(323)
>>>>> : warning C4530: C++ exception handler used, but unwind semantics
>>>>> are not
>>>>> enabled. Specify /EHsc
>>>>> """
>>>>>
>>>>> Stefan
>>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Christoph
>>>>
>>>> _______________________________________________
>>>> cython-devel mailing list
>>>> cython-devel at python.org
>>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>> _______________________________________________
>>> cython-devel mailing list
>>> cython-devel at python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>>
>>>
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>>
>>
>>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel