[Cython] prange CEP updated

[mark florisson]

On 21 April 2011 20:13, Dag Sverre Seljebotn <d.s.seljebotn at astro.uio.no> wrote:
> On 04/21/2011 10:37 AM, Robert Bradshaw wrote:
>>
>> On Mon, Apr 18, 2011 at 7:51 AM, mark florisson
>> <markflorisson88 at gmail.com>  wrote:
>>>
>>> On 18 April 2011 16:41, Dag Sverre Seljebotn<d.s.seljebotn at astro.uio.no>
>>>  wrote:
>>>>
>>>> Excellent! Sounds great! (as I won't have my laptop for some days I
>>>> can't
>>>> have a look yet but I will later)
>>>>
>>>> You're right about (the current) buffers and the gil. A testcase
>>>> explicitly
>>>> for them would be good.
>>>>
>>>> Firstprivate etc: i think it'd be nice myself, but it is probably better
>>>> to
>>>> take a break from it at this point so that we can think more about that
>>>> and
>>>> not do anything rash; perhaps open up a specific thread on them and ask
>>>> for
>>>> more general input. Perhaps you want to take a break or task-switch to
>>>> something else (fused types?) until I can get around to review and merge
>>>> what you have so far? You'll know best what works for you though. If you
>>>> decide to implement explicit threadprivate variables because you've got
>>>> the
>>>> flow I certainly wom't object myself.
>>>>
>>>  Ok, cool, I'll move on :) I already included a test with a prange and
>>> a numpy buffer with indexing.
>>
>> Wow, you're just plowing away at this. Very cool.
>>
>> +1 to disallowing nested prange, that seems to get really messy with
>> little benefit.
>>
>> In terms of the CEP, I'm still unconvinced that firstprivate is not
>> safe to infer, but lets leave the initial values undefined rather than
>> specifying them to be NaNs (we can do that as an implementation if you
>> want), which will give us flexibility to change later once we've had a
>> chance to play around with it.
>
> I don't see any technical issues with inferring firstprivate, the question
> is whether we want to. I suggest not inferring it in order to make this
> safer: One should be able to just try to change a loop from "range" to
> "prange", and either a) have things fail very hard, or b) just work
> correctly and be able to trust the results.
>
> Note that when I suggest using NaN, it is as initial values for EACH
> ITERATION, not per-thread initialization. It is not about "firstprivate" or
> not, but about disabling thread-private variables entirely in favor of
> "per-iteration" variables.
>
> I believe that by talking about "readonly" and "per-iteration" variables,
> rather than "thread-shared" and "thread-private" variables, this can be used
> much more safely and with virtually no knowledge of the details of
> threading. Again, what's in my mind are scientific programmers with (too)
> little training.
>
> In the end it's a matter of taste and what is most convenient to more users.
> But I believe the case of needing real thread-private variables that
> preserves per-thread values across iterations (and thus also can possibly
> benefit from firstprivate) is seldomly enough used that an explicit
> declaration is OK, in particular when it buys us so much in safety in the
> common case.
>
> To be very precise,
>
> cdef double x, z
> for i in prange(n):
>    x = f(x)
>    z = f(i)
>    ...
>
> goes to
>
> cdef double x, z
> for i in prange(n):
>    x = z = nan
>    x = f(x)
>    z = f(i)
>    ...
>
> and we leave it to the C compiler to (trivially) optimize away "z = nan".
> And, yes, it is a stopgap solution until we've got control flow analysis so
> that we can outright disallow such uses of x (without threadprivate
> declaration, which also gives firstprivate behaviour).
>

I think the preliminary OpenMP support is ready for review. It
supports 'with cython.parallel.parallel:' and 'for i in
cython.parallel.prange(...):'. It works in generators and closures and
the docs are updated. Support for break/continue/with gil isn't there
yet.

There are two remaining issue. The first is warnings for potentially
uninitialized variables for prange(). When you do

for i in prange(start, stop, step): ...

it generates code like

nsteps = (stop - start) / step;
#pragma omp parallel for lastprivate(i)
for (temp = 0; temp < nsteps; temp++) {
    i = start + temp * step;
    ...
}

So here it will complain about 'i' being potentially uninitialized, as
it might not be assigned to in the loop. However, simply assigning 0
to 'i' can't work either, as you expect zero iterations not to touch
it. So for now, we have a bunch of warnings, as I don't see a
__attribute__ to suppress it selectively.

The second is NaN-ing private variables, NaN isn't part of C. For gcc,
the docs ( http://www.delorie.com/gnu/docs/glibc/libc_407.html ) have
the following to say:

"You can use `#ifdef NAN' to test whether the machine supports NaN.
(Of course, you must arrange for GNU extensions to be visible, such as
by defining _GNU_SOURCE, and then you must include `math.h'.)"

So I'm thinking that if NaN is not available (or the compiler is not
GCC), we can use FLT_MAX, DBL_MAX and LDBL_MAX instead from float.h.
Would this be the proper way to handle this?