[SciPy-User] < transferred from Scipy-Dev> Re: Seeking help/ advice for applying functions

[Anne Archibald]

On 9 March 2010 16:53, eat <e.antero.tammi at gmail.com> wrote:
> Anne Archibald <peridot.faceted <at> gmail.com> writes:
>
>> Not especially. They are compiled to bytecode, whose execution is not
>> particularly fast. But the big problem is all the baggage of python's
>> nature as a dynamic language: for example each value is allocated with
>> malloc() and contains type information; for another example, each
>> access to a list involves identifying that the object really is a
>> list, dispatching to the list-lookup function, determining the type of
>> the argument (integer, slice object, long integer, other), and bounds
>> checking before finally returning the list element. Thus even tools
>> like cython that let you write, effectively, python code that gets
>> compiled to machine code are not much faster unless you can turn off
>> the dynamic features of python (which cython lets you do, selectively;
>> it's great).
>
> Good to know, I'll better postpone studying cython while I'm still
> learning 'the very basics' of Scipy/ Numpy.
>
>> Think about whether you can write each 'base' function to take arrays
>> as arguments, for example:
>>
>> def F(x, mu, sigma):
>>     return np.exp(-((x-mu)/sigma)**2)
>>
>> If your current code does something like
>>
>> fis = [lambda x: F(x, mui, sigmai) for (mui, sigmai) in zip(muis, sigmais)]
>>
>> r = [f(7) for f in fis]
>>
>> you can rewrite it as the single line
>>
>> r = F(7, muis, sigmais)
>>
>> (if muis and sigmais are numpy arrays). Now you have just a couple of
>> lines of python, and the heavy lifting all happens inside numpy loops.
>
> Good quidelines in general to drive ones design!
>
>> If you have several different functions, look into separating your
>> input arrays based on which function needs to be applied to them;
>
> Hmm, but I think that's what not possible in my particular case! That's
> depends on what happens externally. I don't have any control of it.

What I mean is something like:

A[A>3] = F(A[A>3])
A[A<=3] = G(A[A<=3])

Not that this is the most efficient way to write this sort of thing,
but it is a way to apply the appropriate function to each element in
the array.

>> remember numpy lets you select out all the elements of an array
>> meeting a criterion.
>
> As far as I understod, select and where gives you only the first one.
> In my case I need all of them.

There are many different functions for picking just some elements out
of an array, and for putting results back in the correct places. This
kind of operation - pull out certain elements, do some operation to
them, put the new values back in place of the old - happens all the
time, and numpy has all sorts of tools to support it.

Anne