[SciPy-User] <transferred from Scipy-Dev>Re: Seeking help/ advice for applying functions

[eat]

> 
> On 9 March 2010 09:59, eat <e.antero.tammi <at> gmail.com> wrote:
> > Robert Kern <robert.kern <at> gmail.com> writes:
> >
> >>
> >> Your example is not very clear. Can you write a less cryptic one with
> >> informative variable names and perhaps some comments about what each
> >> part is doing?
> >>
> >
Anne Archibald <peridot.faceted <at> gmail.com> writes:

Hi,

I moved this thread from scipy-dev to scipy-user.

First of all, Thanks Anne you clarified a lot.


> > """
> > Hi,
> >
> > I have tried to clarify my code. First part is the relevant one, the rest 
is
> > there just to provide some context to run some tests.
> 
> The short answer is that no, there's no way to optimize what you're doing.
> 
> The long answer is: when numpy and scipy are fast, they are fast
> because they avoid running python code: if you add two arrays, there's
> only one line of python code, and all the work is done by loops
> written in C. If your code is calling many different python functions,
> well, since they're python functions, to apply them at all you must
> necessarily execute python code. There goes any potential speed
> advantage. (There may be a convenience advantage; if so, you can look
> into using np.vectorize, which is just a wrapper around a python loop,
> but is convenient.)

Not only that I'm new to Scipy/ Numpy but I'm new to python aswell.
I think I assumed that once python functions get compiled they'll would be
(almost) as efficient than builtins.

> 
> That said, I assume you are considering numpy/scipy because you have
> arrays of thousands or more. It also seems unlikely that you actually
> have thousands of different functions (that's an awful lot of source
> code!). So if your "different" functions are actually just a handful
> (or fewer) pieces of actual code, and you are getting your thousands
> of functions by wrapping them up with parameters and local variables,
> well, now there are possibilities. Exactly what possibilities depend
> on what your functions look like - which is one reason Robert Kern
> asked you to clarify your code - but they all boil down to rearranging
> the problem so that it goes back to "few functions, much data", then
> writing the functions in such a way that you can use numpy to apply
> them to thousands or millions of data points at once.

Yes, indeed I don't have so many different 'base' functions, but just
configured (parametrized) different ways. However there are situations
when it's advantage to be able to treat your fuctions as 'black boxes'.
For example when parametrization is based on optimization.

Yes, I'll also recognize that my original question was too generic. I'll
try to be more specific next time.

> 
> > Also originally I should have posted this to Scipy-User list. Would it be
> > more appropriate to continue the discussion there?
> 
> Probably.
> 
> Anne

Thanks a lot,
eat

> 
> > Regards,
> > eat
> > """
> >
> > import numpy as np
> >
> > ## relevant part
are in scipy-dev