[Tutor] improving speed using and recalling C functions

[Gabriele Brambilla]

but main is the program that contains everything.

I used the profile in this way:

import cProfile

import pstats

def mymain():

#all the code

#end of main indentation

cProfile.run('mymain()', 'restats', 'time')

p = pstats.Stats('restats')

p.strip_dirs().sort_stats('name')

p.sort_stats('time').print_stats(10)

So all the function I used are contained in main(), so even all the others
that are appeared.

Gabriele

2014-04-10 19:41 GMT-04:00 Martin A. Brown <martin at linux-ip.net>:

>
> Gabriele,
>
>
>          21071736 function calls in 199.883 seconds
>>
>
> The 21 million function calls isn't really a surprise to me, given 18
> million calls to file.write().  Given that the majority of the time is
> still spent in skymaps5.py, I think you'll need to instrument that a bit
> more to figure out where the hotspot is.
>
>    Ordered by: internal time
>>   List reduced from 188 to 10 due to restriction <10>
>>
>>   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>>        1  149.479  149.479  199.851  199.851 skymaps5.py:16(mymain)
>> 18101000   28.682    0.000   28.682    0.000 {method 'write' of 'file'
>> objects}
>>
>> the major time is required by mymain that is the whole program. the write
>> on file is an operation that I do in the end but if I increase the number
>> of data it doesn't increase (I tried doubing the sample of values and I
>> know why it's behaving in this way) the third are interpolate and kappa:
>> the two functions (one called inside the other one)
>>
>
> This is a good finding, in fact.  Now, you know which module contains the
> bottleneck.  Is your CPU pegged when you run that skymap5.py code?  Can you
> run the profiler on the skymaps5.py code, too, to locate the specific
> hotspot?  Great to see that you are using the profiler, effectively, too!
>
>
>  So they are the ones that are taking time.
>> Now what can I do?
>>
>
> I think you now need to profile skymaps5.py code?  Specifically the stuff
> in main().
>
> OK, so I have not myself used scipy.interpolate.interp1d before, but I
> went to have a look at it.  So, you feed interp1d() an x and a y (e.g. a
> plot line on a diagram), and it essentially produces its best guess of a
> function which will fit that curve, correct?
>
> Well, this is how it performs with random input and an identity function:
>
>   element count 100, duration 0.000
>   element count 1000, duration 0.001
>   element count 10000, duration 0.005
>   element count 100000, duration 0.055
>   element count 1000000, duration 0.858
>   element count 10000000, duration 30.404
>
> So, with 10 million inputs on an admittedly brain-dead function, there's
> not a performance bottleneck.  If you can find the parts of your
> skymaps5.py code that are the bottleneck, then you could post it here.
>
> I hadn't thought of using interpolate, myself, as I didn't even know it
> existed.
>
> Thanks and good luck,
>
> -Martin
>
>
> --
> Martin A. Brown
> http://linux-ip.net/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20140410/5c9bf66c/attachment.html>