[SciPy-dev] Fwd: [SciPy-user] MemoryError in scipy_core

[Travis Oliphant]

Chris Fonnesbeck wrote:

>>On 11/16/05, Travis Oliphant <oliphant at ee.byu.edu> wrote:
>>    
>>
>>>>Good news: the leak appears to be fixed. cbm.py is chugging along at
>>>>constant memory usage (pre-burn-in, of course). Only took 41 messages
>>>>to sort it out!
>>>>
>>>>
>>>>        
>>>>
>>>Fantastic.   Your code is actually a pretty good test because apparently
>>>you are using a lot of scalars.  The performance of the array scalars
>>>will be improved as time goes on.  I'd be interested in hearing how your
>>>code performs relative to the Numeric version because you have a lot of
>>>small arrays in your code base --- and a lot of scalars.
>>>
>>>      
>>>
>
>Travis,
>
>I have profiled the Numeric-based and scipy-based versions of my code.
>The Numeric version is just under twice as fast. The slowest bits
>appear to be in the numeric.py and oldnumeric.py modules, although the
>f2py stuff appears a little slower also. Here are the two profiles:
>
>  
>
One thing that can help immediately is that everything calling 
oldnumeric in your code should be replaced with the appropriate method 
if possible.   Your code is spending quite a bit of time in asarray 
which is simply how the old function calls are implemented.  Replacing 
these calls for objects you know are already arrays with the appropriate 
method can probably eliminate most of these asarray calls and therefore 
speed things up.  If you don't know whether the object is an array or 
not, then you, of course, still need to use the function.

The other thing to also consider is that resize should be used to 
actually change the number of elements in the array.  If all you are 
doing is reshaping the array but keeping the same number of elements, 
then reshape is more appropriate.

-Travis