[Numpy-discussion] Catching out-of-memory error before it happens

Chris Barker - NOAA Federal chris.barker at noaa.gov
Fri Jan 24 10:57:14 EST 2014


c = a + b: 3N
c = a + 2*b: 4N

Does python garbage collect mid-expression? I.e. :

C = (a + 2*b) + b

4 or 5 N?

Also note that when memory gets tight, fragmentation can be a problem. I.e.
if two size-n arrays where just freed, you still may not be able to
allocate a size-2n array. This seems to be worse on windows, not sure why.

a += b: 2N
np.add(a, b, out=a): 2N
b *= 2; a += b: 2N

Note that simply loading a and b requires 2N memory, so the latter code
samples are near-optimal.

And will run quite a bit faster for large arrays--pushing that memory
around takes time.

-Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140124/f7101f31/attachment.html>


More information about the NumPy-Discussion mailing list