[Numpy-discussion] Deprecating PyDataMem_RENEW ?
Timothy Hochberg
tim.hochberg at ieee.org
Mon May 5 12:59:46 EDT 2008
On Mon, May 5, 2008 at 5:44 AM, David Cournapeau <
david at ar.media.kyoto-u.ac.jp> wrote:
> Hi,
>
> While working again on the fftpack module, to clean things up and
> speed some backends (in particular fftw3, which is really sub-optimal
> right now), I remembered how much unaligned data pointer in numpy arrays
> hurt performances. So I would like to relaunch the discussion on aligned
> allocators and default alignement for numpy arrays :
>
> http://www.mail-archive.com/numpy-discussion@scipy.org/msg04005.html
>
> Basically, what I have in mind is, in a first step (for numpy 1.2):
> - define functions to allocate on a given alignement
> - make PyMemData_NEW 16 byte aligned by default (to be compatible
> with SSE and co).
>
> The problem was, and still is, realloc. It is not possible to implement
> realloc with malloc/free (in a portable way), and as such, it is not
> possible to have an aligned realloc.
>
> In numpy, we can always replace realloc by malloc/free, because we know
> the size of the old block: would deprecating PyMemData_RENEW and
> replacing them by PyMemeData_NEW/PyMemData_FREE be possible, such as to
> make all numpy arrays follow a default alignement ? There are only a few
> of them in numpy (6 of them), 0 in scipy, and I guess extensions never
> really used them ?
I don't think you would want to do this in the core of PyArray_FromIter;
presumably realloc can sometimes reuse the existing pointer and save on
allocating a new chunk of memory. Since there are lots of allocations in
fromiter, this could potentially be a big performance hit. (At least I think
so, realloc has always been kinda voodoo to me). One could use
PyMemData_NEW/PyMemData_FREE in the final allocation to make sure that the
data is alligned, we allready do a realloc there to dump any extra space.
Or, possibly better, one could choose which allocation strategy to use here
depending on whether the data was alligned or not.
>
>
> cheers,
>
> David
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
--
. __
. |-\
.
. tim.hochberg at ieee.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20080505/389c9a04/attachment.html>
More information about the NumPy-Discussion
mailing list