[Numpy-discussion] numpy's future (1.1 and beyond): which direction(s) ?

[Gnata Xavier]

David Cournapeau wrote:
> Hi,
>
> 	numpy 1.0.5 is on the way, and I was wondering about numpy's future. I
> myself have some ideas about what could be done; has there been any
> discussion behind what is on 1.1 trac's roadmap ? Some of the things I
> would like to see myself:
> 	- a framework for plug-in architecture, that is the ability for numpy
> to load/unload some libraries at runtime, plus a common api to access
> the functions. Example: instead of calling directly atlas/etc..., it
> would load the dll at runtime, so that other libraries can be loaded
> instead (numpy itself could load different runtimes depending on the
> CPU, for example: SSE vs SSE2 vs SSE3, multi-thread vs non
> multi-thread). That would require the ability to build loadable
> libraries (numscons, or a new numpy.distutils command).
> 	- a pure C core library for some common operations. For example, I
> myself would really like to be able to use the fft in some C extensions.
> Numpy has a fft, but I cannot access it from C (well, I could access the
> python fft from C, but that would be... awkward); same for blas/lapack.
> I really like the idea of a numpy "split" into a core C library reusable
> by many C extensions, and python wrappers (in C, cython, ctypes,
> whatever). That would be a huge work, of course, but hopefully can be
> done gradually and smoothly. Only having fft + some basic blas/lapack
> (dot, inv, det, etc...) and some basic functions (beta, gamma, digamma)
> would be great, for example.
> 	- a highly optimized core library for memory copy, simple addition,
> etc... basically, everything which can see huge improvements when using
> MMX/SSE and co. This is somewhat linked to point 1. This would also
> require more sophisticated memory allocator (aligned, etc...).
>
> What do people think about this ? Is that a direction numpy developers
> are interested in ?
>
> cheers,
>
> David
>
>   
Looks great :)
Something like http://idlastro.gsfc.nasa.gov/idl_html_help/TOTAL.html 
(Thread Pool Keywords) would be nice.
A "total like" function could be a great pathfinder a put threads into 
numpy keeping the things as simple as they should remain.
Not sure we need that is numpy in 1.1 but IMHO we need that in a near 
future (because every "array oriented" libs are now threaded).

Xavier