[Matrix-SIG] Numpy in Viper

[skaller]

Pearu Peterson wrote:

[Viper/NumPy]

> >       d) Should we attempt to make the two mechanisms compatible
> >
> > It is my belief that (d) would be the best option IF it is, in fact
> > possible. (c) is the next best option.
> 
> Hi!
> 
> I am not familiar with this J stuff but I do have some questions:
> First, can you give some (rough) estimate how it would affect the
> performance?

	No. I can tell you about FISh 1 itself: quicksort,
generated as C, is faster than qsort by a significant percentage
(some figure are availeble from the FISh home page).

	When it comes to Viper built in arry handling,
as opposed to NumPy, it will depend a lot on the implementation,
in two ways: first, on the low level data structures: NumPy
is probably optimal there, and a native ocaml version can
only be slower.

	On the other hand, when it comes to higher level
constructions, NumPy is grossly inefficient. This is not
so much a fault of NumPy, but rather due the limited ability
of the Python compiler to optimise it.

	The typical problem for matrix operations is
the creation of spurious temporaries, for example,
consider:

	a = b + c + d + e - f

This can be done without any temporary storage at all,
assuming the assignment to 'a' is destructive.

In addition, there is a fairly large overhead, adding
successive pairs, compared to adding all 5 corresponding
components simultaneously.

[Viper currently does binary operations here, for
other types, but it recognizes the special case
already at won't be too hard to optimise]

> Second, how it would affect writing Py/C API's?
> I'll presume that the notion of contiguousness is there, right?

	At present, Viper does not support CPython API at all.
It may, perhaps, do so, if a Python CAPI compatible back end
is fitted to the compiler. [At present, there is no compiler,
nor any decision on the first back end to write]

	The question is: why would a _numerical_ programmer
want a C API at all, if they had a high performance compiler?
[The answer 'compatibility' is obvious. Any other?]

-- 
John Skaller, mailto:skaller@maxtal.com.au
1/10 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
downloads: http://www.triode.net.au/~skaller