[off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Todd Miller]

On Mon, 2004-06-28 at 17:14, Sebastian Haase wrote:
> On Monday 28 June 2004 12:03 pm, Todd Miller wrote:
> > On Mon, 2004-06-28 at 13:59, Chris Barker wrote:
> > > Sebastian Haase wrote:
> > > > BTW, from the profiling/timing I did you can tell that wxPyPlot
> > > > actually plots 25000 data points in 0.1 secs - so it's _really_ fast
> > > > ...
> > >
> > > Actually, it's probably not that fast, if you are timing on
> > > Linux/wxGTK/X-Windows. X is asyncronous, so what you are timing is how
> > > long it takes your program to tell X what to draw, but it may take
> > > longer than that to actually draw it. However, what you are timing is
> > > all the stuff that is effected by numarray/Numeric.
> > >
> > > I worked on part of the wxPython DC.DrawXXXList stuff, and I really
> > > wanted a Numeric native version, but Robin really didn't want an
> > > additional dependency. We discussed on this list a while back whether
> > > you could compile against Numeric, but let people run without it, and
> > > have it all work unless Someone actually used it. What makes that tricky
> > > is that the functions that test whether a PyObject is a Numeric array
> > > are in Numeric... but it could probably be done if you tried hard enough
> > > (maybe include just that function in wxPython...)
> >
> > numarray-1.0 has two macros for dealing with this:  PyArray_Present()
> > and PyArray_isArray(obj).  The former (safely) determines that numarray
> > is installed, while the latter determines that numarray is installed and
> > that obj is a NumArray.  Both macros serve to guard sections of code
> > which make more extensive use of the numarray C-API to keep them from
> > segfaulting when numarray is not installed.  I think this would be easy
> > to do for Numeric as well.
> >
> > One problem is that compiling a "numarray improved" extension requires
> > some portion of the numarray headers.  I refactored the numarray
> > includes so that a relatively simple set of 3 files can be used to
> > support the Numeric compatible interface (for numarray).  These could
> > either be included in core Python (with a successful PEP) or included in
> > interested packages.  This approach adds a small source code burden
> > somewhere,  but eliminates the requirement for users to have numarray
> > installed either to run or compile from source.
> >
> > I'll send out the draft PEP later today.
> >
> > Regards,
> > Todd
> 
> My original question was just this: Does anyone know why numarray is maybe 10 
> times slower that Numeric with that particular code segment 
> (PySequence_GetItem) ?

Well, the short answer is probably: no.

Looking at the numarray sequence protocol benchmarks in
Examples/bench.py, and looking at what wxPython is probably doing
(fetching a 1x2 element array from an Nx2 and then fetching 2 numerical
values from that)... I can't fully nail it down.  My benchmarks show
that numarray is 4x slower for fetching the two element array but only
1.1x slower for the two numbers;  that makes me expect at most 4x
slower.  

Noticing the 50k __del__ calls in your profile,  I eliminated __del__
(breaking numarray) to see if that was the problem;  the ratios changed
to 2.5x slower and 0.9x slower (actually faster) respectively.

The large number of "Check" routines preceding the numarray path (I
count 7 looking at my copy of wxPython) has me a little concerned.  I
think those checks are  more expensive for numarray because it is a new
style class.  I have a hard time imagining a 10x difference overall, 
but I think Python does have to traverse the numarray class hierarchy
rather than do a type pointer comparison so they are more expensive.

Is 10x a measured number or a gut feel?

One last thought:  because the sequence protocol is being used rather
than raw array access, compiling matplotlib for numarray (or not) is not
the issue.

Regards,
Todd