[Numpy-discussion] Status of Numeric

[Mike C. Fletcher]

Arthur wrote:

>On Wed, 21 Jan 2004 16:22:43 -0700, Tim Hochberg
><tim.hochberg at ieee.org> wrote:
>  
>
...

>>The first is that last I heard the crossover point where 
>>Numarray becomes faster than Numeric is about 2000 elements. It would be 
>>nice if that becomes smaller, but I certainly wouldn't call it extreme. 
>>In fact I'd venture that the majority of cases where numeric operations 
>>are a bottleneck would already be faster under Numarray. In my 
>>experience, while it's not uncommon to use short arrays, it is rare for 
>>them to be a bottleneck.
>>    
>>
>
>Couldn't one argue that - for example - in 3d graphic applications the
>performance as to short arrays is crucial.
>  
>
I would imagine so, but then I'm a 3D guy, and really, the numeric 
community is not AFAIK mostly 3D guys.  Most in the numeric community 
seem to be the scientific guys, who want to deal with really big 
arrays.  Moderately sized scenegraphs are going to have 1000s of nodes 
with 3 or 4 matrices each (each matrix being 3, 4, or occasionally 16 
values) and then a few hundred larger matrices for defining 
vertex/colour/normal/texture content, plus a few hundred matrices 
created/destroyed per frame (1/60th of a second or so)).  But then most 
Python scenegraph engines (OpenGLContext being the only counterexample 
of which I know) are going to simply expose such matrices from their 
C/C++ engine, not use Num* arrays.

Memory would likely be the most critical question for such 
applications.  Not sure how much extra overhead is really being 
discussed.  4000 extra dictionary-based instances could really add up 
memory-wise.  But a 20 or 30 byte overhead/matrix isn't likely going to 
kill anything that performs adequately in Python to begin with.

>But hoping that in fact numarray does come to be accepted as the
>"standard".  
>  
>
AFAIK we still can't compile PyOpenGL with numarray, so I suppose I'm on 
the fence :) .  Should really figure out what makes it fail some day 
soon I suppose.

>>The second point is the relative speediness of Numeric at low array 
>>sizes is the result that nearly all of it is implemented in C, whereas 
>>much of Numarray is implemented in Python.
>>    
>>
>
>Good for me.  I'll be able to understand more of it.
>  
>
Hmm, this does concern me a little.  3D graphics needs to do a lot of a 
very small number of operations (dot and cross product, and basic 
add/multiply/divide mostly).  If each of those calls is going through an 
extra Python layer, there might be pretty significant slowdowns.

>>This results in a larger 
>>overhead for Numarray, which is why it's slower for small arrays. As I 
>>understand it, the decision to base most of Numarray in Python was 
>>driven by maintainability; it wasn't an attempt to optimize large arrays 
>>at the expense of small ones.
>>    
>>
>
>I was indeed making an assumption here.  Ifs its wrong, its indeed
>wrong.
>  
>
Sounds as though the Python layer is being used to dispatch C or C++ 
coded functions on internal structures of the array, which would be 
consistent with the optimisation pattern (larger the dispatched 
operation, the less percentage overhead involved).

>For reasons that are non-technical, I believe it much to the best that
>the community continue to look to one package of the sort as a
>standard.  To me. even better if it makes it into the core
>distribution.
>  
>
Core or not doesn't really concern me.  But definitely one standard.  
Even now, we're starting to get users stymied by having Numarray and 
having it pretend to be Numpy so that PyOpenGL craps out with nasty 
errors when it tries to do anything with Numpy.

Enjoy,
Mike

_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/