[Numpy-discussion] Some observations or questions about psyco & pyrex

[Perry Greenfield]

Both psyco and pyrex have some great aspects. But I think
it is worth a little reflection on what can and can't be
expected of them. I'm basically ignorant of both; I know
a little about them, but haven't used them. if anything I 
say is wrong, please correct me. I'm going to make some
comments based on inferred characteristics of them that 
could well be wrong.

Psyco is very cool and seems the answer to many dreams. 
But consider the cost. From what I can infer, it obtains
its performance enhancements at least in part by constructing
machine code on the fly from the Python code. In other
words it is performing aspects of running on particular
processors that is usually relegated to C compilers by 
Python.

I'd guess that the price is the far greater difficulty of
maintaining such capability across many processor types.
It also likely increases the complexity of the implementation
of Python, perhaps making it much harder to change and
enhance. Even without it handling things that are needed
for array processing, how likely is it that it will be 
accepted as the standard implementation for Python for 
these reasons alone. 

I also am inclined to believe that adding the support for
array processing to a psyco implementation is a significant
undertaking. There are at least two issues that would have 
to be addressed: handling all the numeric types and exception
handling behavior. Then there are aspects important to us
that include handling byteswapped or non-aligned data.
Having the Python VM handle the efficiency aspects of arrays
simplifies aspects of their implementation as compared
to the current implementations of Numeric and numarray it
doesn't eliminate the need to replicate much of it.
Having to deal with the implemenation for several different
processors is likely to outweigh any savings in the implementation.
But maybe I misjudge.

Pyrex's goals are more realistic I believe. But unless I'm 
mistaken, pyrex cannot be a solution to the problems
that Numeric and numarray solve. Writing a something for
pyrex means committing to certain types. It's great for
writing something that you would have written as a C extension,
but it doesn't really solve the problem of implementing
Ufuncs that can handle many different types of arrays,
and particularly combinations of types. But perhaps I
misunderstand here as well. It certainly would be nice if
it could handle some of the aspects of the Numeric/numarray
API automatically.

So I doubt that either really is a solution for masked arrays
in general.

Perry