[pypy-dev] Extended goals (was: needed 18 month plan of what we would do in developing PyPy if we got funding.)

[Christian Tismer]

Bob Ippolito wrote:

> 
> On Thursday, Sep 11, 2003, at 07:32 America/New_York, Christian Tismer 
> wrote:

[extensions, cleese, ...]

>> Are there other ideas of applications which we would
>> like to include as "working horses" to create interest in PyPy?
> 
> 
> What about a focus on scientific applications, number crunching?  There 
> seems to be a pretty good community around SciPy, Numeric, numarray, 
> etc.  Imagine tweaking a few settings in the PyPy JIT such that it 
> aligns all of your ints, doubles, or floats to a particular memory 
> boundary so the code could be vectorized.. and there could be add-ons to 
> the JIT that allow it to take advantage of AltiVec or SSE2?

Ok, this opens several directions.
One is that we probably will want to (at least I do:) write
code generators for many architectures. This should be
great fun in a high level language like Python, with the
flexibility of PyPy.
The platforms you are addressing are special in the sense
that they use the SIMD model (my load balancing idea between
machines would be MIMD of course, if we do number crunching).

Remarks:
AltiVec is a joint development of Motorola, IBM, and Apple.
There are some document worth reading on their web site:
http://www.simdtech.org/apps/group_public/documents.php

SSE2 (Streaming SIMD Extension 2) is Intel's development.
There is some information here:
http://cedar.intel.com/cgi-bin/ids.dll/topic.jsp?catCode=DCG

I like the idea to support special processor extensions.
What I need is to formulate the reason why this should
particularily be done with PyPy.

Here an attempt:
PyPy is extremely flexible, since it supports static
and dynamic code generation by pluggable processor
architecture modules. PyPy can use special hardware
by itself, or it can cross-compile code for target
machines which will run only a downsized PyPy
instantiation which is runtime only.

One advantage of using PyPy instead of other compiled
libraries is its ability to not only support the
special hardware, but also to adjust to the given
machine properties, like cache sizes, number of
parallel FPUs, memory access speed and memory size.
PyPy is able to probe these parameters and to choose
an optimum implementation for the particular hardware
configuration during startup time.

[please give me more advantages here!]

Working package 0:
Provide a way to align certain data arrays in a way
that supports vectorizing.
Write PyPy implementations of numarray.
[Q.: Should be adopt existing number crunching C
      libraries, or better lift them up into PyPy?
      A whole bunch of new targets might come out here ...]

possible = [
   ( 1, "AltiVec"),
   ( 2, "SSE2"),
   # add some more here as you like
]

for i, name in possible:
     print """
Working package %d:
Implement a code generator that supports the SIMD
instructions of %s.
Provide demonstration code, exhaustive testing
and sample application which show how the numerical
packages take full advantage of the supported harware.
     """ % (i, name)

> Personally, I'm really looking forward to a PPC JIT for Python (even if 
> I have to help write it myself).. especially now that I have a dual 2ghz 
> g5 on order :)

I'd be more than happy to chime such stuff into PyPy,
and I know who'd love to work on it :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/