Pyrex and weave.accelerate

[Patrick Miller]

The Pyrex stuff is really cool.. The focus on my work
on weave.accelerate is a little different (though
same end goal, easier bridge from C/C++).  The nice
bit about weave.accelerate is that it allows a fallback
position to just run the native Python in cases
where the compiler can't run (system resource limits,
Jython, Pippy, etc...)

It appears that the premise of Pyrex (assumption warning!
I'm not reading Greg's mind!) is that C types have
Python equivalents expressible through the API
(e.g. short/int/long are expressible through
the PyInt_AsLong and PyInt_FromLong methods).

The weave accelerate works somewhat the other way,
it presumes the existence of a C/C++ API to Python object types.
Weave.accelerate allows programmers to teach weave
the API for a type and then have it build the
extension module from that knowledge.


Where Eric alludes to scientific needs, he is
referring to array/field acceleration.  Now
the initial target of weave is Numpy arrays...


def div(u):
  d = zeros(len(u),'d')
  for i in range(len(u)-1):
     d[i] = u[i+1]-u[i]
  return d

So, weave.accelerate tries to replace function calls with
equivalent C/C++ code.  Now the nice thing is for the
simulation code I work with that has its own wrapping
scheme for a field class derived from C++ STL vectors<double>.
I can describe the API for this special field type and
then stub in the correct code for Numpy arrays, vector<double>,
specialized field classes, or even normal Python lists.

Weave accelerate uses type inference based on types in real
use, so

def foo(n):
  v = n + 1
  return v

If you call foo(3), weave.accelerate examines the value 3
and determines that it matches the C++ type, long.  It
derives the type of "v" by examining the return type
of a long + long (hence it is long).

If you later call foo(3.3), weave will compiler another version
of the function (this time, v will have C++ type double).

Pyrex looks to extend the language, I was looking at
doing something similar, but with type assertions for
cases where the types were well known.

So as opposed to Pyrex language extensions

def foo(int n):
   cdef int v
   v = n + 1
   return vf


I can encode the types as

def foo(n):
   assert type(n) == IntType, weave.TypeError
   v = n + 1
   return v

Now, I get full fallback to Python (in case weave fails to compile)
with proper typechecking.....

Eric and I will have more details for the O'Reilly conference
paper and talk.... Stay tuned.

Pat

-- 
Patrick Miller | (925) 423-0309 | patmiller at llnl.gov

Change is not made without inconvenience, even from worse to
better -- 16th Century British theologian Richard Hooker

eric wrote:
> 
> 017935351.5088.python-list at python.org> <slrnaasqc7.178.mlh at vier.idi.ntnu.no>
> Subject: Re: A faster Python?, Python compiler, Dylan,...
> Date: Fri, 5 Apr 2002 21:39:55 -0500
> Organization: enthought
> MIME-Version: 1.0
> Content-Type: text/plain;
>         charset="Windows-1252"
> Content-Transfer-Encoding: 7bit
> X-Priority: 3
> X-MSMail-Priority: Normal
> X-Mailer: Microsoft Outlook Express 5.50.4522.1200
> X-Mimeole: Produced By Microsoft MimeOLE V5.50.4522.1200
> 
> Hey Magnus,
> 
> That is exactly what Pat Miller of LLNL is working on.  The function
> weave.accelerate() (in CVS and quite limited right now) is aimed at this
> purpose.  It is a little different than Python2C to satisfy some needs in the
> scientific community.  Also, Greg Ewing's new Pyrex project is a good candidate
> for this sort of thing -- hopefully we can learn from each other because Pyrex
> looks very promising.
> 
> So, the short answer is "good idea -- we're working on it." :-)