[SciPy-dev] Thoughts on weave improvements

[Pearu Peterson]

Hi,

Eric and Pat, thanks for explaining the philosphy of weave and pycod, it
cleared many things out for me. And I feel better now after knowing that
there are ways to get weave/pycod efficient and really useful.

On Mon, 11 Feb 2002, eric wrote:

> I guess we'll have to poll you at the end of the summer to see if we've ( or
> weave... :) changed your mind.  One of its major benefits (C callbacks) will
> require some cooperation with f2py so that Fortran wrappers check whether
> the passed in object has a C representation to call instead of automatically
> calling back into Python.  We'll try and make this as easy as possible, but,
> of course, you'll have to sign on as a weave believer for the integration to
> work well.

In fact, I was thinking the same thing for f2py, that is, if f2py'd
extension function gets a call-back argument that is itself a f2py'd
function (an therefore has a pointer to a real C/Fortran function), then
it is called directly and not by the f2py call-back mechanism.

The implementation of this feature is quite easy in f2py. What kept me
implementing all that are the results of the timing measurments of the
current f2py implementation. Namely, it turns out that the f2py call-back
mechanism is already quite efficent (the time spent in doing this magic
was only about 0.001 milli-seconds per call, error factor is 10) and most
of the time (say 99%) is spent in Python (the test Python function
contained only few scalar multiplications, usage of math.sin, and no
array operations and loops).

So, it is really crucial to get Python functions efficient and (may 
be) later optimize the interfaces (with a possibility of introducing new
bugs;). Seems like weave/pycod will provide a solution here.

You may want to look at fortranobject.c in f2py. Basically,
1) it implements a new type, FortranObject, and the objects of this type
hold C pointers to real C/Fortran functions. These functions are called
from f2py generated extension functions by using the corresponding 
pointers.
2) Also, the FortranObject type implements few methods:
__call__ - so that in Python objects of type FortranObject behave like
normal (well, almost) Python functions,
__get/setattr__ - to return __doc__ strings and also to access Fortran
common blocks or module data, the items of are also returned as
FortranObject's.
The idea is simple but the implementation may look complex, that is
because of Fortran specific stuff (just ignore it).

I guess that the bottom line is that f2py and weave/pycod have much in
common regarding the calls to real C/Fortran functions in run-time and I
would hope that these common parts could be shared in some or another way.

Do we need to introduce some independent (from f2py, weave/pycod,
etc.) specification for objects holding C/Fortran objects and that
implement some common tasks that are useful for both f2py and weave?
Or, may be weave/pycod implement their own, say, CPointerObject, and I
can easily implement hooks for such objects for f2py (as all that f2py
needs to use are the C pointers to C/Fortran functions or data).


Regards,
	Pearu