[SciPy-dev] Cython, f2py and GSoC

[Dag Sverre Seljebotn]

This is in preparation for a GSoC project application; Kurt Smith has
approached me about doing a project on Fortran integration in Cython
with myself as mentor.

Although we started out with a Fortran/Cython perspective, we think
that this potentially affects the SciPy community and f2py as well.

The main issues:
  1) f2py doesn't work that well for Cython, as it requires Python
packing/unpacking of arguments. A more direct call approach is needed.
  2) f2py and Cython has a certain overlap in their implementation (both
generate Python extension modules), and need to tackle many of the same
issues both now and especially in the future

Could we solve this so that in getting Fortran/Cython integration, we
also set up a development path for further development of f2py with
Cython as a backend?

Below is a scetch of our current plan to give you an idea. More full 
specifications etc. will come later and we can have any discussions then.

  1) Add a Cython syntax and API for passing acquired PEP-3118 
buffers/NumPy arrays to external C functions (i.e. as a struct with the 
necesarry information (pointer, shape, strides)). This simply means 
defining a syntax for passing information that Cython already has to an 
external C function.

  2) Create a new tool which uses the parser part of f2py (with any 
necesarry improvements) but adds a different backend which generates a C 
interface to the given Fortran module, along with a Cython pxd file for 
it. (Adding a C .h file target, to get "f2c" functionality, would be 
trivial.)

This will be done using the Fortran 2003 C bindings. So a .f90 file is 
generated which compiles to a C interface to the library. Array 
parameters will be passed as the PEP-3118-like structs we define in 1), 
and so the functions will be callable directly with e.g. NumPy arrays 
from Cython. Copy-in/out might be necesarry for Fortran to be able to 
work with the arrays, if so this will happen in the Fortran wrapper 
generated by this new tool.

  3) One could then add a feature to Cython to automatically export 
external functions as Python functions so that one doesn't have to write 
wrapper stubs. This should bring the functionality to a level comparable 
to current f2py.

Now, how does the SciPy community see this project?

  1) Is there a potential for a joint Cython/SciPy project on 
Fortran/Python integration here? I could do the main mentoring work, but 
support of the idea etc. is important too.

  2) Any co-mentors perhaps on the f2py parser side? Improvements might 
be needed there.

  3) Would you prefer us to a) rip/fork the parser out of f2py and stay 
within the Cython project, or b) work on f2py upstream to add another 
backend? Or something else?

-- 
Dag Sverre