[SciPy-Dev] ODE solvers

[Pauli Virtanen]

04.06.2015, 18:25, Sturla Molden kirjoitti:
> We can get a function pointer as a void* from ctypes too. An ODE solver 
> could accept this too. I would also suggest that we allow an f2py object 
> as callback as it has a _cpointer attribute (PyCObject) which stores the 
> function pointer in a void*.
> 
> So now we have at least four different things that can be allowed as 
> callback:
> 
> 1. Any Python callable object
> 2. Subclass of a certain Cython cdef class
> 3. ctypes function object
> 4. f2py Fortran object
> 
> The latter three we can optimize for maximum efficiency.

For the internal implementation, just using function pointers will be
likely the easiest, because this is what the codes doing the heavy
lifting are in the end expecting.

Such pointers can of course be extracted from the above objects; in
addition Cython exported cdef/cpdef functions might also be made
accessible (via the API Cython uses to implement cimport; however, not
sure how public this API is).

> With ctypes and f2py we need to handle the GIL. It should not be
> released if the ctypes callback is from a PyDLL, but it can be
> released if from a C DLL. And if it is an f2py object we must check
> to see if the callable is threadsafe. (Is this attribute exposed
> somehow?) With Cython it is easier because we can just use a
> different class with nogil callback.

I don't think we can in general know if a callable passed in by the user
needs GIL or not, or whether it is threadsafe. The default assumption
likely should be that it does not need GIL and is threadsafe. If you are
trying to write low-level threaded code, you probably should be aware of
the issues that can arise, and do the necessary locking or "with gil"
yourself.

Calling to f2py and to ctypes function pointer in itself does not need
GIL, as both are just raw function pointers (with ABI that we have to
specify).

The nogil constraint probably can be checked for Cython.

The biggest challenge in doing this is pulling off the actual
implementation in such a way that it can be reused in all of the solvers
without too much work, and hopefully also used elsewhere in Scipy. I
think this is doable, however.