[SciPy-Dev] Enabling use of scipy.special from Numba

[Juan Luis Cano]

(Copying here a comment I left in the original numba issue)

FWIW, this is an experiment I did to wrap directly the underlying C code
with CFFI and call it from numba:

https://github.com/poliastro/pycephes/tree/v0.1.0

([Explanatory article translated to English](
https://translate.google.com/translate?sl=auto&tl=en&u=https%3A%2F%2Fwww.pybonacci.org%2F2016%2F02%2F07%2Fcomo-crear-extensiones-en-c-para-python-usando-cffi-y-numba%2F
))

@horta then followed a similar approach in https://github.com/limix/hcephes.

On Thu, Jun 20, 2019 at 1:54 PM Paul van Mulbregt <p.vanmulbregt at comcast.net>
wrote:

> Hi Josh,
> For those of us who aren't intimately familiar with the incompatibilities,
> could you describe the issue(s) that you run into when mixing the two?
>
> Paul
>
> > On Jun 20, 2019, at 12:01 AM, Joshua Wilson <josh.craig.wilson at gmail.com>
> wrote:
> >
> > Hey everyone,
> >
> > For a while I've been thinking about how we can improve the
> > compatibility of `scipy.special` with Numba. There is a way to do that
> > currently by using `cython_special`:
> >
> > https://github.com/numba/numba/issues/3086
> >
> > but that has a few drawbacks:
> >
> > (1) it's not very user friendly
> > (2) it relies on the (I think?) private implementation details of
> > Cython's `__pyx_capi__` (including the name mangling scheme used for
> > fused functions)
> > (3) it's clumsy for complex functions
> >
> > I think (3) requires changes on the Numba side, e.g.
> >
> > https://github.com/numba/numba/issues/3860
> >
> > For (1) you could imagine us implementing another Numba specific API,
> > but that runs into some immediate issues:
> >
> > - We'd need Numba as a dependency (which has been discussed:
> > https://mail.python.org/pipermail/scipy-dev/2018-March/022576.html)
> > - We don't want to just keep implementing new APIs
> >
> > For those reasons I'd prefer to find another solution. In an ideal
> > world I'd like to solve (2) in such a way that it's easy for a
> > third-party library to wrap `cython_special` in Numba jitted
> > functions. There is a solution to that already that doesn't involve
> > `__pyx_capi__`: a third-party library writes some Cython glue that
> > exports PyCapsules of function pointers to the specializations of all
> > the `cython_special` functions and then you grab those using ctypes to
> > construct the jitted functions. For the Cython side of that see the
> > docs on specialization:
> >
> >
> http://docs.cython.org/en/latest/src/userguide/fusedtypes.html#selecting-specializations
> >
> > But that feels rather convoluted. In an ideal world it would be nice
> > to have an official way to grab the PyCapsules out of `__pyx_capi__`
> > instead of writing a new set of capsules yourself. Note that we run
> > into a similar problem with our own `LowLevelCallable`:
> >
> >
> https://docs.scipy.org/doc/scipy/reference/generated/scipy.LowLevelCallable.from_cython.html#scipy.LowLevelCallable.from_cython
> >
> > Getting the required name of the exported function for a fused
> > function requires searching through the mangled names for the one you
> > want.
> >
> > Out of all that, my main questions are:
> >
> > - Maybe I'm missing something obvious? Do people have better ideas for
> > improving compatibility?
> > - If not, would it be crazy to talk to the Cython folks about an
> > official method for getting the PyCapsules out of `__pyx_capi__`?
> >
> > - Josh
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>


-- 
Juan Luis Cano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20190620/cb176246/attachment-0001.html>