[Numpy-discussion] querying backend information

[Nathaniel Smith]

On Wed, Nov 4, 2015 at 4:40 PM, Stefan Seefeld <stefan at seefeld.name> wrote:
> Hello,
>
> is there a way to query Numpy for information about backends (BLAS,
> LAPACK, etc.) that it was compiled against, including compiler / linker
> flags that were used ?
> Consider the use-case where instead of calling a function such as
> numpy.dot() I may want to call the appropriate backend directly using
> the C API as an optimization technique. Is there a straight-forward way
> to do that ?
>
> In a somewhat related line of thought: Is there a way to see what
> backends are available during Numpy compile-time ? I'm looking for a
> list of flags to pick ATLAS/OpenBLAS/LAPACK/MKL or any other backend
> that might be available, combined with variables (compiler and linker
> flags, notably) I might have to set. Is that information available at all ?

NumPy does reveal some information about its configuration and
numpy.distutils does provide helper methods, but I'm not super
familiar with it so I'll let others answer that part.

Regarding the idea of "cutting out the middleman" and calling directly
into the appropriate backend via the C API, NumPy doesn't currently
expose any interface for doing this. There are some discussions with
Antoine from a few months back about this (and given that you work at
the same place I'm guessing the motivation is the same? :-)). For some
reason I'm failing to find the archives now, but the summary from off
the top of my head is: SciPy does expose an interface for this (via
cython and its PyCapsule tricks -- see [1]), NumPy is unlikely to
because we're wary of adding extra public interfaces and can't
guarantee that we even have a full BLAS/LAPACK available (sometimes we
fall back on a minimal vendored subset that's just enough for our
needs), you probably don't want to try and get into the business of
dynamically hunting down BLAS/LAPACK because it will be brittle and
expose you to all kinds of cross-platform linker issues, and if you
want to pull the clever stuff that scipy is doing out of scipy and put
it into its own dedicated blas/lapack package, then well, we need one
of those anyway [2].

-n

[1] https://github.com/scipy-conference/scipy_proceedings_2015/blob/master/papers/ian_henriksen/cython_blas_lapack_api.rst
[2] e.g. https://mail.scipy.org/pipermail/numpy-discussion/2015-January/072123.html

-- 
Nathaniel J. Smith -- http://vorpus.org