[Numpy-discussion] Proposed fix for MKL and dynamic loading

[Dag Sverre Seljebotn]

Dag Sverre Seljebotn wrote:
> (Apologies if this has been fixed in trunk; I base this on 1.4.0 and no 
> related comments of MKL on the mailing list)
>
> I finally got the latest version of MKL working. What appears to have 
> changed is that the MKL shared libraries will themselves dynamically 
> load different other libraries, depending on the detected CPU.
>
> This is in some ways great news for me, because it means I can avoid 
> worrying about miscompiles when compiling one single version of 
> NumPy/SciPy to use for our heterogenous cluster. So I'd rather *not* 
> link statically [1].
>
> Anyway, after modifying site.cfg [2], things almost work, but not quite. 
> The problem is that Python by default imports shared libs using 
> RTLD_LOCAL. With this patch to NumPy it does:
>
> Change in numpy/linalg/linalg.py:
>
> from numpy.linalg import lapack_lite
>
> to:
>
> try:
>     import sys
>     import ctypes
>     _old_rtld = sys.getdlopenflags()
>     sys.setdlopenflags(_old_rtld|ctypes.RTLD_GLOBAL)
>     from numpy.linalg import lapack_lite
> finally:
>     sys.setdlopenflags(_old_rtld)
>     del sys; del ctypes; del _old_rtld
>
> Questions:
>
> a) Should I submit a patch?
> b) Negative consequences? Perhaps another Python module can now not load 
> a different BLAS implementation? (That still seems better than not being 
> able to use MKL IMO).
> c) Should this only be enabled by a flag somewhere? Where? Or can one 
> just do it regardless of BLAS?
> d) Do I need a "if hasattr" for Windows, or will Windows just ignore it, 
> or does this apply to Windows too?
>
> [1] BTW, I could not figure out how to link statically if I wanted -- is 
> "search_static_first = 1" supposed to work? Perhaps MKL will insist on 
> loading some parts dynamically even then *shrug*.
>   
Forgot this:

[2] Here's my site.cfg:

[mkl]
library_dirs=/mn/corcaroli/d1/dagss/intel/mkl/10.2.3.029/lib/em64t
include_dirs = /mn/corcaroli/d1/dagss/intel/mkl/10.2.3.029/include
lapack_libs = mkl_lapack
mkl_libs = mkl_intel_lp64, mkl_intel_thread, mkl_core, iomp5

Then I need to set LD_LIBRARY_PATH as well prior to running (which I'm 
quite OK with).

Dag Sverre