[Numpy-discussion] ANN: NumPy 1.8.1 release

[David Cournapeau]

On Wed, Apr 2, 2014 at 12:36 AM, Nathaniel Smith <njs at pobox.com> wrote:

> On Tue, Apr 1, 2014 at 11:58 PM, David Cournapeau <cournape at gmail.com>
> wrote:
> > On Tue, Apr 1, 2014 at 6:43 PM, Nathaniel Smith <njs at pobox.com> wrote:
> >>
> >> On Tue, Apr 1, 2014 at 6:26 PM, Matthew Brett <matthew.brett at gmail.com>
> >> wrote:
> >> > I'm guessing that the LOAD_WITH_ALTERED_SEARCH_PATH means that a DLL
> >> > loaded via:
> >> >
> >> > hDLL = LoadLibraryEx(pathname, NULL,  LOAD_WITH_ALTERED_SEARCH_PATH);
> >> >
> >> > will in turn (by default) search for its dependent DLLs in their own
> >> > directory.    Or maybe in the directory of the first DLL to be loaded
> >> > with LOAD_WITH_ALTERED_SEARCH_PATH, damned if I can follow the
> >> > documentation.  Looking forward to doing my tax return after this.
> >> >
> >> > But - anyway - that means that any extensions in the DLLs directory
> >> > will get their dependencies from the DLLs directory, but that is only
> >> > true for extensions in that directory.
> >>
> >> So in conclusion, if we just drop our compiled dependencies next to
> >> the compiled module files then we're good, even on older Windows
> >> versions? That sounds much simpler than previous discussions, but good
> >> news if it's true...
> >
> >
> > That does not work very well in my experience:
> >
> >   - numpy has extension modules in multiple directories, so we would
> need to
> > copy the dlls in multiple subdirectories
> >   - copying dlls means that windows will load that dll multiple times,
> with
> > all the ensuing problems (I don't know for MKL/OpenBlas, but we've seen
> > serious issues when doing something similar for hdf5 dll and
> pytables/h5py).
>
> We could just ship all numpy's extension modules in the same directory
> if we wanted. It would be pretty easy to stick some code at the top of
> numpy/__init__.py to load them from numpy/all_dlls/ and then slot them
> into the appropriate places in the package namespace.
>
> Of course scipy and numpy will still both have to ship BLAS etc., and
> so I guess it will get loaded at least twice in *any* binary install
> system. I'm not sure why this would be a problem (Windows, unlike
> Unix, carefully separates DLL namespaces, right?)


It does not really matter here. For pure blas/lapack, that may be ok
because the functions are "stateless", but I would not count on it either.

The cleanest solution I can think of is to have 'privately shared DLL', but
that would AFAIK require patching python, so not really an option.


, but if it is a
> problem then it's a very fundamental one for any binaries we ship.
>
> Do the binaries we ship now have this problem? Or are we currently
> managing to statically link everything?
>

We currently statically link everything. The main challenge is that 'new'
(>= 4) versions of mingw don't easily allow statically linking all the
mingw-related dependencies. While the options are there, everytime I tried
to do it with an official build of mingw, I had some weird, very hard to
track crashes. The other alternative that has been suggested is to build
one own's toolchain where everything is static by default. I am not sure
why that works, and that brings the risk of depending on a toolchain that
we can't really maintain.

David

>
> -n
>
> --
> Nathaniel J. Smith
> Postdoctoral researcher - Informatics - University of Edinburgh
> http://vorpus.org
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140402/4fcec43f/attachment.html>