[Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Ralf Gommers]

On Tue, Jun 14, 2011 at 5:28 AM, Bruce Southey <bsouthey at gmail.com> wrote:

> On Mon, Jun 13, 2011 at 8:31 PM, Pauli Virtanen <pav at iki.fi> wrote:
> > On Mon, 13 Jun 2011 11:08:18 -0500, Bruce Southey wrote:
> > [clip]
> >> OSError:
> >> /usr/local/lib/python3.2/site-packages/numpy/core/multiarray.pyd: cannot
> >> open shared object file: No such file or directory
> >
> > I think that's a result of Python 3.2 changing the extension module
> > file naming scheme (IIRC to a versioned one).
> >
> > The '.pyd' ending and its Unix counterpart are IIRC hardcoded to the
> > failing test, or some support code in Numpy. Clearly, they should instead
> > ask Python how it names the extensions modules. This information may be
> > available somewhere in the `sys` module.
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
>
> Pauli,
> I am impressed yet again!
> It really saved a lot of time to understand the reason.
>
> A quick comparison between Python versions:
> Python2.7: multiarray.so
> Python3.2:  multiarray.cpython-32m.so
>
> Search for the extension leads to PEP 3149
> http://www.python.org/dev/peps/pep-3149/
>

That looked familiar:
http://projects.scipy.org/numpy/ticket/1749
https://github.com/numpy/numpy/commit/ee0831a8

>
> This is POSIX only which excludes windows. I tracked it down to the
> list 'libname_ext' defined in file "numpy/ctypeslib.py"  line 100:
> libname_ext = ['%s.so' % libname, '%s.pyd' % libname]
>
> On my 64-bit Linux system, I get the following results:
> $ python2.4 -c "from distutils import sysconfig; print
> sysconfig.get_config_var('SO')"
> .so
> $ python2.5 -c "from distutils import sysconfig; print
> sysconfig.get_config_var('SO')"
> .so
> $ python2.7 -c "from distutils import sysconfig; print
> sysconfig.get_config_var('SO')"
> .so
> $ python3.1 -c "from distutils import sysconfig;
> print(sysconfig.get_config_var('SO'))"
> .so
> $ python3.2 -c "from distutils import sysconfig;
> print(sysconfig.get_config_var('SO'))"
> .cpython-32m.so
>
> My Windows 32-bit Python2.6  install:
> >>> from distutils import sysconfig
> >>> sysconfig.get_config_var('SO')
> '.pyd'
>
> Making the bug assumption that other OSes including 32-bit and 64-bit
> versions also provide the correct suffix, this suggest adding the
> import and modification as:
>
> import from distutils import sysconfig
> libname_ext = ['%s%s' % (libname, sysconfig.get_config_var('SO'))]
>
> Actually if that works for Mac, Sun etc. then 'load_library' function
> could be smaller.
>

The issue is that libraries built as Python extensions have the extra
".cpython-mu32", but other libraries on your system don't. And
sysconfig.get_config_var('SO') is used for both. I don't see another config
var that allows to distinguish between the two. Unless on platforms where it
matters there are separate return values in sysconfig.get_config_vars('SO').
What do you get for that on Linux?

The logic of figuring out the right extension string should not be
duplicated, distutils.misc_util seems like a good place for it. Can you test
if this works for you: https://github.com/rgommers/numpy/tree/sharedlib-ext


> Finally the test_basic2 in "tests/test_ctypeslib.py" needs to be
> changed as well because it is no longer correct. Just commenting out
> the 'fix' appears to work.
>
> It works in the case of trying to load multiarray, but the function under
test would still be broken.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110619/397ba8bf/attachment.html>