[SciPy-dev] issues with scipy and eggs

[Robert Kern]

Andrew Straw wrote:
> Robert Kern wrote:
>>No, you need to add the following to the scipy/__init__.py in both scipy_core
>>and full scipy:
>>
>>try:
>>   __import__('pkg_resources').declare_namespace(__name__)
>>except ImportError:
>>   pass
>>
>>That way, pkg_resources knows to keep looking for the other half.
> 
> I'm attempting to do this, but it doesn't work.  If I add these lines to
> core/scipy/__init__.py, scipy core on its own works fine.  However,
> after installing an un-modified full scipy, I get the following error:
> 
> Python 2.4.1 (#2, May  6 2005, 11:22:24)
> [GCC 3.3.6 (Debian 1:3.3.6-2)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> 
>>>>import scipy
> 
> /.sharehome/astraw/py24-amd64/lib/python2.4/site-packages/scipy_core-0.8.2.1625-py2.4-linux-x86_64.egg/scipy/__init__.py:24:
> UserWarning: Module scipy was already imported from
> /.sharehome/astraw/py24-amd64/lib/python2.4/site-packages/scipy_core-0.8.2.1625-py2.4-linux-x86_64.egg/scipy/__init__.pyc,
> but
> /.sharehome/astraw/py24-amd64/lib/python2.4/site-packages/scipy-0.4.3.1482-py2.4-linux-x86_64.egg
> is being added to sys.path
>   tmp_pkg_resources = __import__('pkg_resources')
> Importing test to scipy
> Importing base to scipy
> Importing basic to scipy
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File
> "/.sharehome/astraw/py24-amd64/lib/python2.4/site-packages/scipy_core-0.8.2.1625-py2.4-linux-x86_64.egg/scipy/__init__.py",
> line 70, in ?
>     __doc__ += PackageImport().import_packages()
>   File
> "/.sharehome/astraw/py24-amd64/lib/python2.4/site-packages/scipy_core-0.8.2.1625-py2.4-linux-x86_64.egg/scipy/_import_tools.py",
> line 147, in import_packages
>     return self._format_titles(titles)
>   File
> "/.sharehome/astraw/py24-amd64/lib/python2.4/site-packages/scipy_core-0.8.2.1625-py2.4-linux-x86_64.egg/scipy/_import_tools.py",
> line 90, in _format_titles
>     max_length = max(lengths)
> ValueError: max() arg is an empty sequence
> 
> If I attempt to create an __init__.py in scipy/build/scipy/__init__.py
> before doing "python setup.py bdist_egg" with those lines, I get the
> same error.  (There doesn't seem to be a scipy/__init__.py in full scipy.)

It's been working for me when I make one in scipy/__init__.py; I try not to
directly mess around inside the build directory if I can avoid it.

I think that this is a buglet in _import_tools.py. _format_titles(titles) is
expecting titles to be non-empty. PackageImport is not expecting scipy_core and
the rest of scipy to be in different places.

The workaround for now is to comment out the last line in scipy_core's __init__.py .

The UserWarning is new. I'll have to figure out why that's happening.

>>Also, if both eggs are already on sys.path, then pkg_resources.require() isn't
>>necessary.
> 
> Yes, I'm just going by your previous email which suggested using
> pkg_resources.require().  All my other eggs load fine without this, so I
> figured this might have worked around some scipy funkiness.
> 
>>Additionally, if they weren't the lone pkg_resources.require('SciPy')
>>wouldn't find scipy_core.
> 
> I have no idea what you mean here.

If neither egg had been in your sys.path, then you would have had to call both

  pkg_resources.require('scipy_core')
  pkg_resources.require('scipy')

to make sure both were found. I think. (Note: the capitalization seems to be
fixed, now).

> Having two packages that use the "scipy" namespace seems to be at the
> root of the issue here. (Hence the UserWarning above.) Do we need to do
> something relating to setuptools' namespace packages?

Yes, that's what the fragment I gave does. We can't use the standard setuptools
namespace package mechanism because scipy_core's __init__.py actually contains
code that needs to run.

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter