[Distutils] Confusion about the effect of eggs on import
Phillip J. Eby
pje at telecommunity.com
Sat Feb 10 01:20:31 CET 2007
At 05:30 PM 2/9/2007 -0600, skip at pobox.com wrote:
>(I'm sending this to distutils because I have this vague notion that the
>problem might have something to do with setuptools or eggs on sys.path.)
Their mere presence on sys.path doesn't do anything, if you're not
importing pkg_resources or importing anything from inside the eggs.
However, matplotlib imports pkg_resources and is a (broken) namespace package.
>We have a weird issue trying to import matplotlib at work.
I investigated a little, and here's what's happening. Matplotlib declares
'matplotlib.toolkits' as a namespace package, but you can't make a child
package a namespace package without the parent being one too. A namespace
package combines all packages on sys.path with the same name into a single
package.
So here's what's happening. When setuptools is on sys.path, then importing
matplotlib causes it to declare its namespace, which merges all copies of
matplotlib on sys.path into a single super-package -- and invokes all the
__init__.py's.
When setuptools is not on sys.path, the attempt to declare the namespace
fails (it's wrapped in a try/except in matplotlib/__init__.py), so nothing
else happens.
There are, I think, two problems here. One, is that setuptools shouldn't
be executing multiple __init__.py's for a package, but for backward
compatibility reasons this isn't being dropped until 0.7a1. Two, is that
setuptools currently allows you to declare a child namespace package (like
'matplotlib.toolkits') without explicitly declaring the parent to be a
namespace package. So, sometimes people declare a subpackage without
realizing the parent will also have to be treated as a namespace package.
As a result, they end up thinking that they can include initialization code
in the parent package, when in fact there are many circumstances where it
simply won't work. In this case, matplotlib/__init__.py contains
executable code, which is a no-no for a namespace package.
My suggestion would be that matplotlib use a matplotlib_toolkits namespace
package, rather than attempting to keep matplotlib.toolkits, since it
appears they are relying on a substantial amount of code living in the
__init__.py.
In short, the problem is a matplotlib bug, but in fairness it's probably
due to the sketchy documentation surrounding the proper care and feeding of
namespace packages, coupled with the implicit declaration of namespace
packages. All of the problems are explained in the setuptools
documentation, but unfortunately that's not the same as anybody being able
to figure out that the problems will apply to THEM. ;)
More information about the Distutils-SIG
mailing list