[Distutils] Adding egg support (Resource API) to existing packages

[Phillip J. Eby]

At 11:56 AM 7/14/2005 -0400, Kevin Dangoor wrote:
>On 7/14/05, Paul Moore <p.f.moore at gmail.com> wrote:
> > You don't care about eggs (yet). Your users (one of whom is me) do,
> > and would like to package your code in an egg via python setup.py
> > --compiler-package=setuptools.command bdist_egg. This works fine, with
> > no source changes, but the resulting egg isn't zip-safe because of
> > resource usage issues.
>
>While I agree with Phillip on this one, I do know what you're saying.
>I've wondered that about some of the packages I'm putting into eggs.
>
>I'm pretty certain this situation will arise where a project
>maintainer won't go for a setuptools dependency. Here's one way to
>deal with it:
>
>Let's say the project's code is in the package:
>
>pythonfoobar
>
>What if you toss pkg_resources into pythonfoobar. Then, in all of the
>modules you touch, you just do:
>
>from pythonfoobar import pkg_resources

*shudder*.  Please don't.  pkg_resources needs to be a singleton, partly 
because sys.path is a singleton, and partly because there needs to be a 
single place to register adapters for PEP 302 loaders and importers, and in 
the future, egg listeners.

If you guys *really* want to do this, what we need to do is create a 
super-simple "mini-Me" form of pkg_resources, that provides exactly two 
functions: resource_string and resource_stream.

The code for these will be very short and relatively simple to do, and will 
not impact any other part of the egg runtime system; it will just allow 
portable resource access.

We can call it 'pkg_resources_lite', or something similar, and I can change 
the safety analysis to ignore the fact it uses __file__ (i.e. if the 
module's named pkg_resources_lite).


>2) in pythonfoobar/__init__ just import pkg_resources  (this is
>assuming you're not using the fancy setuptools namespaces... not many
>projects need that, though.)

Namespaces...?  Do you mean namespace packages?


>Ideally, people will go along with setuptools. But, at least this
>workaround won't break people's systems.

Um, yeah.  I think, after all this discussion, that the simple way to fix 
this is going to be for somebody to produce and maintain a 
pkg_resources_lite that people can bundle inside their distributions like 
this.  It'll even be smaller than ez_setup, although it won't come in the 
convenient svn:externals format of ez_setup.

Hm.  You know, if pkg_resources_lite is written so that it tries to import 
from pkg_resources, then I can make it "step aside and let the professional 
handle it" if the user has the full pkg_resources package.  This would let 
it take advantage of any support for registered import hooks, etc. that the 
full package was doing.  Something like this:

     import sys, os

     def _get_path_and_loader(package_name, resource_name):
         try:
             module = sys.modules[moduleName]
         except KeyError:
             __import__(moduleName)
             module = sys.modules[moduleName]
         loader = getattr(module, '__loader__', None)
         parts = resource_name.split('/')
         path = os.path.join(os.path.dirname(module.__file__),*parts)
         return path, loader

     def resource_string(package_name, resource_name):
         path,loader = _get_path_and_loader(package_name, resource_name)
         if loader is None:
             f = open(path,'rb')
             try:
                 return f.read()
             finally:
                 f.close()
         else:
             return loader.get_data(path)

     def resource_stream(package_name, resource_name):
         path,loader = _get_path_and_loader(package_name, resource_name)
         if loader is None:
             return open(path,'rb')
         else:
             from cStringIO import StringIO
             return StringIO(loader.get_data(path))

     try:
         from pkg_resources import resource_string, resource_stream
     except ImportError:
         pass

Only with some docstrings and comments added.  :)  Anyway, this thing would 
work with eggs or py2exe or just plain zipfiles.  It won't necessarily work 
with other PEP 302 loader objects, but neither does pkg_resources unless 
you register suitable adapters for the functionality needed.