[Import-SIG] Loading Resources From a Python Module/Package

[PJ Eby]

On Sat, Jan 31, 2015 at 10:38 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> At the moment, pkg_resources fills in the gap, but that's not
> integrated with the loader system.

Actually, it is.  There's basically a generic function that adapts
loaders to "resource providers".  In a trivial case, a loader can
simply implement the resource provider interface directly, and
register 'lambda self: self' as the adapter function.

I suggest taking a look at the IResourceProvider class, and seeing
whether you want to change anything in how the interface or
implementation work.  You could in fact create ABCs based on the
pkg_resources implementation.

The real question is whether there are any lessons to be learned from
pkg_resources' usage history.  I think the idea of temp files may be a
good one, though there will still be no real cleanup possible in the
case of e.g. C extensions.  You'll have to rely on whatever system
facility exists for temporary file cleanup.  With historical
hindsight, I'd say that I should've made it temp by default, with the
option to set a persistent cache, because a common complaint is that
processes running as special users often can't write to their home
directory (e.g. web servers running as "nobody").

Apart from that, the implementations in pkg_resources can mostly be
pulled for reuse, as well as the interfaces, and I'd suggest doing
exactly that.  There are a lot of non-obvious gotchas dealing with
zipfiles, and the implementation is fairly battle-hardened at this
point.