[Import-SIG] Proposed design for importlib.resources()

[Barry Warsaw]

On Nov 24, 2015, at 08:21 PM, Brett Cannon wrote:

>> Module API vs package API.  Doesn't pkg_resources actually support
>> something similar to both, with the module function providing a convenience
>> API?
>
>Yes, but that doesn't sway me. This isn't a "pkg_resources++" but a "make
>reading data from a package make sense in a modern import world". IOW I'm
>purposefully not using pkg_resources as a template but simply as a
>motivating factor.

You're not taking into account a migration path for existing users of
pkg_resources.  If you make it difficult to convert, then people are much less
likely to do it for existing code, despite the ability to remove a dependency.

If my existing code already has

    from pkg_resources import resource_string as resource_bytes

all I'd need to do is change this one line to

    from importlib.resource import read_bytes as resource_bytes

and I'm done.  If I need to support multiple versions of Python, I can even
do:

    try:
        from importlib.resource import read_bytes as resource_bytes
    except ImportError:
        from pkg_resources import resource_string as resource_bytes

Without this API, it's much more difficult for me to convert my existing code,
either incrementally or whole-hog, because now I have to either add that
convenience function myself (and import it everywhere) or rewrite all my call
sites.  Why bother?

>Unfortunately for you the poll liked the other approach and TOOWTDI. So
>either convince me that resources.read_bytes(pkg, path) is better than
>resources(pkg).read_bytes(path) or consider the bike shed painted. :)

It's not better or worse, it's just different.  As pkg_resources has shown, it
doesn't have to be either-or.

I never saw the poll since I don't pay attention to Google+.  How
representative are those 59 votes of the current pkg_resource users and
potential future users of this API?  If I had seen the poll I would have
complained that it didn't give me a chance to choose both APIs <wink>.

>I'm not convinced it's necessary to provide an equivalent open() yet;

Right, I'm not necessarily advocating for it, just describing what it would
have to do if it were there.  It's something I occasionally wish I had, but
all the building blocks are there to invent it when needed.

>> I do have one use of resource_listdir() which is used to find importable
>> plugin modules at runtime.  It's handy.
>
>I'm going to punt on this for as long as possible because it's asking for
>trouble to get right. For example, if I do resources(pkg).listdir(), then I
>will end up returning relative paths, but if you disassociate those paths
>from pkg then you have lost proper context. You could return tuples of
>(pkg, relative_path), but that just doesn't seem satisfactory either. I'm
>just not convinced yet it is needed enough to support (at least initially).

It's a tougher API to recreate from the building blocks, so it would be nice
not to have to reinvent the wheel everywhere, but it's also a much less common
API.  I'm not at all worried about the disassociation problem, since
os.listdir() gives you relative paths anyway so it's a familiar behavior.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/import-sig/attachments/20151124/923f954a/attachment-0001.sig>