[Python-Dev] zipimport.c broken with implicit namespace packages

[Nick Coghlan]

On 3 January 2016 at 15:29,  <mike.romberg at comcast.net> wrote:
>>>>>> " " == Brett Cannon <brett at python.org> writes:
>
>      > I just wanted to quickly say that Guido's observation as to how
>      > a VFS is overkill is right. Imagine implementing a loader using
>      > sqlite and you quickly realize that doing a dull VFS is more
>      > than necessary to implement what import needs to function.
>
>   I fear I've made a poor choice in calling this abstract class a VFS
> (I'm terrible with names).  I'm not thinking of anything along the
> lines of a full file system that supports open(), seek(), read() and
> everything else.   That for sure would be overkill and way more
> complicated than it needs to be.
>
>   All I'm really thinking about is a simple abstract interface that is
> used by an importer to actually locate and retrieve the binary objects
> that will be loaded.  For the simple case I think just two methods
> would/could server a read only "blob/byte database":
>
>   listdir(path)  # returns an iterable container of "files"/"dirs" found
>                  # at path
>
>   get(path)      # returns a bytes object for the given path

We already have the latter:
https://docs.python.org/3/library/importlib.html#importlib.abc.ResourceLoader.get_data

It's the former that has taken a while to get to, as the 3rd party
pkg_resources module (part of setuptools) already provides a pragmatic
API that also has the virtue of being compatible with both Python 2 &
3, and there a few subtleties related to the possible use of temporary
files that make a robust API design trickier than it first appears to
be.

For folks that are interested in that, folks that aren't following
import-sig in addition to python-dev may want to take a look at
Brett's design for the importlib.resources API:
http://nbviewer.jupyter.org/gist/brettcannon/9c4681a77a7fa09c5347

Cheers,
Nick.

P.S. If anyone actually *does* want a full "virtual file system layer"
API for non-filesystem storage locations:
http://docs.pyfilesystem.org/en/latest/filesystems.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia