[Python-Dev] zipimport.c broken with implicit namespace packages

[mike.romberg at comcast.net]

>>>>> " " == Brett Cannon <brett at python.org> writes:

...

     > So it's possible that some abstraction might be possible, but
     > up to this point there hasn't been a motivating factor for
     > importlib to do this as zipimport is written in C and thus
     > won't benefit from any abstraction that importlib uses in its
     > Python code (hence why zipimport needs to be rewritten). Maybe
     > after we have a zip-backed importer written in Python a common
     > API will fall through and we can abstract that out, but I
     > wouldn't want to guess until the code is written and someone
     > can stare at the two implementations.

  Fair enough.  I too think it is a good idea to make base classes
after their is a need for them and not before.   Some argument could
be made that there is a need now as zipimported modules/packages don't
work exactly the same way as "normal" ones.  But since you plan a
rewrite of zipimport collecting the common stuff after that makes sense.

     > It should also be said that there is nothing requiring that an
     > importer support any concept of a file. Going back to the
     > sqlite database example, you could make it nothing more than a
     > table that maps module name to source code and bytecode:

  Yep.  I was not thinking file either.  I may have said file (again
I'm terrible with names) but I'm thinking an array of bytes with a
string id (which I call a path).  The actual storage could simply be
storing the byte array using the stringID/path.

...

     > That's enough to implement an importer and there is no mention
     > of file paths anywhere. If you really wanted to could do a
     > second table that acted as a database-backed file system for
     > reading package data files, but that is not required to meet
     > the minimum functionality -- and then some thanks to bytecode
     > support -- for an importer.

  Yea the python module name could be viewed as a path
(package.subpackage.module) and stored in a hierarchical way.  Or it
could simply be viewed as a string and stored in some flat database.


  Anyway, back at the ranch...  I've got an extended version of the
patch for issue17633 working on my system.  I've added to the test
case to check for the proper functioning of implicit namespace
packages spread between two zip files.  I still need to add tests for
a namespace package spread between a normal filesystem and a zip
file.  I expect I'll have that done in a day or two.  I'll post it to
the bug tracker.

Mike