[Python-Dev] Better support for consuming vendored packages

[Nick Coghlan]

On 23 March 2018 at 02:58, Gregory Szorc <gregory.szorc at gmail.com> wrote:

> I'd like to start a discussion around practices for vendoring package
> dependencies. I'm not sure python-dev is the appropriate venue for this
> discussion. If not, please point me to one and I'll gladly take it there.
>
>
Since you mainly seem interested in the import side of things (rather than
the initial vendoring process), python-ideas is probably the most suitable
location (we're not at the stage of a concrete design proposal that would
be appropriate for python-dev, and this doesn't get far enough into import
system arcana to really need to be an import-sig discussion rather than a
python-ideas one).


> What we've done is effectively rename the "shrubbery" package to
> "knights.vendored.shrubbery." If a module inside that package attempts an
> `import shrubbery.x`, this could fail because "shrubbery" is no longer the
> package name. Or worse, it could pick up a separate copy of "shrubbery"
> somewhere else in `sys.path` and you could have a Frankenstein package
> pulling its code from multiple installs. So for this to work, all
> package-local imports must be using relative imports. e.g. `from . import
> x`.
>

If it's the main application doing the vendoring, then the following kind
of snippet can be helpful:

    from knights.vendored import shrubbery
    import sys
    sys.path["shrubbery"] = shrubbery

So doing that kind of aliasing on a process-wide basis is already possible,
as long as you have a point where you can inject the alias (and by
combining it with a lazy importer, you can defer the actual import until
someone actually uses the module).

Limiting aliasing to a particular set of modules *doing* imports would be
much harder though, since we don't pass that information along (although
context variables would potentially give us a way to make it available
without having to redefine all the protocol APIs)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180324/c8e08ce5/attachment.html>