[Import-SIG] Import engine PEP up on python.org

[Eric Snow]

On Nov 13, 2011 3:21 AM, "Nick Coghlan" <ncoghlan at gmail.com> wrote:
>
> On Sun, Nov 13, 2011 at 6:28 PM, "Martin v. Löwis" <martin at v.loewis.de>
wrote:
> > Am 13.11.2011 06:12, schrieb Nick Coghlan:
> >> I finally got around to updating the import engine draft PEP and
> >> publishing it on python.org: http://www.python.org/dev/peps/pep-0406/
> >
> > I think the rationale section needs to be improved. In fact, I still
> > don't understand what the objective of this API is (I do understand what
> > it achieves, but it's unclear why having that is desirable, and for what
> > applications).
>
> It's desirable for the same reason *any* form of object-oriented
> encapsulation is desirable: because it makes it easier to *think*
> about the problem and manage interdependencies between state updates.
> I didn't realise the merits of OO designs needed to be justified - I
> figured the list of 6 pieces of interdependent process global state
> spoke for itself.
>
> > I notice that there is overlap both with multiple subinterpreters,
> > and with restricted execution. It hints at providing both, but actually
> > provides neither.
>
> It doesn't claim to provide either - it's sole aim is to provide a
> relatively lightweight mechanism to selectively adjust elements of the
> import system (e.g. adding to sys.path when importing plugins, but
> leaving it alone otherwise).
>
> But having the import state better encapsulated would make it easier
> to improve the isolation of subinterpreters so that they aren't
> sharing Python modules, even if they still share extension modules
> (you could put a single pointer to the import engine on the
> interpreter state rather than storing it in sys the way we do now).
>
> > I think the long-term solution really should be proper support for
> > subinterpreters, where there is no global state in C at all. Extension
> > modules already can achieve this through the PEP 3121 API (even though
> > few modules actually do).
> >
> > If the objective is to have more of the import machinery implemented in
> > Python, then making importlib the import machinery might be best.
>
> Guido already approved (in principle) that change - this PEP would
> actually depend on that being done first (because I think trying to
> build this API directly on top of import.c would be a complete waste
> of time).
>
> > If the objective is to allow hooks into the import procedure, it would
> > be best to just provide the hooks. OTOH, PEP 302 already defined hooks,
> > and it seems that people are happy with these.
>
> No, they're not. Yes, the hooks are *usable*, but they're damn hard to
> comprehend. When even the *experts* hate messing with a subsystem,
> it's a hint that there's something wrong with the way it is set up.

This is the big motivator for my talk proposal at the next PyCon, on
getting the most out of Python imports.  They're woefully under-utilized
IMHO, exactly because the import machinery is generally poorly understood.

-eric

> In
> this case, I firmly believe a big part of the problem is that the
> import system is a complex, interdependent mechanism, but there's no
> coherence to the architecture. It's as if the whole thing was written
> in C from an architectural point of view, but without even bothering
> to create a dedicated structure to hold the state.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Import-SIG mailing list
> Import-SIG at python.org
> http://mail.python.org/mailman/listinfo/import-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/import-sig/attachments/20111113/3470d5d6/attachment.html>