[Python-ideas] Idea to support lazy loaded names.

[Steven D'Aprano]

On Mon, Oct 06, 2014 at 09:34:16PM -0400, Brian Allen Vanderburg II wrote:
> 
> When developing Python libraries (and any other language for that
> matter), items are separated into modules for organization and other
> purpose.  The following is an example fake framework to illustrate this:
[...]
> One idea I had was to support lazy imported names.  I've seen some
> frameworks implement this in various means, and figured the idea is good
> to be part Python.
> 
> The idea is that for a new module attribute to exist: __lazyload__.
> During the access of any attribute for a module, the following would
> happen:

This is a specific example of a more general technique, namely computed 
attributes. Rather than a solution to the specific example, I would 
rather have a general solution to the problem of having computed 
attributes in modules. We have half a solution already: the descriptor 
protocol, the most obvious example being property(). Well, perhaps a 
quarter of a solution.

Assuming property(), or some equivalent, worked in modules, the solution 
to lazy loading is simple: have a getter that returns the attribute if 
it already exists, otherwise initialise it (importing it from elsewhere) 
then return it.

The problem then becomes, how do we get descriptors to be run when 
accessing them via module attributes? The normal process is that if a 
name is found in the instance __dict__, the descriptor protocol is not 
followed:

py> class Test(object):
...     spam = property(lambda self: 23)
...     def __init__(self):
...             self.eggs = property(lambda self: 42)
... 
py> x = Test()
py> x.spam
23
py> x.eggs
<property object at 0xb7168c5c>


This is relevant because modules are instances, and module attributes 
live in the instance __dict__.


>   * If the named attribute already exists, use it
>   * If the named attribute does not already exist:
>     * If a lazy load of the name has already been attempted, result in
>       a NameError
>     * If a lazy load of the name has not yet been attempted:
>        * Check the __lazyload__ module attribute for the name, perform
>          the loading operation and assign the module attribute the
>          value if found or result in a NameError
>        * Even if not found, set a flag that lazy load has been
>          attempted so it will not be attempted again for the same name

I don't think that you should record that the import has been tried and 
failed. Regular imports cache successes, not failures, and lazy imports 
should do the same: just because an import failed a minute ago doesn't 
mean that it will fail now.



-- 
Steven