[Python-ideas] Idea to support lazy loaded names.

[Andrew Barnert]

On Oct 6, 2014, at 20:51, Terry Reedy <tjreedy at udel.edu> wrote:

> On 10/6/2014 11:30 PM, Steven D'Aprano wrote:
> 
>> This is a specific example of a more general technique, namely computed
>> attributes. Rather than a solution to the specific example, I would
>> rather have a general solution to the problem of having computed
>> attributes in modules. We have half a solution already: the descriptor
>> protocol, the most obvious example being property(). Well, perhaps a
>> quarter of a solution.
>> 
>> Assuming property(), or some equivalent, worked in modules, the solution
>> to lazy loading is simple: have a getter that returns the attribute if
>> it already exists, otherwise initialise it (importing it from elsewhere)
>> then return it.
>> 
>> The problem then becomes, how do we get descriptors to be run when
>> accessing them via module attributes? The normal process is that if a
>> name is found in the instance __dict__, the descriptor protocol is not
>> followed:
>> 
>> py> class Test(object):
>> ...     spam = property(lambda self: 23)
>> ...     def __init__(self):
>> ...             self.eggs = property(lambda self: 42)
>> ...
>> py> x = Test()
>> py> x.spam
>> 23
>> py> x.eggs
>> <property object at 0xb7168c5c>
>> 
>> 
>> This is relevant because modules are instances, and module attributes
>> live in the instance __dict__.
> 
> So a 'solution' might be to make modules be instances (but with no __new__ or __init__) of a module metaclass, so that module dicts could act like class dicts with respect to descriptors.  I have no idea how much code this would break ;-).

Didn't we just have this discussion a few weeks ago, in the context of making lazy loading of subpackages easier to implement?

IIRC, the not-obviously-unreasonable options suggested were:

1) Analogy with __new__: For packages only, if there's a __new__.py, that gets executed first. If it "returns" (not sure how that was defined) an instance of a subclass of ModuleType, that instance is used to run __init__.py instead of a normal module instance.

2) Analogy with metaclass= (or with 2.x __metaclass__): If a module (or a package's __init__.py) does some new syntax or magic comment before any non-comment code, it can specify a custom type in place of ModuleType (not sure how that type gets imported and made available).

3) Analogy with re-classing object instances: Just allow modules to set __class__ during execution (or after, if you want). Variations on this include allowing that for all non-heap types, or even getting rid of the concept of non-heap types.

4) Make it easier to write import hooks for this special purpose.

5) Make it easier for a module to construct a ModuleType-subclass instance with a copy of the same dict and replace itself in sys.modules.

It seems like any of these would allow defining properties, other descriptors, and special methods on modules.