[Python-ideas] "Loose" descriptors

[Andrew Barnert]

On Mar 28, 2015, at 19:17, Steven D'Aprano <steve at pearwood.info> wrote:
> 
>> On Sun, Mar 29, 2015 at 11:51:31AM +1300, Greg Ewing wrote:
>> James Edwards wrote:
>>> I envision (but with no real knowledge of the implications of this)
>>> that just as python identifies that `val` in `Holder.__dict__`
>>> implements `__get__`, python could similarly identify that `val` in
>>> `globals()` (or `sys.modules[__name__].__dict__`, or wherever is more
>>> appropriate) implements `__get__` and handle the indirection.
>> 
>> This would slow down *all* lookups of global names in
>> order to support a rarely-used feature. Since global
>> name lookup is critical performance-wise, this is
>> likely to meet a lot of resistance.
>> 
>> Similar arguments have been made concerning the
>> support of descriptors in module namespaces, which
>> is a closely related idea.
> 
> I don't think it's a closely related idea, I think it is *exactly* the 
> same idea. If I've understood him correctly, James wants to use 
> descriptors in global variables, i.e. module namespaces, not just class 
> attributes.

Yeah, that's how I interpreted it too.

The obvious way to do this is to provide some way for a module to define not only an instance of the module type, but a subclass and an instance of that subclass. The subclass then provides a natural place for the descriptors to live (and a natural answer to what gets passed for inst and typ).

There was a proposal for something very much like that some time late last year, IIRC, which might be worth digging up.

(Modules that don't define a subclass could still be easily optimized for fast global lookup, because the compiler knows that the base module class doesn't have any descriptors.)

> You are right that making this the default behaviour would cause 
> slow-down of all module-level lookups for very rare advantage, but what 
> if we moved the descriptor logic into the global __dict__ instead of the 
> compiler? Then, in thory at least, supporting the descriptor protocol 
> for global variables will be opt-in on a per-module basis.
> 
> Here's a sketch of how you might do it.
> 
> class DescriptorDict(dict):
>    def __getitem__(self, key):
>        obj = super().__getitem__(key)
>        if hasattr(type(obj), '__get__'):
>            # I'm not sure what to pass as instance here.
>            return type(obj).__get__(obj, instance)
>        return obj
> 
> 
> __dict__ = DescriptorDict(__dict__)
> 
> 
> I'm sure there are complications I haven't even begun to imagine, but 
> here are a few obvious ones:
> 
> (1) Can modules use a dict subclass for __dict__? 
> (2) Can you replace the __dict__ on the fly like this?
> (3) If not, how do you bootstrap the module to have a magic dict instead 
>    of an ordinary dict?
> (4) The descriptor protocol expects to pass the instance and/or class 
>    to the __get__ method, as well as `self`. self is the descriptor 
>    itself; what is instance? The module?
> 
> 
> I don't know if it can be done, but if it could be, it would certainly 
> open the doors for a lot of interesting experiments.
> 
> 
> 
> -- 
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/