[Python-ideas] Override dict.__new__ to raise if cls is not dict; do the same for str, list, etc.

[Steven D'Aprano]

On Thu, Apr 21, 2016 at 10:35:01PM -0400, Random832 wrote:
> On Thu, Apr 21, 2016, at 22:00, Steven D'Aprano wrote:
> > I don't understand your question. Or rather, if I have understood it, 
> > the question has a trivial answer: you don't have to override anything.
> 
> I thought it went without saying 

Obviously not.

> that I want to be able to get items
> other than by having them already stored in the dict under the exact
> key. 

"Items"? Like items in a dict? Then you already have at least two ways: 
you override __getitem__, or add a __missing__ method to the dict.

What makes you think these techniques don't work?


> Like if I wanted to make a case-insensitive dict, or one where
> numbers are equivalent to strings, or have it materialize a default
> value for missing keys.
> 
> > "So what method should be overridden..."
> > 
> > but the answer to this in general must be "What makes you think a single 
> > method is sufficient?"
> 
> Well, I was assuming it doesn't call *more than one* method for the
> specific task of looking up an attribute by name, 

Where does attribute lookup come into this?


> even if there are
> other methods for other tasks that I would have to override to get the
> whole package of what behavior I want. As near as I can tell, it doesn't
> call *any method at all*, but instead reaches inside PyDictObject's
> internal structure.

Are you still talking about *attribute lookup* or are you back to *key 
lookup*?

Honestly Random, you have to be clear as to which you want, because they 
are different thing. You can't just jump backwards and forwards 
between talking about dicts and attributes and expect people to 
understand what you mean.

Classes and instances may not have a __dict__ at all, if they define 
__slots__ instead, or if they are builtins:

py> (1).__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__dict__'



> > This doesn't just apply to dicts, it applies in 
> > general to any class.
> > 
> > "What method do I override to make a subclass of int implement 
> > arithmetic with wrap-around (e.g. 999+1 = 0, 501*2 = 2)?"
> 
> Right, but I'm asking which method *one particular* expression calls.
> This is more like "Which method on an object implements the + operator",

An excellent example, because the answer is *two* methods, __add__ and 
__radd__. You've made my point for me again.


> only it's "Which method on an object's type's namespace dict implements
> the ability to look up attributes on that object?"

What makes you think that such a method exists?

Have you read the rest of this thread? If not, I suggest you do, because 
the thread is all about making unjustified assumptions about how objects 
are implemented. You are doing it again. Where is it documented that 
there is a method on the object __dict__ (if such a __dict__ even 
exists!) that does this?

So you are asking the wrong question. Don't ask about an implementation 
detail that you *assume* exists. You should ask, not "which method", but 
"how do I customise attribute lookup?"

And you know the standard answer to that: override __getattribute__ or 
__getattr__. Do they not solve your problem?

Another smart question might be, what are the constraints and rules for 
customizing behaviour by setting the object __dict__ to something other 
than a regular builtin dict? I don't know the answer to that.

But don't make assumptions about the implementation, and having made 
those assumptions, assume that everyone else shares them and that they 
"go without saying". *Especially not* in a thread that talks about how 
silly it is to make those assumptions.


> Given a type, how do I get a reference to the dict instance that was
> passed to the type's constructor?

I believe that the answer to that is, you can't.


> > > Alternatively, where, other than object and class dicts, are you
> > > actually required to have a subclass of dict rather than a UserDict or
> > > other duck-typed mapping?
> > 
> > Anywhere you have to operate with code that does "if isinstance(x, 
> > dict)" checks.
> 
> Fix the other code, because it's wrong. If you can't, monkey-patch its
> builtins.

You cannot assume that it is "wrong" just because it is inconvenient for 
you. Do you seriously think that monkey-patching the built-ins in 
production code is a good idea?



-- 
Steve