[ python-Bugs-1164631 ] super(...).new( ... ) behaves "unexpected"

Sat Mar 19 11:32:52 CET 2005

Bugs item #1164631, was opened at 2005-03-16 17:07
Message generated for change (Comment added) made by brenck
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1164631&group_id=5470

Category: Type/class unification
Group: Python 2.4
>Status: Deleted
Resolution: None
Priority: 5
Submitted By: Dirk Brenckmann (brenck)
Assigned to: Nobody/Anonymous (nobody)
Summary: super(...).__new__( ... ) behaves "unexpected"

Initial Comment:
Hello there,

python code and trace output enclosed.

What I did:

1. Metaclass inheritence:

   type <-- MA <-- MB <-- MC

2. Created Class A using __metaclass__= MC

3. Create Class B(A) using __metaclass__= MB

...although this might seem strange, it should work...

When taking a look at the trace, you will notice one line 
that goes like:

'-------> why does super( MA, metacls ).__new__ call MC.
__new__ in next line ????????????????????'

if you run the code, you will find it three times. That's ok. 
In my trace I just replaced two occurences of that line by 
">>" to enable focussing on the Problem...

What I would expect the code to do is the following:

1. Create a Class A which is of type MC

2. Create a Class B(A) which is of type MB

What the interpreter does is different:

1.   Create a Class A which is type MC

2.1 Nearly create a Class B which is of type MB.

2.2 In type.__new__( ... ) change it's mind.

2.3 Due to the superclass A<MC> of B, create some 
class A which is of type MC as well. Although  B 
contains a __metaclass__ = MB statement.

Well - that's what I experienced is "the way windows 
works", so I ran the code on Solaris again but the 
behaviour remains reproduceable...

I would consider it a bug therefor. If it's not a bug, I would 
expect an Exception which tells me where I did wrong...

Thanx for your time and efforts

----------------------------------------------------------------------

>Comment By: Dirk Brenckmann (brenck)
Date: 2005-03-19 10:32

Message:
Logged In: YES 
user_id=360037

Okay, that's the point.
Thanx for your time and work - bug becomes deleted.

----------------------------------------------------------------------

Comment By: Nick Coghlan (ncoghlan)
Date: 2005-03-19 01:30

Message:
Logged In: YES 
user_id=1038590

Additional text from the descrintro essay, indicating that
it is deliberate that C3 does not trigger an exception:

"""However, if one of the base metaclasses satisfies the
constraint (including the explicitly given __metaclass__, if
any), the first base metaclass found satisfying the
constraint will be used as the metaclass."""

So, unless Guido chooses to change the desired behaviour,
these two snippets pretty much cover what needs to be added
to the docs.

----------------------------------------------------------------------

Comment By: Nick Coghlan (ncoghlan)
Date: 2005-03-19 00:17

Message:
Logged In: YES 
user_id=1038590

To address the documentation side, the following text from
Guido's descrintro essay could be added to the official
documentation:

"""For new-style metaclasses, there is a constraint that the
chosen metaclass is equal to, or a subclass of, each of the
metaclasses of the bases. Consider a class C with two base
classes, B1 and B2. Let's say M = C.__class__, M1 =
B1.__class__, M2 = B2.__class__. Then we require
issubclass(M, M1) and issubclass(M, M2). (This is because a
method of B1 should be able to call a meta-method defined in
M1 on self.__class__, even when self is an instance of a
subclass of B1.)""" 

----------------------------------------------------------------------

Comment By: Nick Coghlan (ncoghlan)
Date: 2005-03-19 00:14

Message:
Logged In: YES 
user_id=1038590

Minimal case that should throw an exception on C3, but
doesn't (instead, it silently replaces the explicitly
requested metaclass M1 with its subclass M2):

class M1(type):
  pass

class M2(M1):
  pass

class C1(object):
  __metaclass__ = M1

class C2(C1):
  __metaclass__ = M2

class C3(C2):
  __metaclass__ = M1

----------------------------------------------------------------------

Comment By: Jim Jewett (jimjjewett)
Date: 2005-03-18 19:14

Message:
Logged In: YES 
user_id=764593

Yes, it is intentional.  

class Derived(Base): ...

should mean that you can use an instance of Derived 
anywhere you need an instance of Base.

There are ways to break this, but it isn't a good idea.

Letting Derived ignore part of the metaclass (and creation 
instructions) of Base would make it much easier to create an 
invalid Derived by accident -- and the error could show up as 
memory corruption, instead of something meaningful.

----------------------------------------------------------------------

Comment By: Dirk Brenckmann (brenck)
Date: 2005-03-18 15:48

Message:
Logged In: YES 
user_id=360037

Ok - I think I found the reason for the behaviour I explained 
above ...

typeobject.c:
  function type_new(...)
   [...]
   /* Determine the proper [...]
  <line 1611, 1612, 1613>
    if (PyType_IsSubtype(tmptype, winner)) {
	winner = tmptype;
	continue;
    }

These three lines mean, it would never be possible to create a 
subclass using a (metatype which is a subclass of the 
metatype) of it's superclass.
In such cases, (even) a metaclass (explictly set by a 
programmer) is ignored and replaced by a metaclass which 
python decides...

... do you really want this to be that way????

This means, one always ends up in a (meta-)class hierarchy 
that is fixed. Programmers dynamically can't decide anymore, 
if there should be a level in their class hierarchy which should 
*NOT* use the 'highest subclassed metaclass' in their class 
hierarchy.

I would consider this a problem, because the use of 
__metaclasses__ will be implicitly restricted

----------------------------------------------------------------------

Comment By: Dirk Brenckmann (brenck)
Date: 2005-03-18 14:41

Message:
Logged In: YES 
user_id=360037

Ok - I think I found the reason for the behaviour I explained 
above ...

typeobject.c:
  function type_new(...)
   [...]
   /* Determine the proper [...]
  <line 1611, 1612, 1613>
    if (PyType_IsSubtype(tmptype, winner)) {
	winner = tmptype;
	continue;
    }

These three lines mean, it would never be possible to create a 
subclass using a (metatype which is a subclass of the 
metatype) of it's superclass.
In such cases, (even) a metaclass (explictly set by a 
programmer) is ignored and replaced by a metaclass which 
python decides...

... do you really want this to be that way????

This means, one always ends up in a (meta-)class hierarchy 
that is fixed. Programmers dynamically can't decide anymore, 
if there should be a level in their class hierarchy which should 
*NOT* use the 'highest subclassed metaclass' in their class 
hierarchy.

I would consider this a problem, because the use of 
__metaclasses__ will be implicitly restricted

----------------------------------------------------------------------

Comment By: Dirk Brenckmann (brenck)
Date: 2005-03-18 14:13

Message:
Logged In: YES 
user_id=360037

Ok - I think I found the reason for the behaviour I explained 
above ...

typeobject.c:
  function type_new(...)
   [...]
   /* Determine the proper [...]
  <line 1611, 1612, 1613>
    if (PyType_IsSubtype(tmptype, winner)) {
	winner = tmptype;
	continue;
    }

These three lines mean, it would never be possible to create a 
subclass using a (metatype which is a subclass of the 
metatype) of it's superclass.
In such cases, (even) a metaclass (explictly set by a 
programmer) is ignored and replaced by a metaclass which 
python decides...

... do you really want this to be that way????

This means, one always ends up in a (meta-)class hierarchy 
that is fixed. Programmers dynamically can't decide anymore, 
if there should be a level in their class hierarchy which should 
*NOT* use the 'highest subclassed metaclass' in their class 
hierarchy.

I would consider this a problem, because the use of 
__metaclasses__ will be implicitly restricted

----------------------------------------------------------------------

Comment By: Dirk Brenckmann (brenck)
Date: 2005-03-18 10:38

Message:
Logged In: YES 
user_id=360037

Ok - I think I found the reason for the behaviour I explained 
above ...

typeobject.c:
  function type_new(...)
   [...]
   /* Determine the proper [...]
  <line 1611, 1612, 1613>
    if (PyType_IsSubtype(tmptype, winner)) {
	winner = tmptype;
	continue;
    }

These three lines mean, it would never be possible to create a 
subclass using a (metatype which is a subclass of the 
metatype) of it's superclass.
In such cases, (even) a metaclass (explictly set by a 
programmer) is ignored and replaced by a metaclass which 
python decides...

... do you really want this to be that way????

This means, one always ends up in a (meta-)class hierarchy 
that is fixed. Programmers dynamically can't decide anymore, 
if there should be a level in their class hierarchy which should 
*NOT* use the 'highest subclassed metaclass' in their class 
hierarchy.

I would consider this a problem, because the use of 
__metaclasses__ will be implicitly restricted

----------------------------------------------------------------------

Comment By: Dirk Brenckmann (brenck)
Date: 2005-03-16 17:11

Message:
Logged In: YES 
user_id=360037

Sorry - 2.3 must be corrected:

2.3 Due to the superclass A<MC> of B, create some 

class B which is of type MC as well. Although B 

contains a __metaclass__ = MB statement.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1164631&group_id=5470

[ python-Bugs-1164631 ] super(...).__new__( ... ) behaves "unexpected"

[ python-Bugs-1164631 ] super(...).new( ... ) behaves "unexpected"