[ python-Bugs-1683368 ] object.__init__ shouldn't allow args/kwds

[SourceForge.net]

Bugs item #1683368, was opened at 2007-03-18 20:32
Message generated for change (Comment added) made by blakeross
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1683368&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Blake Ross (blakeross)
Assigned to: Guido van Rossum (gvanrossum)
Summary: object.__init__ shouldn't allow args/kwds

Initial Comment:
object.__init__ currently allows any amount of args and keywords even though they're ignored. This is inconsistent with other built-ins, like list, that are stricter about what they'll accept. It's also inconsistent with object.__new__, which does throw if any are provided (if the default __init__ is to be used).

To reproduce:

object.__init__(object(), foo, bar=3)

This is a slight irritation when using cooperative super calling. I'd like each class' __init__ to cherry-pick keyword params it accepts and pass the remaining ones up the chain, but right now I can't rely on object.__init__ to throw if there are remaining keywords by that point.

----------------------------------------------------------------------

>Comment By: Blake Ross (blakeross)
Date: 2007-03-21 15:03

Message:
Logged In: YES 
user_id=1747060
Originator: YES

Looks good. I skimmed briefly the tests you mentioned. The issue with
test_array appears to be exactly the kind of bug this is intended to
identify: it calls array.__init__(...), but array doesn't have its own
initializer, so object's is used.

I'd guess that the others are failing due to whatever the problem with
pickling is (test_descr uses pickling). I haven't looked into that yet.

I'm sure cooperative super calling of __init__ isn't all that common (it
seems like the mechanism itself isn't used much yet, and may not be until
it's done via keyword) but it doesn't seem like such a bad practice,
especially when mixins are in the picture. There doesn't seem to be a great
alternative.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-20 14:24

Message:
Logged In: YES 
user_id=6380
Originator: NO

I should mention that if we can't get the strict version of this in 2.6,
we should be able to get it into 3.0.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-20 13:54

Message:
Logged In: YES 
user_id=6380
Originator: NO

Here's a stricter version. Unfortunately it breaks a couple of standard
modules; this is a confirmation of my doubts whether the style of
cooperative super calling of __init__ that you use is really the most
common or "best practice".

So far I have only fixed string.py (which would otherwise prevent
extensions from being built); I haven't looked into why the other tests
fail: test_array, test_cpickle, test_descr, test_pickle (and maybe more?).

My conclusion: this would probably break too much code to be worth it.  So
I'll have to revert to the previous version.  But anyway, here it is for
your perusal.
File Added: new_init_strict.patch

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-19 18:27

Message:
Logged In: YES 
user_id=1747060
Originator: YES

I think making the check more rigid is a good idea, since this should
throw:

class a(object):
   def __init__(self, foo):
       super(a, self).__init__(foo)
   def __new__(cls, foo):
       return object.__new__(cls)
a(1)

(minor typo in the patch: "solution it" -> "solution is")

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-19 16:35

Message:
Logged In: YES 
user_id=6380
Originator: NO

This smells enough like a new feature that it couldn't go into 2.5.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-19 16:31

Message:
Logged In: YES 
user_id=6380
Originator: NO

Attached is a patch that implements this proposal, adding copious
commentary.  It doesn't seem to break anything in the test suite.

I wonder if we should even make the check more rigid: check the argument
list if either the current method *is* overridden or the other one *is not*
overridden.  This would make super calls check the arguments even if the
other method is overridden.  What do you think?
File Added: new_init.patch

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-19 15:45

Message:
Logged In: YES 
user_id=1747060
Originator: YES

Makes sense. I don't think we can ever be completely correct here since
we're inferring intent from the presence of __init__/__new__ that's liable
to be wrong in some cases, but it's likely correct often enough that it's
worth doing.

If I understand correctly, we want to be more forgiving iff one of the two
methods is used, so it seems like we should be complaining if both are used
*or* if neither is used. After all, I could add a __new__ to my coop use
case and I'd still want object to complain. If that's the case, both
object_new and object_init should be complaining if ((tp->tp_new ==
object_new && tp->tp_init == object_init) || (tp->tp_new != object_new &&
tp->tp_init != object_init)).

Of course, for the paranoid, there's always the risk that __new__ will
modify these class functions and change the outcome :) For instance, if a
class had a __new__ and no __init__ and its __new__ changed __new__ back to
object.__new__, object_init on that run would be fooled into thinking it's
using the defaults for both and would complain. I think this could only be
fixed in type_call, which is rather ugly...but then, this *is* a special
case of the "call __init__ after __new__" behavior, and we're trying to
solve it within the methods themselves. Perhaps this last point is academic
enough to be ignored...I don't know why anyone would do this, although the
language makes it possible.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-19 13:01

Message:
Logged In: YES 
user_id=6380
Originator: NO

I'll try to explain why I did it this way.  I was considering the single
inheritance case implementing an Immutable object, which overrides __new__
but has no need to override __init__ (since it's too late to do anything in
__init__ for an Immutable object).  Since the __init__ still gets called it
would be annoying to have to override it just to make the error go away if
there was a check in __init__.  The other case is overriding __init__
without overriding __new__, which is the most common way of doing Mutable
objects; here you wouldn't want __new__ to complain about extra args.  So
the only time when you'd want complaints is if both __new__ and __init__
are the defaults, in which case it doesn't really matter whether you
implement this in __init__ or in __new__, so I arbitrarily chose __new__.

I wasn't thinking of your use case at the time though (cooperative super
calls to __init__, which still isn't something I engage in on a day-to-day
basis).  I wonder if the right thing to do wouldn't be to implement the
same check both in __init__ and in __new__.

Am I makign sense?

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-03-19 01:56

Message:
Logged In: YES 
user_id=849994
Originator: NO

I don't really understand either why object_new() checks the arguments,
not object_init():

"""
static int
object_init(PyObject *self, PyObject *args, PyObject *kwds)
{
	return 0;
}

/* If we don't have a tp_new for a new-style class, new will use this
one.
   Therefore this should take no arguments/keywords.  However, this new
may
   also be inherited by objects that define a tp_init but no tp_new. 
These
   objects WILL pass argumets to tp_new, because it gets the same args as
   tp_init.  So only allow arguments if we aren't using the default init,
in
   which case we expect init to handle argument parsing. */
static PyObject *
object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
	if (type->tp_init == object_init && (PyTuple_GET_SIZE(args) ||
	     (kwds && PyDict_Check(kwds) && PyDict_Size(kwds)))) {
		PyErr_SetString(PyExc_TypeError,
				"default __new__ takes no parameters");
		return NULL;
	}
	return type->tp_alloc(type, 0);
}
"""

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1683368&group_id=5470