[Patches] [ python-Patches-995939 ] Avoid calling tp_compare with different types

[SourceForge.net]

Patches item #995939, was opened at 2004-07-22 14:14
Message generated for change (Comment added) made by ddorfman
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=995939&group_id=5470

Category: Core (C code)
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Dima Dorfman (ddorfman)
Assigned to: Nobody/Anonymous (nobody)
Summary: Avoid calling tp_compare with different types

Initial Comment:
C implementations of the tp_compare slot usually expect both of their 
arguments to have the type for which that slot is defined (this isn't 
documented anywhere that I can find, but many core types assume this, 
and extension authors are likely to do the same). A problem occurs if a 
user-defined nb_coerce slot (__coerce__) returns objects with different 
types. Avoid the problem by refusing to call a non-Python tp_compare 
unless the arguments have the same type. Indiscriminately calling 
tp_compare with different types is wrong as long as there are 
implementations that don't check the type of the second argument, but 
Python implementations should be allowed to receive a different type.

Other options for fixing the problem:
  - Make PyNumber_CoerceEx require that the results have the same 
type. This would prevent a user-defined __coerce__ from 
communicating an arbitrary object to a user-defined __cmp__. 
Furthermore, there are uses for coerce besides comparison, and those 
uses might not require identical types.
  - Change tp_compare implementations to check the type of the 
second argument. This might be the most desirable long-term solution, 
but as I understand it, tp_richcompare is preferred over tp_compare for 
new code, and old (current) code is likely assuming that the types are 
the same.

Addresses bug #980352


----------------------------------------------------------------------

>Comment By: Dima Dorfman (ddorfman)
Date: 2004-08-05 09:29

Message:
Logged In: YES 
user_id=908995

The patch is intended to be complete, but it's certainly possible that I 
missed something. Every other call to tp_compare checks that the 
operands have the same type or the same compare function; the only 
exceptions are if one of operands passes PyInstance_Check or if it's 
_PyObject_SlotCompare (it's treated specially in try_3way_compare).

I'm actually more concerned about restricting too much than having 
missing something. The tests I wrote try to make sure that the 
user-visible changes are limited to the absence of a crash, but I'm not 
sure that I didn't accidently break some code that used to work.

Comparing subclasses of builtins to builtins still works, too. On line 599 
of the post-patch object.c, we see:

	/* If both have the same (non-NULL) tp_compare, use it. */
	if (f != NULL && f == w->ob_type->tp_compare) {
		c = (*f)(v, w);
		return adjust_tp_compare(c);
	}

which takes care of that case.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2004-08-03 09:36

Message:
Logged In: YES 
user_id=4771

Is your patch complete?  There seem to be a lot of places in object.c that call tp_compare slots.  Shouldn't we try to make sure that none of these places can ever be called with two different types?

It also looks like we have to do something special about a value of &_PyObject_SlotCompare in tp_compare, because this function from typeobject.c checks itself the type of its arguments and does specific things with them.  This should be thought about.

Moreover there is the issue of the user subclassing built-in types, e.g. PyInt_Type.tp_compare is capable of comparing different user subclasses of "int" for each other.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=995939&group_id=5470