[Tutor] New Style Classes, __getitem__ and iteration

[Kent Johnson]

On Mon, May 19, 2008 at 5:21 PM, Marilyn Davis <marilyn at deliberate.com> wrote:
> Hi Tutors and Tutees,
>
> I've been teaching Python quite a while and a brilliant student asked a
> question that made me realize a big hole in my understanding.
>
> I think it is really magical that, when I define __getitem__ on classic
> classes, the built-in iterator uses it.
>
> But, when I make the same class as a new style class, I lose this behavior.

There are a couple of possible explanations, I'm not sure which is
right without looking into the implementation of sorted().

Wesley may be right, that sorted() sees that its argument is a list
and uses a lower-level access than __getitem__() to copy the list for
sorting.

In general, list operations are not implemented in terms of
__getitem__(). In other words, you can't subclass list(), override
__getitem__() and expect to change how all list operations work.
__getitem__() is not a bottleneck procedure.

It's also possible that sorted() just expects a sequence and uses the
iterator protocol to access the values of its argument. This is
certainly the case with your Circle class. This class doesn't define
an __iter__() method so Python uses the older iteration protocol based
on __getitem__().

With NewCircle, even if sorted() is accessing it with an iterator,
your __getitem__() will not be called because list implements
__iter__() so iteration will use the modern iteration protocol.

------------------

OK, I looked at the source a bit.

builtin_sorted(), in bltinmodule.c, calls PySequence_List() to create
a list to sort.
PySequence_List(), in abstract.c, creates a new list, then calls
_PyList_Extend() to add the new items.
_PyList_Extend(), in listobject.c, calls listextend() which is the
implementation of list.extend()
listextend() is clearly optimized for lists and tuples. I didn't dig
farther than that. For other data types it iterates over the argument.

A good discussion of the iterator protocol is here:
http://www.python.org/doc/2.2.3/whatsnew/node4.html

Kent