Lazy-evaluation lists/dictionaries

[Jon Ribbens]

I have a need, in a Python C extension I am writing, for lists and
dictionaries with "lazy evaluation" - by which I mean that at least
some of the values in the lists/dictionaries are "proxy objects"
which, rather than returning as themselves, should return the thing
they are a proxy for when retrieved. This is because retrieving
the proxied objects is expensive and only a small minority of them
will actually be accessed, so retrieving them all before they are
actually accessed is massively inefficient.

With object attributes, this can be easily done with the descriptor
protocol and having properties which look like simple objects but
actually cause a method to be invoked when they are accessed. However
there doesn't seem to be any equivalent way of doing this for
retrieving items from lists or dictionaries - unfortunately, the
method implementations of list() and dict() are full of direct
accesses to the underlying data pointers rather than indirecting
through obj->tp_as_mapping->mp_subscript or whatever.

Is there any better way to do this other than simply re-implementing
these types from scratch, emulating all their methods and operations?
(i.e. using UserList/UserDict). I was under the impression that that
sort of thing was supposed to have gone since Python 2.2 or so.