Bug? If not, how to work around it?

[Bengt Richter]

On Thu, 07 Aug 2003 00:57:30 +0100, Gonçalo Rodrigues <op73418 at mail.telepac.pt> wrote:

>On 6 Aug 2003 19:35:18 -0400, aahz at pythoncraft.com (Aahz) wrote:
>
>>In article <u8m2jvsus3om8r96ggnej9s7u3iecv3e29 at 4ax.com>,
>>Gonçalo Rodrigues  <op73418 at mail.telepac.pt> wrote:
>>>
>>>>>> class Test(object):
>>>... 	def __init__(self, obj):
>>>... 		self.__obj = obj
>>>... 	def __getattr__(self, name):
>>>... 		return getattr(self.__obj, name)
>>>... 	
>>>>>> a = Test([])
>>>>>> a.__iter__
>>><method-wrapper object at 0x0112CF30>
>>>>>> iter(a)
>>>Traceback (most recent call last):
>>>  File "<interactive input>", line 1, in ?
>>>TypeError: iteration over non-sequence
>>>>>> 
>>>
>>>Is this a bug? If not, how to code Test such that iter sees the
>>>__iter__ of the underlying object?
>>
>>As Mark guessed, iter() goes directly to the attribute rather than using
>>the __getattr__ machinery of the class.  However, you can intercept it
>>using a metaclass, but that requires a bit of fancy footwork to reach
>>down into the instance to get self.__obj.
>
>Actually I came across this because of a little metaclass I was
>coding. And my head hurts already as it is...
>
>So, is it a bug? In other words, should I open a bug report?
>
I don't think it's a bug, but there's a lot to explore ;-)
Here is something I've been hacking at to do some exploring. Maybe you will find it interesting:

====< for_grodrigues.py >=============================================
class Test(object):
    def __init__(self, *args, **kw): pass
    def __new__(cls, obj):
        # helper to capture method in closure for method delegator
        def mk_meth_deleg(obj, meth, name):
            def meth_deleg(arg0, *args, **kw):
                if isinstance(arg0, type):
                    return meth(arg0, *args, **kw)
                else:
                    return meth(obj, *args, **kw)
            return meth_deleg
            
        # get the objects's class dict to build a new class
        proxy_cdict = obj.__class__.__dict__.copy()
        for name, meth in proxy_cdict.items():
            if( name.startswith('__') and
                callable(meth) and
                #not name.startswith('__getattr') and
                not name == '__init__'
            ):
                m = mk_meth_deleg(obj, meth, name)
                proxy_cdict[name] = m

        # customize proxy class as desired here
        def __getitem__(self, i):
            if i==2: return '[Test-modified %r]'%obj[i]
            else: return obj[i]
        proxy_cdict['__getitem__'] = __getitem__
       
        #derive from object's class if possible for stuff we haven't caught??
        try:
            type('',(obj.__class__,), {})
        except TypeError:
            proxy_base = object
        else:
            proxy_base = obj.__class__
        proxy_class = type(
            obj.__class__.__name__+'_proxy',
            (proxy_base,),
            proxy_cdict)
        proxy_obj = proxy_class()
        proxy_obj.instattr = 'proxy_obj instance attribute set by Test'
        return proxy_obj
======================================================================
Sample interaction:

 >>> from for_grodrigues import Test
 >>> a = Test([1,2,3])
 >>> a
 [1, 2, 3]
 >>> type(a)
 <class 'for_grodrigues.list_proxy'>
 >>> a.__iter__
 <method-wrapper object at 0x00902990>
 >>> iter(a)
 <listiterator object at 0x00902970>
 >>> a.__iter__()
 <listiterator object at 0x00902910>
 >>> ait = iter(a)
 >>> ait.next()
 1
 >>> for i in a: print i,
 ...
 1 2 3

Note that the __iter__ bypassed the doctored __getitem__ of our proxy object, vs:

 >>> for i in range(3): print a[i],
 ...
 1 2 [Test-modified 3]


 >>> b = Test(None)
 >>> type(b)
 <class 'for_grodrigues.NoneType_proxy'>
 >>> b
 None
 >>> b[2]
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "for_grodrigues.py", line 26, in __getitem__
     if i==2: return '[Test-modified %r]'%obj[i]
 TypeError: unsubscriptable object

Now we try to iterate over the string, but it doesn't provide __iter__ apparently,
since we can see that it wound up using __getitem__:

 >>> c = Test('abcd')
 >>> for x in c: print x,
 ...
 a b [Test-modified 'c'] d

vs, again the list:
 >>> a
 [1, 2, 3]
 >>> type(a)
 <class 'for_grodrigues.list_proxy'>
 >>> for i in a: print i,
 ...
 1 2 3

 >>> c
 'abcd'
 >>> c[::-1]
 'dcba'
That doesn't use __getitem__ either.

By inheriting from the object's class, we pick up base class methods:

 >>> class A(object):
 ...     def afoo(self): return 'afoo()'
 ...
 >>> class B(A):
 ...     def bfoo(self): return 'bfoo()'
 ...
 >>> b = B()
 >>> pb = Test(b)
 >>> pb.bfoo()
 'bfoo()'
 >>> pb.afoo()
 'afoo()'
 >>> type(pb)
 <class '__main__.B_proxy'>

There are interesting issues of proxy object attribute space vs the real object vs the
proxy class methods vs the real object's methods, vs inheritance for it, etc.
and you could alter the priorities.

You can mess with it yourself, and maybe eliminate some hacking cruft ;-)

Regards,
Bengt Richter