[Python-Dev] Patching builtin_id to allow for C proxy objects?

[Stefan Behnel]

Tom Whittock, 27.06.2011 12:48:
> I'm writing a module to proxy C++ objects into Python for a large C++
> application. There are hundreds of thousands of C++ objects, some of
> which are temporary while others are very long lived.
>
> Currently every time one of these objects is accessed from Python, a
> new "myproxy" instance is created. So if I were to access the same
> field of an object twice, I would receive two python objects proxying
> the same underlying C++ object. This messes up "id" and "is"

Note that "is" actually compares the addresses, not the id().


> and is
> causing me issues when, for example, I run into circular references
> when enoding json or otherwise attempt to determine whether two proxy
> objects refer to the same C++ object.
>
> I can't see how to cache the "myproxy" objects instead of returning
> new instances - due to the architecture of the C++ application,
> there's no weak reference support at all, and the number of objects is
> very large.
>
> My current plan would be for me to override the id builtin to return
> the underlying C++ object instance pointer instead of the PyObject
> instance pointer in the case of the "myproxy" object type

Where would you get the proxy object from anyway?

IMHO, there are two obvious way get what you want: map the C++ object 
address (integer!) to a proxy object using a dict, or use a backpointer 
from the C++ object to its proxy. The second is substantially faster, but 
may require changes to the C++ class struct.

I don't see how changes to CPython's core can help you here.

Stefan