[C++-sig] Re: The "return always existing pointer throws dangling error" problem

[David Abrahams]

"Niall Douglas" <s_sourceforge at nedprod.com> writes:

> On 21 Oct 2003 at 16:30, David Abrahams wrote:
>
>> > Let me clarify: From what I understand, most return policies destroy
>> > their associated C++ object instance when its python representation
>> > expires. Mostly that's a copy of the C++ object instance and when
>> > this is the case, the throwing of the error is correct.
>> >
>> > However, if the object instance was allocated by new (and thus its
>> > policy was manage_new_object), 
>> 
>> Objects don't have call policies; those are associated with
>> methods/functions.
>
> I had thought that when a function with a return policy returns 
> object X, the lifetime ie; validity of X is tied to something else?

Sometimes, if that's what the policy says to do with it.  That doesn't
mean the policy is associated with the object.  In the case of
manage_new_object the resulting Python object holds the C++ object
with an auto_ptr, and all information about the policy is lost.  There
are other ways to get an identical Python object without using
manage_new_object.

>> > return_internal_reference is also different - if you attach
>> > lifetime to self which most do, then when a virtual override in
>> > python returns an internal reference
>> 
>> ?? huh ??
>> A virtual override in Python can only return a Python object.
>> There's no such thing as an "internal reference" in that context.
>
> So when there is a virtual MyObject *MyClass::foo() and my wrapper of 
> MyClass overrides default foo() with one calling call_method<MyObject 
> *>(self, "foo") then call_method is returning a python object?

No, call_method is clearly returning a MyObject*.  The "virtual
override in Python" is returning a Python object.

> How can that possibly be when foo() and all overrides of it must 
> return MyObject * (with the exception of covariant returns)?

Well, of course the Python function doesn't literally override the C++
one; you can't do that since the two languages don't (and couldn't)
have a common ABI.  The last actual override of the C++ virtual
function is the one you implement, which invokes call_method.  That,
in turn, looks up a Python method with the appropriate name and
invokes it.  The only way C++ gets involved again before call_method
picks up the resulting Python object and converts it to C++ is when
that Python method either happens to be a wrapped C++ function
(e.g. the default implementation of MyClass::foo forwarded from the
wrapper class), or when the Python method happens to call some
wrapped C++ function directly or indirectly.

>> > to either C++ or python code invoking that function
>> 
>> When a virtual function overridden in Python is invoked from Python,
>> C++ and BPL are never involved.  The function is simply looked up and
>> called in the usual Python way.  When invoked from C++, the call
>> policies are never involved, since they only apply to wrapped C++
>> functions and a Python function is actually being called.
>
> So you're saying that the _class.def() part has absolutely zero 
> relation to the call_method part? They are two bits of absolutely 
> unrelated code?

No, they have many relationships.  I'm not sure which kind of
relationship you have in mind, though.

> If so, then wow. How could any transient objects created by 
> call_method know their contents are destined for C++ world? 

They don't.

> Maybe that's actually the problem?

I have no idea yet.  Maybe when you post some *compilable*, *testable*
sample code I'll have some idea.

>> > surely BPL is attaching the lifetime of the internal reference to
>> > the "self" instance and not to the temporary object created during
>> > call_method?  Therefore surely the error is reporting the wrong
>> > thing dangling?
>> 
>> Given everything I've written above, what you're saying makes no
>> sense to me.
>
> Once again, I appear to misunderstand how BPL works :(
>
>> More details about your problem are needed.  My suggestion: for the
>> time-being, stop trying to second-guess the design of Boost.Python and
>> just describe what you're trying to do with a *minimal example*.  If
>> we can't come up with a good solution, then we can talk about design
>> changes.
>
> Done below.

I should've said "minimal but complete".

>> > I appreciate that this has been addressed before, and that Dave who
>> > is <censored> has said he's not sure about the correct solution. 
>> > But I can't see how BPL doesn't know where one of its own pointer
>> > containers got its pointer from?
>
> Err, the <censored> bit was saying a compliment, not anything 
> negative in any way whatsoever - I was using a construct borrowed 
> from German (consequence of being European). My apologies if you took 
> offense at that.

I didn't at all; I was just too embarrassed to quote you saying that
;->

>
> Ok, my problem in condensed form (identical to the two problems as 
> referenced in the initial post):

I'll be the judge of that... when you post a complete example.

> namespace FX {
> class FXMetaClass;
>
> class FXObject
> {
> public:
>     static const FXMetaClass metaClass; // metadata for this subclass
>     virtual const FXMetaClass *getMetaClass() const { return 
> &metaClass; } // metadata for most derived subclass
> };
>
> class FXApp : public FXObject
> { ... };
> };
>
> struct FX_FXObject_Wrapper: FX::FXObject
> {
>     const FX::FXMetaClass* getMetaClass() const {
>         return call_method< const FX::FXMetaClass* >(self, 
> "getMetaClass");
>     }
>
>     const FX::FXMetaClass* default_getMetaClass() const {
>         return FX::FXObject::getMetaClass();
>     }
> };
>
> ...
> void Export_FXObject()
> {
>     class_< FX::FXObject, FX_FXObject_Wrapper >("FXObject", init<  
>>())
>         .def("getMetaClass", &FX::FXObject::getMetaClass, 
> &FX_FXObject_Wrapper::default_getMetaClass, 
> return_internal_reference< 1 >())
>         ;
>     class_< FX::FXMetaClass >("FXMetaClass", init< const 
> FX::FXMetaClass& >())
>     ...
> }
>
> And when you do in python:
> a=FXApp()
  ^^^^^^^^^
You haven't even given me the code needed to make this line possible.
Please try to make it easy to help you.


-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com