[C++-sig] Re: Problem: held type, std::auto_ptr, ReferenceErrors.

[Prabhu Ramachandran]

>>>>> "DA" == David Abrahams <dave at boost-consulting.com> writes:

    >> If it will help other users in the future I will try and
    >> summarize this discussion and put it together so it can be part
    >> of the docs.  I personally feel that a high level "how it works
    >> - with examples" kind of write up would make it easier for
    >> characters like me to understand what is going on without
    >> having to dive into the code.  If you think this is a good
    >> idea, I'll try and work on something when I get the time.

    DA> Yes, please!  BTW, I've had several promises of such
    DA> documentation efforts but so far nobody has had the fortitude
    DA> to turn my ramblings into a real document... so you could
    DA> distinguish yourself by following through!

OK, I have three hard deadlines coming up so time is short right now
(as can be seen from my delayed response) *but* I hope to get
something going by September end and something reasonable by mid Oct.
One of those deadlines I mentioned above is a lightning talk (10
minutes) I'm planning on giving at SciPy'03 on "Wrapping with SWIG and
Boost.Python: a comparison".  The emphasis is to provide a basic
comparison and feel for wrapping with either tool (nothing too deep or
advanced).  I'll make those slides also available to the general
public once the talk is over.  I will also show you the slides
*before* I present it so that I don't say something incorrect about
Boost.Python.  I think that might also prove useful to some users.  I
know I'm no expert on either tool but I have used both to wrap some
non-trivial code.

    DA> For example, here's something which was supposed to become
    DA> real documentation:
    DA> http://www.boost-consulting.com/boost/libs/python/doc/internals.html
    DA> (it's in your CVS tree).

Yes, I looked at it and noticed the messages on the list.  My plan is
not to go into such depth but to provide a higher level overview.  It
might even be a repetition of the tutorial with a little more detail
but I hope re-reading similar material from different angles with more
examples will give users a clearer understanding.

[snip]

    >> As an aside, I think Nicodemus and I would be happy if you did
    >> run Pyste sometime. :)

    DA> Yes, so much to do; so little time.  I'm interested, but I
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Amen to that!

    DA> don't see that there would be huge benefits to the community
    DA> if I did that.

Yes, I understand.

    >> Anyway, here is hopefully a minimal test case for "Problem 1".

    DA> <snip>

    >> $ python test_bug.py In add_wrapper Segmentation fault

[snip]

    DA> Hmm, but with gcc-2.95.3, I get:

    DA>   In add_wrapper Traceback (most recent call last):
    DA>     File "foo.py", line 12, in ?
    DA>       func(x)
    DA>   AttributeError: 'wrapper_descriptor' object has no attribute
    DA>   'f' [5464 refs]

    DA> Hmm, the latter are using Python 2.3; the former using 2.2.2.
    DA> Which version of Python are you using?

Python 2.2.1 (Debian GNU/Linux stable).

[snip]

    DA> Ick.  Here's the problem: the "self" pointer is dangling by
    DA> the time B_Wrapper::f is invoked.  This just reinforces my
    DA> previous conclusion that we need a more formalized way to
    DA> handle these Wrapper classes, e.g. with a common base class
    DA> something like this one:

[snip]

    DA> And then we need an adopt policy something like what Daniel
    DA> Wallin has in luabind, but which is smart enough to detach()
    DA> all VirtualDispatcher objects.

Ahh, I see.  So, I'll go over this again.  When I do:

 b = B()
 h.add(b)
 del b

the Python object holding the B_Wrapper is gone (del b).  So
B_Wrapper's 'self' member is garbage.  This is why I needed to do 'del
b' or use 'h.add(B())' to run into the problem.

Hmm, I think I can use the VirtualDispatcher idea in my case since
AFAIK I simply need to add a detach() call inside the add_wrapper
where the ownership is transferred.  I'll also have to make the
wrapper accept a std::auto_ptr<B_Wrapper> instead of std::auto_ptr<B>.
Am I missing a subtle issue here?  From what I understand the "adopt
policy" will do this in a transparent way at the Boost.Python level.


[ call_method and dangling reference problem ]
    >> Is there an easier way that Boost.Python could handle this
    >> better?

    DA> Easier for you; harder for me.  I don't have time to solve
    DA> these problems quickly at a deep level in Boost.Python, at
    DA> least, not for free ;-)

I understand. :)

Anyway, I'm not in a big hurry and am hoping these issues will be
solved sometime in the future.

Till then *my* Boost.Python wrappers will be non-optimal.  The trouble
is that I have quite a few classes that need to hand off the ownership
to a class that looks like the Holder shown in the above.  I also will
run into problems with the call_method dangling reference problem.
Essentially I have about 70 odd classes to wrap (not all of them have
the problem, of course!) and don't have the time to hand wrap them
all.  Yes, I can generate basic wrappers with Pyste and modify them
suitably and use the modified wrappers.  However, I would like to do
this whole thing using Pyste alone which is why I've been banging away
at Pyste and bothering Nicodemus so much.  This is also a way for me
to contribute back to you guys.  Essentially, from my efforts so far I
have pretty much everything else working except for these two problems
(VirtualDispatcher and the call_method dangling reference issue).  I
might run into more problems but I can't predict those.  None of this
stuff is urgent, so I can wait.

So a humble request.  Is there some kind of *approximate* time frame
when these two particular issues will be solved?  If its not too long
I'll wait.  If not I'll simply use the current Pyste generated files,
modify them by hand (based on your hints) and use the modified files
to generate the wrappers.

Thanks once again for everything Dave!  Nicodemus, thanks for the
Pyste support!

cheers,
prabhu