[C++-sig] Boost Python v2 - extending and embedding - memory leaks, compiler warnings

[Tex Riddell]

--- David Abrahams <dave at boost-consulting.com> wrote:
> Tex Riddell <texrmex at yahoo.com> writes:
>
*snip* 
> > I'm using VC7 IDE to build my project and link to python22.lib and
> > boost_python.lib (built with VC7 command line tools using bjam).  
> 
> Proceed at your own risk. Many people report problems which are due
> to
> getting the wrong project configuration using the IDE. I strongly
> suggest you use the -n -a option with some of the bjam examples to
> see
> how the command-line options come out, so you can match them.

This, and Bret's suggestion, I'm sure will prove to be most helpful.

> > I'm attempting to extend and embed in my existing project, so I'm
> > not building a dll, but and exe.  First of all, can I do this?  
> 
> Yes. However, there are some caveats. Boost.Python keeps some Python
> objects alive in global variables (it has some registries and a few
> other resources it needs). There is currently no integration with
> PyFinalize() to get these cleaned up, so if you call PyFinalize()
> your
> application may crash on exit as the reference counts are decremented
> and the dead Python interpreter is invoked. This is something we plan
> to address in a future version.

That would be most helpful, how do you suggest I proceed in the mean
time?  I don't think it will be acceptable for my uses to just live
with leaks or crashes.  Is there any way I can clean up these objects
before the app exits?

> > When it links it says "Creating library ....lib and object
> ....exp",
>        ^^
> What is "it"?

Sorry, that's VC7's linker.  In light of the build information I can
get from bjam, are there any simple examples of an app that extends and
embeds, instead of building to a .pyd?  Then I might be able to clean
up my build configuration.

> > and still creates an exe.  If I don't include my test module (and
> > thus boost/python.hpp) I don't get this message.  I'm assuming that
> > by defining a module, boost defines exports and the linker then
> > creates the lib and exp, since there were exports?  Although I've
> > built dlls before, I really don't know the inner workings of the
> > process.
> 
> If you're talking about what you're building in the IDE, I have no
> clue and no control over that. Boost.Python itself lives in a DLL,
> though if you're determined to you there are ways to link it into
> your
> app directly. Normally, the code your module gets from boost.python
> generates only /imports/ from boost_python.dll, so I really have no
> idea.

My intent was not to link Boost.Python directly, but to embed my
extension directly in my app, instead of producing a .pyd.  That, along
with embeding the interpreter.  I have done this in the past with
Boost.Python v1, but not v2.

> > The following is a brief example of my module.  At the bottom is a
> > simple test function that I'm calling just to test importing of the
> > module.  When I call TestModule(), I get 2 memory leaks from the
> module
> > definition section, one from the class definition
> 
> Yes, extension modules are never unloaded, and classes in the
> extension module are kept alive additionally by Boost.Python's
> registry.

Is this to accomidate cross-extension module communication?

*snip*

> > In v1 there was a vc project you could use for building boost
> > python.  This works well if you are planning on linking to a vc
> > built project, since you can make sure your settings, paths and
> libs
> > are consistent between the boost python lib and your project.  I
> > currently have 3 different sets of build tools (yes, 3 compilers)
> on
> > this box, with several different versions of standard and other
> > libraries, so making sure I have control of the paths and settings
> > used is very important.
> 
> That's why we created Boost.Build. I'm currently building and testing
> with 11 different compilers on this machine alone, and many others on
> other machines.
> 
> > This is one reason I'm weary of the jam environment.  It's yet
> > another build environment, one that I'm not familiar with, and one
> > that I'm not going to be using with any other projects except to
> > build boost python.  
> 
> Your loss ;-)

> > So a question would be: how easy would it be for me to set up a VC
> > project for building boost python?  Has this already been done?  Or
> > am I barking up the wrong tree?
> 
> Personally I think you're barking up the wrong tree, since
> maintaining
> platform-specific solutions like that is difficutl, but other people
> have done it and I believe I'm probably going to end up having to
> supply one eventually :(

Well being able to see all the commands generated by bjam will solve
the unknown factor with integration with our project tools, which will
never be changed to jam (not up to me ;).

> -- 
>                        David Abrahams
>    dave at boost-consulting.com * http://www.boost-consulting.com
> Boost support, enhancements, training, and commercial distribution

Thanks for all your help on this!  Now I might just be ready to ask my
*real* BPL questions :P ... when I have time to get back to this that
is.

-Tex

__________________________________________________
Do you Yahoo!?
U2 on LAUNCH - Exclusive greatest hits videos
http://launch.yahoo.com/u2