[C++-sig] location of initialization matters?

[Martin Blais]

hi guys

I've got a query related to initialization and Boost.Python.  in a
project i'm working on for a client, they have a pretty involved
initialization framework which allows dynamically-loaded libraries
to be initialized in the same order as the inter-library
dependencies.  Basically, for each package, they have a
registration procedure that occurs upon loading the library (using
static initialization), each package registers itself to a global
singleton, and after all the libraries are loaded, the registered
initialize/finalize functions are called in the proper order, order
which is determined at compile time with auto-generated code that
uses the linking info from their makefile.  This requires specific
function calls to their single library registry before loading any
DLLs and after DLL loading has terminated, to trigger the
initialization.  They have a C++ interface for that (i.e. a guard).

To solve the problem of loading Python extension modules linked
with their C++ libraries, I have overridden the definition of
import to make the calls to their init begin (before performing the
real import) and init end (right after) which then triggers their
init framework calls.  Thus the initialization order is:

0. a call to import one of the extension modules is made;

1. from my overridden import, I call their init singleton to
    prepare for lib registration;

2. then I call original python import mechanism to load the
    ext. module as usual

3. the library gets loaded and registers itself to their init
    singleton; all dependent DLLs are also loaded and register to
    the init singleton as well (also registering their
    dependencies);

4. the usual ``initMODULE`` gets called by python, Boost.Python's
    init_module gets called;

5. import exits, and i make a call to their init singleton to
    signal we're done; at this point, all of their libraries'
    initialize() functions are called in the appropriate dependency
    order by their init singleton.

The problem I'm having, is that I need for their libraries to have
been initialized before I run the my Boost.Python code to export
references, because some values that I want to export get set only
during their initialization framework.  Now this means that I need
to perform the last two steps like this instead:

...

4. initMODULE gets called by python, does nothing because I
    declared almost nothing (just the module_builder object);

5. import exits, and i call their init singleton to termination
    initialization; at this point, all of their libraries' initialize()
    functions are called in the appropriate order by their init
    singleton.

6. within those initialize functions of each package, I run the
    code that I wrote that uses Boost.Python to register the classes
    and references to be exported to the Python interface.

Basically, this means that the Boost.Python declarations are not
executed within the ``init<module>`` function/scope anymore, but
rather a little bit later, within their own init framework.

This used to work with Boost.Python v1; now, I'm porting to v2, and
peeking inside the new Boost.Python, I see that there is the notion
of a "current scope".  The only thing that seems a prerequisite is
that when initMODULE gets called, Python seems to require that
Py_InitModule gets called.  I can do that by using the usual
Boost.Python macro, with an empty body for that function (i.e. not
export any symbols).

(I hope my description was clear enough.)

A few questions:
----------------

A. do I need to recreate that scope object to be able to register
    new definitions?  Will new definitions work fine even if not run
    from within the initMODULE function called from Python?  In
    other words, can I arbitrarily define new types and bind new
    functions after initialization?

B. I couldn't find documentation other than a note about "Automatic
    Cross-Module Type Conversions" about cross-module linking and
    usage.  If my module links with other extension modules, do I
    need to call ``PyImport_ImportModule`` for this to work?

    I need to link with the other module for the symbols to resolve
    (this is under XP).  I figure that if I do indeed move the init
    code within their own init framework, simply linking with
    another ext. module will call the boost code without having to
    tell Python about the virtual "import" that this creates (and I
    want to avoid having that function called twice, or at least
    protect against it).  I'm not sure if that will register the
    module that is linked to properly though, i.e. if module B links
    with module A but does not otherwise import it, will module A
    show up as imported to the script?

    And what if module A is imported beforehand in the script?
    Hmm...

C. Do you see any issues that may arise from this funky usage?