[C++-sig] Re: Re: sub module support in V2

[Dave Hawkes]

"David Abrahams" <david.abrahams at rcn.com> wrote in message
news:1f3401c20de3$226029e0$6601a8c0 at boostconsulting.com...
>
> ----- Original Message -----
> From: "Dave Hawkes" <daveh at cadlink.com>
> Newsgroups: gmane.comp.python.c++
> To: <c++-sig at python.org>
> Sent: Friday, June 07, 2002 12:14 AM
> Subject: [C++-sig] Re: sub module support in V2
>
>
> >
> > "David Abrahams" <david.abrahams at rcn.com> wrote in message
> > news:1efd01c20dc9$a1aa10e0$6601a8c0 at boostconsulting.com...
> >
> >
> > I don't know of any platforms that do that as it would be difficult to
> > implement unless all functions were exported,
>
> All functions are exported on most Unix variants.

Though they will only know about each other if they are explicity linked to
each other. If we have to separate execution units linked to a common shared
library, they would not be linked to each other.

>
> > though come to think of it I
> > seem to remember a codewarrior for mac option that exports everything,
> but
> > then I suppose all your shared libraries together become a single
> > translation unit. That aside I can probably fix it by making the
> module_link
> > a base class of a new class that is created using the module name which
> then
> > holds the static singleton.
>
> That's very complicated. There's only one outer-level extension module
> being loaded at any given time. Why don't you just keep track of that in a
> single global which lives in the library.

Because if we have multiple extension libraries linked to a common BPL then
there will be more than one 'global' module as far as the BPL is concerned
when these extensions are imported by the same python session.

>
>
> > I'm not sure how to accomplish this by using an unnamed namespace,
> probably
> > the solution I outlined above is adequate and won't rely on the ODR.
>
> I'm a little less worried about relying on the ODR than on relying on a
> violation of it. That is, the language standard says "there's one copy of
> any static data member of a class", and you said "I'm going to write code
> that only works if there are multiple copies". Of course, you can't always
> rely on things to work the way the standard says when shared libs are
> involved, but relying on things NOT working the way the standard says
seems
> a bit... perverse to me.

Unless the extension library is actually linked to the other this should not
be a problem, but I think I have a better way of doing it which should be
more robust. I'll test and let you know.

>
> > > >
> > > > This macro would automatically chain from BOOST_PYTHON_INIT even in
> > > separate
> > > > code modules.
> > >
> > > Why do we need the macro?
> > >
> >
> > To save the user the bother of calling the other init functions directly
> > inside BOOST_PYTHON_MODULE_INIT
>
> What other init functions? Why doesn't it just happen inside the module
> constructor?
>

It sounds like I haven't explained what this achieves very well so I'll give
a code example:

instead of:

BOOST_PYTHON_MODULE_INIT(MyMod)
{
    module("MyMod").Add(...
    ...
    OtherModules1();
    OtherModules2();
}

OtherModules1()
{
    module("MyMod.SubMod1").Add(...
    ...
}

OtherModules2()
{
    module("MyMod.SubMod2").Add(...
    ...
}

we would have something like:

BOOST_PYTHON_MODULE_INIT(MyMod)
{
    module("MyMod").Add(...
    ...
}

BOOST_PYTHON_MODULES()
{
    module("MyMod.SubMod1").Add(...
    ...
}

BOOST_PYTHON_MODULES()
{
    module("MyMod.SubMod2").Add(...
    ...
}

where the BOOST_PYTHON_MODULES() could be located in separate source files.

> > > > transparantly, a sub-module can be defined before its parent.
> > >
> > > Bizarre. What makes that work?
> > >
> >
> > As part of the process of creating a sub-module, each parent must be
> checked
> > to see if it exists at each step. If it does not exist we could either
> > assert or transparantly create it anyway. I just chose to do the latter.
> > Then if the parent is referenced later we just pull a reference to the
> > previously created but yet unused module.
>
> Ah. Defined, but not created.
>

In fact it has to be created so the sub-module attribute can be added. It is
just not passed back at the time.

> > > > > 8.
> > >
> > > PyObjects don't have destructors. Do you mean its __del__ method? It
> still
> > > won't throw a C++ exception.
> > >
> >
> > Yes I mean __del__ method which in our case can eventually land up
> calling:-
> >
> > delete static_cast<function*>(p);
>
> Which still doesn't throw.
>

It might not throw a c++ exception, but it can sill throw an access
violation, for example, which is caught in catch(...) and possibly saves the
interpreter from exiting, though may be we don't want that functionality
anyway?

> > Also note that there is a final catch(...) which catches more than just
> C++
> > exceptions when using MSVC. In MSVC it will also catch access violations
> and
> > floating point exceptions (etc.).
>
> And screw up your JIT debugging. Yes, I know.
>
> > As an aside these are best handled by SEH
> > anyway, particularly when they are continuable
>
> What SEH exceptions are continuable?
>

Usually floating point exceptions where we use the MS _fpieee_flt
functionality. Typically done in CAD packages that heavily use floating
point so we can correct minor overflow/underflow/Inexact problems without
aborting the program. Of course libraries with catch(...) handlers are a
menace in this case as it prevents the outer floating point SEH block from
catching the exception.

> > , but I'm not going to suggest
> > an SEH block function for continuable floating point exceptions prior to
> the
> > catch(...)...
>
> If you know a way to make some things smoother on Windows, I'm listening,
> as long as it doesn't actually cover up bugs.
>

I'll look in to it. The best solution may well be to give the user an option
of avoiding the final catch(...) handler somehow so any outer SEH is not
messed up. Unknown c++ exceptions would have to be dealt with by the user as
well, but anyone doing this in the first place should be more than capable
of dealing with that minor problem.

> > > So what? We're linked to a shared runtime.
> > >
> >
> > That maybe, but MS still provides a separate local heap for different
> dlls,
>
> That's perverse (seems to be my adjective-of-the-day).
>
> > just try a straight malloc in one dll, passing the pointer to another
and
> > freeing it. new and delete just use the regular malloc and free and have
> the
> > same issues if care is not taken.
>
> It works just fine for me. The enclosed Boost.Build project does just that
> with the debugging runtime and raises no complaints.
>

I checked my projects and for some reason I was using the single threaded
libraries on the extension module.

> > Also not that, in fact, we may not even have a common run time for a
> number
> > of reasons. Sometimes the libraries are statically linked for
> localisation
> > reasons or to get around some MFC bugs/features that can only be
resolved
> by
> > statically linking.
>
> I'm not supporting that configuration. I suppose I might consider adding
> the virtual destructor with a switch for people who wanted to do that, but
> I wouldn't make any guarantees about it.
>
> > > > I got exceptions as a result of this and adding
> > > > the virtual destructor fixed it.
> > >
> > > Please provide a reproducible test case.
> > >
> >
> > If your still not sure a virtual destructor is needed I'll look at
> packaging
> > up my MSVC test workspace files and sending them to you. Alternatively I
> > could use one of the other techniques that don't require a virtual
> > destructor to fix the problem.
>
> I just want proof that there is a local heap per-DLL.
>

Is it a good idea to rely on there being a single common shared library for
allocation? What if the user overloads the global operator new function in
the extension library for customised memory allocation?

Dave Hawkes