[C++-sig] EXTERNAL: Re: Odd dlopen behavior

[Niall Douglas]

On 2 Feb 2012 at 6:00, Davidson, Josh wrote:

> Neil, great information, but I did track this problem down to a quirk with Py++.
> [snip]
> After finding references to this problem as far back as 2006, I decided
> to switch over to balanced_split_module.  This has its own set of
> problems.  The first is that it is highly prone to divide by zero
> errors.  One quick way to reproduce this issue is to wrap one class and
> specify a split count of 2.  Obviously not a wise combo, but it's an
> easy error case that Py++ should handle. 

It's been a long time since I used Py++, or indeed BPL. Neither has 
seen much work done on them in recent years, so I should imagine both 
will have suffered from a certain amount of bitrot.

> So anyways, the root of *this* problem is how balanced_split_module
> creates its registration functions.  For each extension, it creates one
> register function for each file it writes in the form:  void
> register_classes_<N>()    Obviously, these collide when you create more
> than one extension using balanced_split_module and enable RTLD_GLOBAL. 

In my own code, registration functions are always static and pass 
their own address into the runtime. The runtime does a two-pass 
initialisation, so first off it eliminates any duplicate registration 
functions whose address lies within the same DLL. Then and only then 
does it start complaining.

That solution allows you to throw registration functions anywhere and 
let the runtime sort out what's what. It also lets you operate 
per-DLL and DLL-specialised registries, something I suggested to Jim 
Bosch for the next release of BPL a few months back. Knowing which 
DLL registered what is also very useful for debugging.

> One quick solution to this problem would be to prepend the extension
> name to the name of the registration functions, e.g.:
> <module>_register_classes_<N>  Since the module name is used to name the
> files, its easily accessible and would solve a lot of problems.  Of
> course, if you have modules with the same name in different packages you
> would run into this again. 

This is a bad solution. Firstly, who is to say that type Foo in 
extension X is or is not the same as type Foo in extension Y even if 
both have the same type, same length, same traits and live in the 
same namespace? Yes I know it's a violation of ODR, but in the real 
world maybe they are the same, and maybe they aren't - people use 
DLLs to violate ODR all the time, it's one of their big utilities. 
You need a way of explicitly specifying if the types are equal or 
aren't. Type registration functions aren't the place to do it, 
however in my own code I have a concept of "type conversion" 
registration functions which are used to declare runtime type 
conversions which kick in if the static (metaprogrammed) type 
conversions fail. When you say type Foo is the same between 
registries X and Y it simply bumps that type out of X and Y and into 
the common parent registry between X and Y, so searches fail in X and 
Y and jump into the parent where they get resolved as being 
identical.

Now me and Dave Abrahams come to a disagreement after this - he does 
not feel that there ought to be separate static and runtime type 
registries. And I can see his point. But we disagree :). I'm very 
fond of binding together other people's libraries in order to 
personally avoid writing as much code as possible, so I think in 
terms of ways of getting other people's code to cooperate, and for 
that I like to override when needs be which is blatent ODR breaking. 
Dave has a different approach, and therefore a different philosophy.

(And before Dave points out that that wasn't my philosophy in the 
past, and indeed he once argued with me strenuously to use Boost 
rather than write my own metaprogramming library, I admit I was wrong 
and he was right)

However, back to the point. It would appear you're running into well 
known limitations in BPL. Some years ago, people's needs were 
generally simpler and BPL worked without issue for them. As years 
move on, more and more complex use cases are becoming common.

Bitrot. It consumes all code eventually.

Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.