[C++-sig] dynamic compile and "to-Python converter ... second conversion method ignored"

[troy d. straszheim]

Stefan Seefeld wrote:
> On 11/20/2009 05:50 AM, Eilif Mueller wrote:
>> Hi,
>> 
>> I'm using boost.python to interface with some dynamically generated
>>  C++ code, employing some scipy.weave facilities to cache the
>> results. When I force a recompile and import, I get the following
>> message:
>> 
>> RuntimeWarning: to-Python converter for __randint already
>> registered; second conversion method ignored.
>> 
>> where __randint is the dynamically generated class.  As such, the 
>> changes to the code (__randint) are not represented in the newly 
>> imported module.
>> 
>> Is it possible to override this behaviour so that the new class 
>> replaces the registration for the old one?
>> 
> 
> I very much doubt that what you want is possible: All extension
> modules using boost.python share a single global 'registry' that
> holds type-conversion information. Thus it is impossible to have that
> loaded while a separate process is adding new entries to it.
> 
> (It's a global symbol that is seen by all DSOs. Having two DSOs
> provide that symbol, but with different content, is illegal.)
> 

I'd like to fix this, now, it is starting to cause me pain.

For instance, it also rears its ugly head if you import two libraries
that both wrap std::vector<double>, for instance if you try to use the 
IceTray physics analysis suite together with Paul Kunz's wonderful 
Hippodraw.

Another situation that came up recently was that the pair<T,U> inside 
std::map<T, U> and another container of T,U (that used pair<T,U> 
internally) collided.  I think it should be possible to specify that a 
type's wrapper is replaceable (if you know, for instance, that two 
projects wrap std::pair<string,int> the same way).

I haven't thought about the interface much yet.  Here's something I just 
decided against, a per-type global 'replacability' flag, that warns or  not:

template <typename T>
void replacable<T>(bool quietly);

BOOST_PYTHON_MODULE(mymod)
{
   replacable<std::pair<string,int> >(true);
}

But this doesn't help the situation where you're trying to use two
different python modules together, neither of which specifies that pairs
are replacable.  It seems that boost.python should expose some
way to specify replacability at runtime.  How about this.  A type's 
converters are 'locked', 'verbose', and 'nonthrowing' by default (like 
they are now).
Boost python modules expose one function

   def __lock__(type, lock, verbose, throw): ...

at module scope that allows you to specify whether

- types are locked down or replacable
- collisions should be complained about to stdout
- collision events should throw

e.g.

   >>> import M1

   # silently replace T's
   >>> M1.__lock__(M1.T, lock=False, verbose=False, throw=False)

   # verbosely refuse replacement of tees
   >>> M1.__lock__(M1.T, lock=True, verbose=True, throw=False)

And that this __lock__ function would also be available on the C++ side
for setting defaults:

BOOST_PYTHON_MODULE(M1)
{
   class_<T>("T")
     ...

   lock<T>(false, false, false); // silent, replacable, nonthrowing

}

Just a sketch.  Thoughts?

-t