[C++-sig] Re: Interest in luabind

[Daniel Wallin]

> >> I think that problem is a little more complicated than
> >> you're making it out to be, and that your method ends
> up
> >> being slower than it should be in inheritance graphs of
> >> any size.  First of all, inheritance may be a DAG and
> you
> >> have to prevent infinite loops if you're actually going
> >> to support cross-casting.  Secondly Boost.Python caches
> >> cast sequences so that given the most-derived type of
> an
> >> object, you only have to search once for a conversion
> to
> >> any other type, and after that you can do a simple
> >> address calculation.  See
> >> libs/python/src/object/inheritance.cpp.  This probably
> >> should be better commented; the algorithms were hard to
> >> figure out and I didn't write down rationale for them
> :(
> >> On the other hand, maybe being fast isn't important in
> >> this part of the code, and the cacheing should be
> >> eliminated ;-)
> >
> > We only support upcasting, so our method isn't that
> slow.
>
> Surely you want to be able to go in both directions,
> though?
> Surely not everyone using lua is interested in just speed
> and not usability?

We probably would like to be able to go in both directions.
We also don't want to force the user to compile with RTTI
turned on, so we currently supply a LUABIND_TYPEID macro to
overload the typeid calls for a unique id for the type. This
of course causes some problems if we want to downcast, so we
would need to be able to turn downcasting off, or let the
user supply their own RTTI-system somehow.

>
> > Generally it's just a linked list traversal. We don't
> > cache though, and caching is a good thing. :)
>
> Yeah, probably.  It would be good to share all of that.

Sure.

>
> >> > Ok, how do you handle conversions of lvalues from c++
> >> > -> python?  The to_python converter associated with a
> >> > UDT does rvalue conversion and creates a new object,
> >> > correct?
> >>
> >> Yeah.  Implicit conversion of lvalues by itself with no
> >> ownership management is dangerous so you have to tell
> >> Boost.Python to do it.  I'm sure you know this, though,
> >> since luabind supports "parameter policies."  Bad name,
> >> though: a primary reason for these is to manage return
> >> values (which are not parameters).  So I wonder what
> >> you're really asking?
> >
> > We convert lvalues to lua with no management by default.
> I
> > don't think this is more dangerous than copying the
> > objects, it's just seg faults instead of silent
> > errors.
>
> <shiver>
> Your way, mistakes by the user of the *interpreter* can
> easily crash the system.  My way, only the guy/gal doing
> the
> wrapping has to be careful:
>
>          >>> x = X()
>          >>> z = x.y
>          >>> del x
>          >>> z.foo()  # crash
>
> The users of these interpreted environments have an
> expectation that their interpreter won't *crash* just
> because of the way they've used it.
> </shiver>

Right, I thought you always copied the object. My mistake.

>
> > Both ways are equaly easy to make mistakes with.
>
> Totally disagree.  Done my way, we force the guy/gal to
> consider whether he really wants to do something unsafe
> before he does it.  You probably think I copy objects by
> default, but I don't. That was BPLv1.  In BPLv2 I issue an
> error unless the user supplies a call policy.
>
> Finally, let me point out that although we currently use
> Python weak references to accomplish this I realized last
> night that there's a *much* easier and more-efficient way
> to do it using a special kind of smart pointer to refer to
> the referenced object.

Ah ok, function which returns lvalues causes compile time
errors. When we decided to do it our way we thought
returning unmanaged lvalue's would be the most common usage.
We only considered copying the object as an alternative,
perhaps it's better to give compile time errors.

>
> > Our policies primary reason is not to handle return
> > values, but to handle conversion in both directions. For
> > example, adopt() can be used to steal objects that are
> > owned by the interpreter.
> >
> > void f(A*); def("f", &f, adopt(_1))
>
> What, you just leak a reference here?  Or is it something
> else?  I had a major client who was sure he was going to
> need to leak references, but he eventually discovered that
> the provided call policies could always be made to do
> something more-intelligent, so I never put the
> reference-leaker in the library.  I haven't had a single
> request for it since, either.

The above is (almost) the equivalent of:
void f(auto_ptr<A>*);

It is very useful when wrapping interfaces which expects the
user to create objects and give up ownership.

>
> > But yes, the name should indicate both
> > directions..  ConversionPolicy perhaps.
>
> Hmm, this is really very specific to calls, because it
> *does* manage arguments and return values.  I really think
> CallPolicy is better.  In any case I think we should
> converge on this, one way or another; there will be more
> languages, and you do want to be able to steal my users,
> right? <wink>.  That'll be a lot easier if they see
> familiar terminology ;-)

I don't think it's specific to calls, but to all conversion
of types between the languages. We can use policies when
fetching values from lua, or when calling lua functions from
c++:

A* stolen_obj = object_cast<A*>(get_globals(L)["obj"],
adopt(result));

call_function<void>(L, "f", stolen_obj) [ adopt(_1) ];

And yeah, of course stealing your users is our goal. :)

> >> I'm not committed to the idea of a single registry.  In
> >> fact we've been discussing a hierarchical registry
> system
> >> where converters are searched starting with a local
> >> module registry and proceeding upward to the package
> >> level and finally ending with a global registry as a
> >> fallback.
> >
> > Right, that seems reasonable.
>
> Cool.  And let me also point out that if the module
> doesn't
> have to collaborate with other modules, you don't even
> need
> a registry lookup at static initialization time.  A static
> data member of a class template is enough to create an
> area
> of storage associated with a C++ type.  There's no central
> registry at all in that case.  I have grave doubts about
> whether it's worth special-casing the code for this, but
> it
> might make threading easier to cope with.

I can't see why it would be worth it. If the module doesn't
interact with other modules threading wouldn't be an issue?
So how could it make it easier?

>
> >> My big problem was trying to figure out a scheme for
> >> assigning match quality.  C++ uses a kind of
> >> "substitutaiblity" rule for resolving partial ordering
> >> which seemed like a good way to handle things.  How do
> >> you do it?
> >
> > We just let every converter return a value indicating
> how
> > good the match was, where 0 is perfect match and -1 is
> no
> > match. When performing implicit conversions, every step
> in
> > the conversions inreases the match value.
> >
> > Maybe I'm naive, but is there need for anything more
> complicated?
>
> Well, it's the "multimethod problem": consider base and
> derived class formal arguments which both match an actual
> argument, or int <--> float conversions.  How much do you
> increase the match value by for a particular match?

I don't know if I get this.

We just increase the match value by one for every casting
step that is needed for converting the types.

>
> >> > Right. The requirement I was aiming to resolve was
> that we need a
> >> > different set of parameters when doing our
> conversions.
> >>
> >> I consider that an implementation detail ;-)
> >>
> >> > You have your PyObject*, we have our (lua_State*,
> int).
> >>
> >> What's the int?
> >
> > An index to the object being converted on the lua stack.
>
> Oh, I guess lua hasn't handed you a pointer to an object
> at
> that point?  Well, OK.

Right. Also, you can't get a pointer to all objects in lua,
only "userdata" objects. If the object being converted is of
primitive type, you can only access it directly from the
stack with lua_toXXX() calls.

>
> >> We need separate registries within Boost.Python too; we
> >> just don't have them, yet.  There's also a potential
> >> issue with thread safety if you have modules using the
> >> same registry initializing concurrently.  With a single
> >> Python interpreter state, it's not an issue, since
> >> extension module code is always entered on the main
> >> thread until a mutex is explicitly released.  Anyway, I
> >> want to discuss the whole issue of registry isolation
> in
> >> the larger context of what's desirable for both systems
> >> and their evolution into the future.
> >
> > Right. For luabind it seems reasonable to accept a
> single
> > registry for every module, and perhaps global registry
> > used by interacting modules as well.
> >
> > It doesn't seem that interesting to register different
> > conversions for different states anymore. (at least not
> to
> > me, but I could be wrong..). But if we where to increase
> > the isolation of the registries, each state could just
> as
> > well get their own registry.
>
> Let's continue poking at the issues until we get clarity.
>

Yeah, I'll have to think about this for a bit, the whole
registry thing is quite new to me.

--
Daniel Wallin