[C++-sig] Re: Interest in luabind

[Daniel Wallin]

>
> I just love how GNUs is able to straighten out that nasty
> Outlook
> (Express) wrapping.  Unfortunately it looks like you're
> using
> webmail, or you could use OE-QuoteFix :(

I am using webmail, my home connection is down so I either
use this or configure mutt. :)

>
> "Daniel Wallin" <dalwan01 at student.umu.se> writes:
>
> >> In fact, the more I look at the syntax of luabind, the
> more I like.
> >> Using addition for policy accumulation is cool.  The
> naming of the
> >> policies is cool.
> >
> > It does increase compile times a bit though
>
> What, overloading '+'?  I don't think it's significant.

I meant composing typelist's with '+' opposed to composing
the typelist manually like in BPL.

> >>
> >> >> > This doesn't increase compile times.
> >> >>
> >> >> Good.  Virtual functions come with bloat of their
> own, but that's
> >> >> an implementation detail which can be mitigated.
> >> >
> >> > Right. The virtual functions isn't generated in the
> >> > template, so there is very little code generated.
> >>
> >> I don't see how that's possible, but I guess I'll
> learn.
> >
> > We can generate the wrapper code in the template, and
> store
> > function pointers in the object instead of generating a
> > virtual function which generates the wrapper functions.
>
> Well, IIUC, that means you have to treat def() the same
> when it
> appears inside a class [...] as when it's inside a module
> [ ... ],
> since there's no delayed evaluation.
>
>     ah, wait: you don't use [ ... ] for class, which gets
> you off
>     the hook.
>
>     but what about nested classes?  Consistency would
> dictate the
>     use of [ ... ].

Right, we don't have nested classes. We have thought about a
few solutions:

class_<A>("A")
  .def(..)
  [
    class_<inner>("inner")
      .def(..)
  ]
  .def(..)
  ;

Or reusing namespace_:

  class_<A>("A"),
  namespace_("A")
  [ class_<inner>(..) ]

We thought that nested classes is less common than nested
namespaces.

> >> The ordering issues basically have to do with the
> requirement that
> >> classes be wrapped and converters defined before they
> are used,
> >> syntactically speaking.  That caused all kinds of
> inconveniences in
> >> BPLv1 when interacting classes were wrapped.  OTOH I
> bet it's
> >> possible to implicltly choose conversion methods for
> classes which
> >> you haven't seen a wrapper for, so maybe that's less of
> a problem
> >> than I'm making it out to be.
> >
> > Ok. In BPLv1 you generated converter functions using
> friend
> > functions in templates though, and this was the cause
> for
> > these ordering issues?
>
> That was one factor.  The other factor of course was that
> each class
> which needed to be converted from Python used its own
> conversion
> function, where a generalized procedure for converting
> classes will do
> perfectly well.

Right. We have a general conversion function for all
user-defined types. More on this later.

>
> There is still an issue of to-python conversions for
> wrapped classes;
> different ones get generated depending on how the class is
> "held".
> I'm not convinced that dynamically generating the smart
> pointer
> conversions is needed, but conversions for virtual
> function
> dispatching subclass may be.

I don't understand how this has anything to do with
ordering. Unless you mean that you need to register the
types before executing python/lua code that uses them, which
seems pretty obvious. :)

>
> >> >> How will this work when multiple extension modules
> need to
> >> >> manipulate the same types?
> >> >
> >> > I don't know. I haven't given that much thought. Do
> you see
> >> > any obvious issues?
> >>
> >>
> >> Hmm, maybe I'm on drugs.  The biggest problems in BPLv1
> in this
> >> area were because the converters for a given class were
> generated,
> >> essentially, by its class_<...> instantiation.  But I
> have already
> >> said that from-python conversion for a given wrapped
> class is
> >> normally done statically.
> >>
> >> User-defined converters still need to be exposed to all
> the
> >> extensions which use the types, somehow.  It would be
> better not to
> >> replicate that code.
> >
> > I haven't thought about that at all. But it's a good
> point, and it's
> > impossible to not replicate the code with static
> dispatch.
>
> Right.  BPLv1 used a nonuniform system of explicit
> importing
> converters from other modules (thanks, Ralf!) but we went
> to uniform
> dynamic dispatching for v2.
>
> >> >> How do *add* a way to convert from Python type A to
> C++ type B
> >> >> without masking the existing conversion from Python
> type Y to C++
> >> >> type Z?
> >> >
> >> > I don't understand. How are B and Z related? Why
> would a
> >> > conversion function for B mask conversions to Z?
> >>
> >> Sorry, B==Z ;-)
> >
> > Ah, ok. Well, this isn't finished either. We have a
> > (unfinished) system which works like this:
> >
> > template<>
> > struct implicit_conversion<0, B> : from<A> {};
> > template<>
> > struct implicit_conversion<1, B> : from<Y> {};
> >
> > Of course, this has all the problems with static
> dispatch as
> > well..
>
> And with multiple implicit conversions being contributed
> by multiple
> people.  Also note that in many environments there's no
> guarantee that
> different extension modules won't share a link namespace,
> so you have
> to watch out for ODR problems.

Right. We didn't really intend for luabind to be used in
this way, but rather for binding closed modules. It seems to
me like this can't be very common thing to do though, at
least not with lua. I have very little insight in how python
is used.

>
> >> >> Me too; I just have serious doubts that once you
> factor in
> >> >> everything else that you want going on (e.g. derived
> <==> base
> >> >> conversions), the ability to dynamically register
> conversions has a
> >> >> significant cost.
> >> >
> >> > You might be right. I'll investigate how runtime
> dispatch
> >> > would affect luabind the next couple of days, in
> particular
> >> > I will look at what this would do to our policy
> system.
> >>
> >> OK.  Incidentally, I find much of what you're doing
> very appealing,
> >> and I think that if we could find a way to share a lot
> of
> >> technology it would be fantastic.
> >
> > I think so too. I'm looking around in BPL's conversion
> system now
> > trying to understand how I incorporate it in luabind.
>
> I am not convinced I got it 100% right.  You've forced me
> to think
> about the issues again in a new way.  It may be that the
> best answer
> blends our two approaches.

Your converter implementation with static ref's to the
registry entry is really clever. Instead of doing this we
have general converters which is used to convert all
user-defined types. To do this we need a map<..> lookup to
find the appropriate converter and this really sucks.

As mentioned before, lua can have multiple states, so it
would be cool if the converters would be bound to the state
somehow. This would probably mean we would need to store a
hash table in the registry entries and hash the lua state
pointer (or something associated with the state) though, and
I don't know if there is sufficient need for the feature to
introduce this overhead.

I don't know if I understand the issues with multiple
extension modules. You register the converters in a map with
the typeinfo as key, but I don't understand how this could
ever work between dll's. Do you compare the typenames? If
so, this could never work between modules compiled with
different compilers. So it seems to me like this feature
can't be that useful, what am I missing?

Anyway, I find your converter system more appealing than
ours. There are some issues which need to be taken care of;
We choose best match, not first match, when trying different
overloads. This means we need to keep the storage for the
converter on the stack of a function that is unaware of the
converter size (at compile time). So we need to either have
a fixed size buffer on the stack, and hope it works, or
allocate the storage at runtime.

For clarification:

void dispatcher(..)
{
  *storage here*
  try all overloads
  call best overload
}

--
Daniel Wallin