[C++-sig] Boost.Python v2: char const*, char, and array conversions

[David Abrahams]

Because of the importance of char const* as a representation for C-style
strings, we currently have a built-in lvalue converter from Python strings
to char const*. Reviewing Boost.Python's standard conversion rules at
www.boost.org/libs/python/doc/v2/feb2002.html#conversion, it's easy to see
that to get a char const* or char const* const&, we need to find an lvalue
converter for char. However, there are a few messy consequences:

1. a function which accepts char* (non-const) can accept arguments
   which are immutable Python strings. The C++ code can then modify
   the contents of the Python string.

2. a function which accepts char or char const& can also accept
   arguments which are Python strings. That's fine as far as it
   goes; we /do/ have a char lvalue there. However, there's no
   checking for value truncation. In other words, if I wrap:

      char g(char x) { return x; }

   I can call it thus from Python:

      >>> g('foobar')
      'f'

   This is not in keeping with the behavior of conversion to
   integral types, for example, which causes an exception when
   a floating point source value overflows.

The first problem above makes it clear to me that my prior conception that
"Python doesn't understand const-ness" was just wrong. What it doesn't
understand is the notion of a const reference to a non-const object: either
an object is mutable or it isn't. To solve it I propose to create the
notion of a "const lvalue converter". The new from_python table would then
look like:

Argument Type             Eligible Converters
-------------             -------------------
T                         T rvalue, T [const] lvalue
T const
T volatile
T const volatile
T const&

T const*                  T [const] lvalue
T const volatile*
T const volatile&
T const* const&
T const volatile*const&

T*                        T non-const lvalue
T volatile*
T&
T volatile&
T* const&
T volatile*const&

[const] means both const and non-const above

I'm not sure what the best approach for problem #2 is. The issue here is
that char is unlike other types: the availability of a char lvalue (at
least an immutable one) usually means there's an array, and further that
the length of the array can be determined by looking at its elements! One
possibility would be to special-case the rule for char, so that when the
C++ argument doesn't require an lvalue we explicitly check to see that it
is not a Python string whose length != 1.

However, the above approach is specific to the combination C++ char/Python
string. It wouldn't even work with a user-defined type containing
null-terminated char* strings. Thinking a little further, it could be
generalized to add a new kind of lvalue converter: the sequence converter.
A sequence converter would have to be able to report the end of the
sequence. A T sequence converter would only be eligible for matching a T
argument from the first group if the sequence's length were 1. That would
prevent the 'foobar'->'f' truncation above in a more general way. It could
also form the basis of a future feature which allows us to wrap functions
accepting iterator ranges and (iterator,length) pairs by converting a
single Python sequence argument into multiple C++ arguments. As I learned
to my utter "oh, cool!"-ness at the Python10 conference, SWIG supports this
sort of thing.

So then the table becomes:

Argument Type             Eligible Converters
-------------             -------------------
T                         T rvalue, T [const] lvalue, T [const] seq(1)
T const
T volatile
T const volatile
T const&

T const*                  T [const] lvalue, T [const] seq
T const volatile*
T const volatile&
T const* const&
T const volatile*const&

T volatile*               T non-const lvalue, T non-const seq
T&
T volatile&
T* const&
T volatile*const&


Thoughts?

-Dave


+---------------------------------------------------------------+
                  David Abrahams
      C++ Booster (http://www.boost.org)               O__  ==
      Pythonista (http://www.python.org)              c/ /'_ ==
  resume: http://users.rcn.com/abrahams/resume.html  (*) \(*) ==
          email: david.abrahams at rcn.com
+---------------------------------------------------------------+