[C++-sig] register_boost_array_from_python

[David Abrahams]

From: "Ralf W. Grosse-Kunstleve" <rwgk at yahoo.com>


> > callback_from_python takes possession of its argument just like
reference<>
> > does. It looks like you're dropping a reference in the transfer from
the
> > tuple.
>
> callback_from_python does not accept increment_count like reference<>
> does. Are you suggesting that I use Py_INCREF directly?

That's one solution.

I think you can see now why I didn't just slap the name "from_python<>" on
callback_from_python and expose it to users...

> > How about a STL-container_from_something-which-supports-iter()
converter?
> > That's what people really need.
>
> Sounds great. As it stands (new code below) I am iterating over the
> sequence twice: first in convertible() and then again in construct().
> Do all Python iterators support this?

I don't think so; an iterator could work like a C++ input iterator, and
there's no way to introspect about it to find out which category you
have...

...well, that's not strictly true. If the type supports __getitem__ then
you're probably getting a random-access iterator.

> Hm, if data->convertible is not 0, will construct() be called without
> further preconditions?

The conversion function will only be called if /all/ the converters needed
to handle the function's arguments report non-zero from their convertible()
function... i.e. only if overload resolution succeeds.

> If so, I could manage my own memory where I
> store the result of iterating over the input sequence.

I'm not sure that I am providing a way for you to manage any extra memory.
The only destructor actions in the rvalue_from_python converter are in
converter::from_python_data::~from_python_data().

> Can I then
> simply assign ownership in construct() (to avoid a copy)?

I don't understand what you mean here, sorry.

> Or, would it make sense to add, say, rvalue_stage0_data* to the
> signature of convertible()?

We could extend rvalue_data to contain an additional pointer to an abstract
base class which can be used to store extra dynamically-allocated data
during conversion, and which is deleted in the destructor. However, that
would impose some overhead in each wrapped function to delete this extra
data for each argument. I'm not sure that it's worth it, since the extra
data is seldom needed. I want to keep the automatically-generated function
wrappers as lean-and-mean as possible.

However, I'm not dead-set against it. If you want to try this tack, I'd
like to see some measurements (of the optimized result) showing that the
time/space overhead is negligible when the facility isn't actually used.

> About the code below: there are now three array adaptors that I have
> stress-tested (so far Linux/gcc only) with boost::array, std::vector and
> the "small" type of my array family which is similar to the
> "fixed_capacity_vector" proposed a while ago by Synge Todo.
> Example usage:
>
>   bpl_utils::register_array_from_python_list_or_tuple<
>     boost::array<double, 3>,
>     bpl_utils::fixed_size_array_registration_adaptor>();
>
>   bpl_utils::register_array_from_python_list_or_tuple<
>     std::vector<double>,
>     bpl_utils::variable_size_array_registration_adaptor>();
>
>   bpl_utils::register_array_from_python_list_or_tuple<
>     af::small<double, 6>,
>     bpl_utils::fixed_capacity_array_registration_adaptor>();

Nice! those last template arguments are more like Policies than Adaptors,
though. I like the general approach.

> If you are interested I'd be happy to contribute this +
> improvements/fixes that you might suggest + documentation to the core
> library.

Of course I am!

My suggestions:

1. Make it more generic: the names shouldn't mention arrays, since it
should apply to std::list also. Either Container or Sequence might be the
right name. Use the presence of __iter__ and/or __getitem__ to determine
elegibility of the actual argument.

2. The one for variable-sized containers should construct the thing
in-place using a C++ iterator range when used with a conforming compiler.

3. Efficiency: don't build an intermediate tuple when given a list; Just
use the Python iterator

4. Consider whether it's possible to factor out any part of this code into
bpl.dll, to avoid instantiating it in every extension module.

Nice work,

Dave