boost::python, returning new PyObject references

[David Abrahams]

----- Original Message -----
From: "Arnaldur Gylfason" <arnaldur at decode.is>
> David
>
> Thanks.
> I would very much like to discuss the design with you and contribute. I
> took a look at objects.hpp and it was a good thing
> you pointed it out to me. I had already started last night designing a few
> things that are already there.
> I am going to take a better look at it tonight and think about what I
would
> consider the best design for this hierarchy.
> It is maybe a better idea to have object hold a ref instead of inheriting
> from it. I suppose at least that it will be simpler.

I think it would be a misuse of inheritance to do otherwise.

> I'm going to think about it tonight though.
> I'm also going to delve into the boost::python source to understand it
all.

Have fun!

> Any hint about contents of each file would be greatly appreciated.
> I have mainly looked at reference.hpp, conversion.hpp, callback.hpp  (+
> objects.hpp now).
> I have glimpsed through some of the other files but not really read them
> and understood.

There's a lot there and much of it is messy. I hope that we'll be cleaning
all of this up and dividing it into smaller, more
understandable/maintainable parcels over the next few months.

> The hierarchy I had in mind is something like this:
>
>
> -- object (PyObject_*)
>        |

    This inheritance must be private, or object must provide
    a restricted interface.

>        |-- mapping (PyMapping_*)
>        |    |
>        |     -- dictionary (PyDict_*)
>        |
>        |-- sequence (PySequence_*)
>        |    |
>        |    |-- list (PyList_*)
>        |    |
>        |    |-- string (PyString_*)
>        |    |
>        |    |-- tuple (PyTuple_*)
>        |    |
>        |    |-- unicode (PyUnicode_*)
>        |    |
>        |     -- array

    I think that under Python2.2, Dict is both a sequence and
    a mapping! The upshot is that either everything inherits
    virtually from object, or we remove object as a base class
    altogether. I'm inclined toward the former, but could go
    either way.

>        |
>        |-- number (PyNumber_*)
>        |    |
>        |    |-- int (PyInt_*)
>        |    |
>        |    |-- long (PyLong_*)
>        |    |
>        |    |-- float (PyFloat_*)
>        |    |
>        |     -- complex (PyComplex_*)
>        |
>        |-- buffer<T> (PyBuffer_*)
>        |
>        |-- file (PyFile_*)
>        |
>         -- module (PyModule_*)

This looks really nice.

> After taking a look at objects.hpp I'll sketch some of those
> interfaces/classes (that are not already in objects.hpp)  and send you.
> The idea behind buffer<T> is to interprete the memory buffer as an array
of
> T. It would also compute it's begin and end in the constructor and
> provide a begin() and end(). This would be helpful while working with
> array. We could easily go back and forth between Python and C++ without
> copying memory and
> could use the STL algorithms and more.

That's terriffic. The "iterators" used in some of the objects which return
proxies should probably be rewritten in terms of iterator_adaptor. They
could be conforming input iterators, at least (possibly also output
iterators; I'm not sure yet -- for that we'd need a new iterator tag).

> Another thing, I was thinking about providing an interface to some of
these
> Python classes that model some of the concepts in STL and thus provide
> access to it.

What do you have in mind? Could you be more specific?

> Later, by inspecting the memory layout of some of the classes (or adding
> new ones) we could have sequence and mapping objects in Python that have
> efficient iterators in
> C++ that work on the same memory layout.

I don't get it yet; I'd love it if you'd say more.

> In the end the idea is to be able to use python as the development
> environment but get an efficient implementation almost for free using
> boost::python and later additions.
> (We´re talking about the future here of course).

Sounds fascinating, but I need to get a clearer picture of your intention.

Regards,
Dave