boost::python, returning new PyObject references

[David Abrahams]

> > http://www.python.org/doc/current/api/abstract.html seems to indicate
> > otherwise.
>
> These protocols are just convenient groupings for sets of operations.
>
> > Also PyMapping_Check et. al, and such functions in python itself
> > as callable(), isinstance(), and issubclass(), though admittedly the
lack of
> > functions like issequence() or ismapping() seems to remove the emphasis
on
> > this idea.
>
> isinstance() and issubclass() don't prove anything -- they make tests
> on the class hierarchy available for those that find such tests
> convenient.  (Read their docs -- they take a second argument that's a
> class.)
>
> callable() is a test for a *single* operation, "function call".

I understand. Remember when we discussed generic programming and Concept
defintions a while back? A single, sufficiently important operation may be
all that's needed to capture a Concept. "Callable" has granularity
comparable to that of the Assignable and CopyConstructible concepts we use
in C++, so it is natural to think of this function as an introspection that
defines a Concept. Among the other concept models used in the C++ standard
library we also have 5 flavors of Iterators (input, output, forward,
bidirectional, random-access), Sequence, and Associative Container (c.f.
Mapping). Maybe now you can see why C++ programmers are likely to find a
strong analogy with what's going on in this are of the Python docs.

> The others are left out on purpose, except there are some remnants
> (e.g. in the operator module) where I couldn't curtail the enthusiasm
> of others. :-)
>
> > I guess there's no point in arguing with GVR about the nature of
> > Python.  FWIW, though, at least a few of us independently drew the
> > same conclusions about the existence and possible importance of
> > categories based on the documentation and the facilities provided by
> > the language.
>
> Maybe the docs need to be clarified.

Probably. Note also that Paul Dubois seems to have made similar inferences
from looking at the design of CXX. About 1/4 of the way down the page at
http://cxx.sourceforge.net/ there is a refinement hierarchy for the CXX
object wrappers.

> You cannot write a function that
> decides Sequence-ness of an arbitrary object, you can only write
> something that recognizes clear hits and clear misses correctly -- but
> there's a gray area where the proper approach is to test for the
> existence of a specific operation.  Fortunately, starting in 2.2, you
> can check whether e.g. __getitem__ exists and that will give you the
> right answer for classes as well as for built-in types -- although 3rd
> party extensions may still be incompletely introspectable.

Only in pure python, right? Extension writers can still look at the slots to
see what operations may be supported. A malicious author who fills in slots
with functions that always raise exceptions is doing something that doesn't
even play well with the native Python interpreter code anyway, IIRC (I know,
descriptors are an exception to this rule, but they're new).

> > > so there's no point in trying.  There's a
> > > set of possible operations and each object type can decide which
> > > subset of those it supports.  Certain subsets make the type smell more
> > > numeric, others make it smell like a sequence, but there's no absolute
> > > answer.  It's *possible* though unlikely to have a sequence that
> > > doesn't support len(), for example.
> >
> > I knew that; since the boundaries are so malleable I was trying to
> > capture your mental model. Maybe your mental model is actually "one
> > big soup of types with no structure, even informally". In that case,
> > we may have to rethink the idea of classification altogether.
>
> My mental model has some types that are clearly sequences, mappings,
> etc., and some that are clearly not, but no exact rule to always
> decide what something is.  This has changed; ten years ago, I *did*
> think these were recognizable categories, but the language and my
> thinking have evolved away from this.

In that case, I'll suggest again that you consider heading back the way you
came ;-). Some clear and officially sanctioned Concept definitions would be
very useful to writers of generic Python code. Note that while the ability
to introspect about Concepts is nice, it's not the most important reason to
define them**. The reason  you want concepts is so that writers of generic
code can specify the contract with their clients.

It isn't a disaster if you pick the "wrong" definition, e.g. for Sequence.
If you decide that a Sequence implements __len__, and someone wants to write
a generic function which depends on the rest of the Sequence requirements
but not __len__, she has two choices:

1. Ignore this difference and document that their function requires a
Sequence parameter
2. Make up her own Concept definition which precisely describes the
requirements imposed by her function

Of course, it's much better if you can try to delineate all of the useful
gradations in a particular domain at once, so that the community doesn't end
up with lots of different names for the same concept. You might provide
Sequence and MeasurableSequence, for example.

In any case, it seems as though Arnaldur and I have a choice. We can:

1. Ignore the possibility of Concepts altogether and implement a single
generalized object interface with all capabilities.

2. Acknowledge that Concepts would be useful, but the ones that the Python
docs seem to imply are not really intentional. They seem to have the right
names, but we'd need to change them and probably carve them up a bit, and
that would only cause confusion, so again we throw up our hands.

3. Work out a set of concept definitions that make sense, and carefully
document them so that people are aware that "Sequence" might mean something
different when we say it than when the Python docs say it. Then implement
C++ classes to represent those concepts.

I'm in favor of #3, but open to others.
Opinions, anyone?

-Dave

**When Concepts were first applied in C++ nobody knew how to introspect
about them other than to let the compiler bark when generic code uses an
unimplemented operation. We now know how to check a very few requirements
without causing errors when they're not satisfied, and we can often generate
more coherent error messages than what you'd get otherwise in the other
cases, but the use of Concepts remains focused on specification, not
introspection.