[C++-SIG] Restarting

[Geoffrey Furnish]

Hello all,

I'd like to revive this sig.  I know I've been quiet of late, but hey, 
so has everyone else.

In the interest of restarting the discussion, I guess I'll try to
restate what I think we need to accomplish here.  But I represent it
only as my opinion.  I hope others will comment, or suggest wholly
different agendas, whatever.  Anyway, this post is just intended to
kick start renewed discussion of what the goals should be and the
means for achieving them.

First off, "THE GOAL", in my mind, is to come up with a set of core
enabling code that would ultimately be directly absorbed by Guido into 
the Python distribution.  When it's all said and done, I want Python
extension programmers to be able to pull down a stock distribution,
and using that, write extension modules in C++ with the full
conveniences accorded by the C++ language.  A lot of the action for
Python, in the circles I travel, is in the business of hooking Python
up to other code.  I frequently find myself writing this "glue code"
that we call Python compiled extensions, which talks to Python's C api 
in a pretty low level C-ish way, and talks to  other compiled
assets, using fairly upscale C++.  I'd like the code which lives in
this space, to be able to talk to the Python API in a similarly
upscale fashion.

I've discussed this with Guido a few times over the last two years.
He's a little concerned about the idea of absorbing too much code that 
he doesn't understand, into the core, because of the support issues.
And he wants to see a serious consensus emerge on the SIG.  [Guido,
please feel free to further clarify here].

Both of these seem like very reasonable issues to me, but I'm
optimistic that both can be satisfactorily redressed.  Regarding the
first point, I figure that much of the C++ support machinery that I
envision being developed in this SIG, is pretty lightweight stuff.
For example, the CXX_Objects package that Paul has been distributing,
is a pretty thin veneer over the Python C API, which happens to really 
make life a lot easier on the Python extension programmer working in
C++, without really representing all that much code.  I think that is
representative of how most of the work of this SIG can come out.
Moreover, I think there's a pretty good set of dividing lines between
some of the different subsystems, so that we should be able to
advocate some of them to Guido for inclusion into the core Python
distribution, without necessarily having to couch it as an
all-or-nothing kind of arrangement.  As for consensus, well, everybody 
speak up!

In the sig charter, I proposed the following list of example topics
for this sig.  I'll just quote that list, and insert some comments
betwinst. 

 > 1.Autoconf support for enabling C++ support in Python. This must be
 > managed in a way which does not change the C API, or the behavior of
 > Python if not configured for C++ support, in any (observable?) way. In
 > other words, conventional C extension modules must continue to work,
 > whether Python is configured with C++ support or not.

I've recently produced a new patch set for fixing the Python-1.5.2
configure script to support C++.  Yell loud if you don't agree, but
I'm under the impression that there is a pretty universal awareness of 
the basic issue that we need main() compiled in C++ in order to
support the possibility of the python executable hosting other C++
code buried in extension modules.  This much we need, and I anticipate we
can all agre on, almost irrespective of exactly /what/ the specific
C++ interface we come up with is.  Or stated another way, people
wanting to write Python extension modules in C++ will need this, no
matter exactly what they do for the rest of the C++ binding.

I will put the latest version of this patch up on the C++ Sig web site 
soon.  I have a little learning to do about how to administer the www
site for this sig, but as soon as I can, I'll post a patch against the 
1.5.2 distribution which adds autoconf support for C++ enabled python
configurations. 

 > 2.Type safety in the Python API. 

There's a lot here, but one of the most basic things I was thinking
about when I wrote that, was this business of parsing tuples,
especially arg tuples.  Right now you do this with a format string,
and you'd better do it right!  What I'd like to see instead, is a
system that uses C++'s innate awareness of the types of identifiers,
to directly construct those format strings on your behalf.  There are
probably similar issues involved with extracting elements from the
various Python containers.

 > 3.Introducing C++ classes for the various major Python abstractions,
 > such as dictionaries, lists, tuples, modules, etc. 

Paul and I spent some real time talking over this when I was at LLNL,
and his CXX_Objects package is a really good incarnation of the basic
ideas I think are important here.  I've got to get reacquainted with
the Python C aPI and see how much coverage we have right now.  A
pretty basic way of saying what classes I figure we need, is to look
at the list of all Python C API entry points.  They all look sort of
like: Py<Pkg>_<Operation>.  To me, this would seem to translate
naturally to:

namespace Py {
    class <Pkg> {
        PyObject *<Operation>( ... );

The idea then is to come up with C++ classes representing the major
subdivisions of the Python C API, with member functions corresponding
to the various function entry points in this package family in the C
API.  That's at least a pretty decent first approximation to what
should be done.  A little additional work to bring [] subscripting
operators to Dicts and so forth, seems prudent as well.

So anyway, I think Paul's CXX_Objects thing looks to me like a pretty
significant step along the path toward "wrapping" the Python C API.

 > 4.Providing semantics for these classes which are natural both to the
 > Python programmer, as well as the C++ programmer. For example, the
 > PyDict class should have an operator[], and should also support STL
 > style iterators. Many issues of this form.

Here I was trying to say that the wrapper classes implemented above,
should be good STL citizens.  However, that was only loosely defined
at the time that I wrote it.  Now I would point to the recently
published "Generic Programming and the STL" by Matt Austern, which
contains comprehensive documentation of the services exported by the
STL components.  We should try to make our classes that wrap the
Python C API, support the services of similarly named STL classes as
much as possible.

There may be some issues here of just how far we can go with this.
Python is very dynamically typed, and C++ is statically typed.  For
instance, a PyDict holds PyObject *'s, not int's.  So there may be
some issues about just how "normal" we can make the Python C++ wrapper 
classes look, in comparison to their STL counterparts, but we should
do what we can.

In particular, I think it ought to be possible to to at least
implement enough services to support a wide sampling of the STL
algorithms, operating over the elements of the Python containers.  For 
example, maybe we need to write a functor library so you can write
something like:

Py::Dict a;
// Fill a with elements, either from C++ or from Python script
// language.
sort( a.begin(), a.end(), PyLexicographicalLess<Py::String>() );

Or whatever.  Anyway, hopefully you can understand the basic idea I'm
trying to convey. 

 > 5.Method invocation. How to make the invocation of C++ member
 > functions as natural and/or painless as possible.

I have code that makes it possible to invoke methods of an object in
the Python language, and have it result in invoking methods of an
underlying C++ object.  I'll dust that code off and try to post it
soon.  

There were some comments on this code about a year ago, and I tried to 
fold in some of the constructive criticisms that I received, when last 
I rewrote that.  

Anyway, I'll get that cleaned up, and post it to the www site so
poeople can look it over.

 > 6.Error handling. How to irradicate NULL checking in favor of C++
 > style exceptions.

I really want to expunge this business of NULL checking of the results
of Python C API calls.  The wrapper classes envisioned above, should
do that, and throw appropriate exceptions when something goes gonzo in
the C aPI functions they call.  Then in the C++ code which uses these
wrapper classes, you could replace all blecherous NULL checking, with
a conventional C++ try block.  Or even just let it go unhandled, if
you have been suitably rigorous in the use of ctor/dtor pairs.

For this to work, there needs to be a try/catch block around this
methodobject invoker thingie in the Python core.  Again, I have code
that implements an "exception safe" method invocation semantic.  I'll
repackage that for 1.5.2, and post it for comment.



In any event, I hope people will comment on this, and voice any other
views or raise any other issues that ought to be heard and considered
by all.  Also, if you have code which you think should be considered
by the larger community in the context of this effort to codify a C++
interface for Python extension programming, then by all means, send it 
to me, and I'll try to post it on the SIG www page.  We should get all 
the ideas and competing propositions out on the table where every one
can see, review, comment, contribute.

One last thing for this initial attempt at revitalizing this SIG.  In
the past, there has been a lot of talk about the dearth of compilers
at the level anticipated by the code that I and Paul have posted.
There are some developments on this front that I think are worthing
bringing to people's attention:

1) Sun C++ 5.0 is out.  Although still a far cry short of "ANSI C++",
it is a huge step up over their previous 4.2 product.  They left out
partial specialization and member templates, which is extremely
dissapointing, but still, the overall product is a huge step up over
its direct predecessor.

2) MS VC++ 6.0 is reputed to have support for "inline member
templates" and most of the rest of the modern template machinery.
Evidently it still lacks out of line member templtes, and partial
specialization, but--so I am told--on the whole it is a big step
forward.  

3) For those who may not have heard yet, EGCS has been named the
official successor to the GCC throne.  This is extremely good news for 
the C++ community.  I've been working with developer snapshots of egcs 
for several months now.  10/5/98 was the snapshot that marked the
viability of egcs in my book.  I'm pretty sure that current egcs
snapshots will successfully handle code at the level of what Paul and
I have put up on this list before.  I'll be running precise
evaluations of this over the next few days, but I'm pretty sure late
model egcs snapshots are "there".  Expect a GCC 3.0 shortly, which
will be bascially a shaken down release of the current egcs developers 
snapshots. In other words, we are probably just days or weeks away
from having an extremely high quality freeware virtually-ISO C++
compiler. 

Cheers to all.  Please express your views..

-- 
Geoffrey Furnish            Actel Corporation        furnish at actel.com
Senior Staff Engineer      955 East Arques Ave       voice: 408-522-7528
Placement & Routing     Sunnyvale, CA   94086-4533   fax:   408-328-2303

"... because only those who write the code truly control the project."
						      -- Jamie Zawinski