[PYTHON C++-SIG] LLNL Python/C++ integration: current status

[Geoffrey Furnish]

Donald Beaudry writes:
 > "Barry A. Warsaw" <bwarsaw at CNRI.Reston.Va.US> wrote,
 > > Could you give me a quick primer on what the "traits" technique is?
 > > This is definitely something new to me.
 > 
 > Check out <http://www.cantrip.org/traits.html>, but I'm real
 > interested in hearing about how this is going to get applied to the
 > C++/Python issue.

Here's a quick example.  The following is the Foo::crash method I
presented yesterday, augmented to pull out a couple of arguments from
the args tuple.

PyObject *Foo::crash( PyArgs *args )
{
    cout << "In Foo::crash\n" << flush;

    {
	int x, y;
	args->ParseTuple( x, y );
	cout << "x=" << x << " y=" << y << endl;
    }

    throw "flaming oblivion";

    Py_INCREF( Py_None );
    return Py_None;
}


Notice several things:

1) No &'s !!!
2) No format string
3) No explicit error checking.

Here are some possible ways one might call this thing from python.  If
you have the patience to study this closely, it will all make sense:

mousey[34] ../python 
>>> import cxxtst
>>> f = cxxtst.new_Foo()
>>> f.crash( 2 )
In Foo::crash
Traceback (innermost last):
  File "<stdin>", line 1, in ?
TypeError: function requires exactly 2 arguments; 1 given
>>> f.crash( 2, 3 )
In Foo::crash
x=2 y=3
Traceback (innermost last):
  File "<stdin>", line 1, in ?
RuntimeError: a C++ exception has occurred: flaming oblivion
>>> f.crash( 2, 3, 4 )
In Foo::crash
Traceback (innermost last):
  File "<stdin>", line 1, in ?
TypeError: function requires exactly 2 arguments; 3 given
>>> f.crash( 2., "hello" )
In Foo::crash
Traceback (innermost last):
  File "<stdin>", line 1, in ?
TypeError: illegal argument type for built-in operation
>>> f.crash( 2., 3. )
In Foo::crash
x=2 y=3
Traceback (innermost last):
  File "<stdin>", line 1, in ?
RuntimeError: a C++ exception has occurred: flaming oblivion
>>> f.crash( 2.4, -6.6 )
In Foo::crash
x=2 y=-6
Traceback (innermost last):
  File "<stdin>", line 1, in ?
RuntimeError: a C++ exception has occurred: flaming oblivion
>>> f.crash( "xxx", 2 )
In Foo::crash
Traceback (innermost last):
  File "<stdin>", line 1, in ?
TypeError: illegal argument type for built-in operation
>>> 

So that seems like it does just what it should do, but is much less
arduous to code than what you would do in C.  Less arduous means I can
do more python extension work without reaching for the bottle of
Advil, and will be less likely to get anything wrong.  Abolishing the
format string is especially important.  What if Foo was really

template<class T>
class Foo { ... }

so that the "int x, y" in Foo::munge() above was really "T x, y;" ???
How would you handle that with format strings?  This is just a
rehashing of the same old argument which causes C++ to have an
iostreams library instead of using stdio.  I assume the import of this
argument is clear to anyone with basic familiarity with using
templates in C++.

Now, how do we implement this?  Here is the PyArgs class:

//----------------------------------*-C++-*----------------------------------//
// Copyright 1996 The Regents of the University of California. 
// All rights reserved.
//---------------------------------------------------------------------------//

#ifndef PyX_Args_hh
#define PyX_Args_hh

#include "Python.h"

#include "PyX_xcpt.hh"

#include <string>

template<class T>
class pytraits {
  public:
    typedef PyObject *holder_t;
    static const char *fmt() { return "O"; }
    static void post_config( const holder_t& h, T& t ) { t = h; }
};

template<> class pytraits<int> {
  public:
    typedef int holder_t;
    static const char *fmt() { return "i"; }
    static void post_config( const holder_t& h, int& t ) { t = h; }
};

template<> class pytraits<float> {
  public:
    typedef float holder_t;
    static const char *fmt() { return "f"; }
    static void post_config( const holder_t& h, float& t ) { t = h; }
};

template<> class pytraits<double> {
  public:
    typedef double holder_t;
    static const char *fmt() { return "d"; }
    static void post_config( const holder_t& h, double& t ) { t = h; }
};

//===========================================================================//
// class PyArgs

// The purpose of this class is to serve as the C++ replacement for the
// "PyObject *args" which is the conventional second argument to compiled
// extension functions. 
//===========================================================================//

class PyArgs : public PyObject {
  public:
    int ParseTuple()
    {
	int result = PyArg_ParseTuple( this, "" );
	if ( !result ) {
	    throw PyException();
	}
	return result;
    }

    template<class X1>
    int ParseTuple( X1& x1 )
    {
	pytraits<X1>::holder_t h1;

	std::string fmt = pytraits<X1>::fmt();

	int result = PyArg_ParseTuple( this,
				       const_cast<char *>(fmt.data()),
				       &h1 );
	if (!result) throw PyException();

	pytraits<X1>::post_config( h1, x1 );

	return result;
    }

    template<class X1, class X2>
    int ParseTuple( X1& x1, X2& x2 )
    {
	pytraits<X1>::holder_t h1;
	pytraits<X2>::holder_t h2;

	std::string fmt = pytraits<X1>::fmt();
	fmt += pytraits<X2>::fmt();

	int result = PyArg_ParseTuple( this,
				       const_cast<char *>(fmt.data()),
				       &h1, &h2 );
	if (!result) throw PyException();

	pytraits<X1>::post_config( h1, x1 );
	pytraits<X2>::post_config( h2, x2 );

	return result;
    }

    template<class X1, class X2, class X3>
    int ParseTuple( X1& x1, X2& x2, X3& x3 )
    {
	pytraits<X1>::holder_t h1;
	pytraits<X2>::holder_t h2;
	pytraits<X3>::holder_t h3;

	std::string fmt = pytraits<X1>::fmt();
	fmt += pytraits<X2>::fmt();
	fmt += pytraits<X3>::fmt();

	int result = PyArg_ParseTuple( this,
				       const_cast<char *>(fmt.data()),
				       &h1, &h2, &h3 );
	if (!result) throw PyException();

	pytraits<X1>::post_config( h1, x1 );
	pytraits<X2>::post_config( h2, x2 );
	pytraits<X3>::post_config( h3, x3 );

	return result;
    }
};

#endif				// PyX_Args_hh

//---------------------------------------------------------------------------//
//                              end of PyX_Args.hh
//---------------------------------------------------------------------------//

The plan is to have these member templates for each number of args
from 0 up through some reasonable number, maybe 10 or 20.  They are
obviously going to get a little gross to write, but so what, we just
put this grunge in the PyArgs class, and clients never have to think
about it.  Clients just say, "I need an int, a float, and a foobar out
of that argument list", and so they declare an int, a float and a
foobar, and ask the arg tuple to hand them over.  

A particularly nice thing about the "traits" technique, is that it is
trainable.  ParseTuple don't have no format code for your
UniverseSimulator class?  No problem, bury the conversion logic inside
a specialization, pytraits<UniverseSimulator>.  That is what the
post_config business is for.  You might need to pass an "O" for a
given argument, since ParseTuple doesn't have format codes for things
other than Python intrinsic types, and then you can use the
post_config() method to do something meaningful about additional
unpacking of the PyObject *.  Check that it really is a
UniverseSimulator, check that the universe hasn't died from heat
death, or been sucked into a black hole, whatever, and then build up
your C++ object from the PyObject *.  You would do this by hand over
and over without this, but with traits you just code up the conversion
semantics one time, and presto--you've trained ParseTuple to know how
to pick apart complex arg tuple strings with user defined extension
types.

-- 
Geoffrey Furnish		email: furnish at llnl.gov
LLNL X/ICF			phone: 510-424-4227	fax: 510-423-0925

_______________
C++-SIG - SIG for Development of a C++ Binding to Python

send messages to: c++-sig at python.org
administrivia to: c++-sig-request at python.org
_______________