[C++-SIG] Python calling C++ issues

[Geoffrey Furnish]

I for one am losing track of the argument here.  Everything you say
below about the mechanism that Python uses for finding and invoking
functions of modules, is well known, and if you didn't read it
anywhere first, can be directly observed from the Python C source.
The part of your post which relates to a factual rendition of how
things works in CPython today, is not in contention.

Your claim that this can all be handled with a single base class
is--until you demonstrate it by posting ocde which works as you
posit--at best understood to be an hypothesis.  Several of us on this
list, including myself, Paul, Barry, and doubtless others, have worked 
out schemes largely similar to what you describe.  What has not yet
been done, is to field an implementation that both has the desired
features, works with exactly one base class for all extensions, and is 
correct according to the C++ language standard.  I have pursued a
slightly different approach which is correct according to the language 
standard, but uses templates to fabricate on demand, much of the
machinery which is neded to make all the funciton invocations.  I am
sorry that I have not yet been able to get this fully dusted off and
whipped into presentable form.  Still working on that.  Meanwhile,
Paul is pursing a system which does not have that characteristic, but
instead works more like what you describe.  But he is also not done,
and still working out final details.

I invite you to develop and post code which works as you claim can be
done.  We would all love to see a tangible incarnation of the design
ideas you espouse.  In other words, show us the beef.

skaller writes:
 > Geoffrey Furnish wrote:
 > 
 > > Okay, great.  So we can invoke the virtual function len() on an
 > > extension object of unknown type.
 > 
 > 	yes.
 >  
 > > Now what's the plan for invoking non virtual methods of C++ objects
 > > without knowing their type?
 > 
 > 	That is impossible. It is not necessary to be able
 > to do this, since the Python parser cannot generate any
 > code which calls it. Bytecode can only call 'virtual functions'
 > on PyObjects.
 > 
 > 	Perhaps, you mean, how do we do this:
 > 
 > 	object.method(arguments)
 > 
 > And the answer is: the same way python does. It is the only way.
 > Python calls 'getattr' to fetch 'method' by name.
 > The resulting function fetched has a fixed calling sequence.
 > Otherwise the bytecode interpreter could never call it.
 > It accepts a pair of arguments, a tuple and a dictionary.
 > 
 > 	[Actually, there is a complication with an obsoleted 
 > flag, determining if **kwds is support -- but lets ignore that for
 > clarity].
 > 
 > 	In other words, it is impossible to call ANYTHING
 > from python OTHER than one of the virtual methods.
 > And so that is all that needs to be provided.
 > 
 > 	Now, this is not to say that C++ cannot
 > call other C++ method directly. But you have to understand
 > that a C++ method like
 > 
 > 	void method (int x, float y) const
 > 
 > cannot ever be called by Python. Here's the sequence of events:
 > 
 > 	1) Python calls the C 'getattr' method of the
 > single, fixed C++ wrapper type: this is a C function.
 > 
 > 	2) The C function casts the object pointer
 > to the abstract base type of the C++ heirarchy.
 > 
 > 	3) Then, it calls the C++ virtual getattr method,
 > after suitably rearranging the arguments.
 > 
 > 	4) The C++ getattr method has to return something
 > that can be converted into a function which Python can call;
 > that is, it has to return a C function which takes a tuple
 > and dictionary as an argument.
 > 
 > 	It finds this function in a table. Because C is a bit
 > brain dead, there is no way to create a closure here. 
 > [This is possible using Viper, since the implementation
 > language, ocaml, supports them: generic extensions in Viper
 > are much easier than in C or C++ [but binding existing
 > C libraries is, conversely, harder]
 > 
 > 	The C function takes the first element of the tuple
 > argument, which is the object, and casts it to the CONCRETE
 > type begin defined, and calls the corresponding C++ method,
 > munging arguments as required. [Smly, the return value
 > of the C++ method has to be converted to a PyObject for the C
 > function to return]
 > 
 > 	There are many ways to do this; using SWIG is one;
 > but it is IRRELEVANT. You can write it by hand for example.
 > The point is that only ONE abstract C++ class is required
 > as a base, and only ONE Python type object is required to
 > wrap it, and this WILL handle every single C++ class
 > suitably derived from the abstract base.
 > 
 > 	I hope this is clear: the lookup of specific
 > non-virtual method on a class is independent of the
 > _automatically_ polymorphic lookup of the various
 > python virtual methods, including getattr and,
 > as you mentioned, len.
 > 
 > 	It makes sense, to have a specific derived
 > class, in which the virtual getattr is overriden,
 > and uses a particular data structure and algorithm
 > to do attribute lookup; for example, for the kind of
 > table SWIG or other automatic generator generates.
 > 
 > 	If I were implementing a getattr method by
 > hand, I'd be tempted to us an STL map, myself,
 > since it is likely to MUCH faster than the standard
 > linear search typically used implementing
 > extension types.
 > 
 > 	Anyhow, the point is that there is nothing
 > new here, this is no different to C: finding attributes
 > of objects, including methods, has nothing to do with
 > the Python interface to the type object: it is universally
 > achieved by a type specific getattr method.
 > 
 > -- 
 > John Skaller, mailto:skaller at maxtal.com.au
 > 10/1 Toxteth Rd Glebe NSW 2037 Australia
 > homepage: http://www.maxtal.com.au/~skaller
 > voice: 61-2-9660-0850
 >