From ephi at adital.com  Wed Dec  1 19:37:13 1999
From: ephi at adital.com (Ephi Dror)
Date: Wed, 1 Dec 1999 10:37:13 -0800
Subject: [C++-SIG] Python calling C++ issues
References: <5865800428DFD21199020090274E8C9554B4D0@cisncdc.cissv.canon.com>
Message-ID: <028301bf3c2b$16ca9a20$b7e856ce@london>

Hi All,

Thanks a lot for all the support we got so far.

Unfortunately, as of this moment, we have not yet successfully run our
simple spam program.

It is partially worked but then we got the two unexplained strange phenomena
that as of now, nobody seems to understand why.
Also, even if we got those problems solved using compiler xyz version 123
who knows what other problems are waiting for us in the next corner.

It may work perfect on NT, Win98 , SUN, etc platforms but we need it to work
on  RedHat 6.1 using the  distributed Python 1.5.2 and  GNU as it come out
of the box when you install  Linux 6.1. For us, it just does not work.

I am personally very disappointed from all of this. I really truly wanted to
give python a chance. I got the impression by reading one of the Python
books back cover that what we want to do is common practice and should be  a
plug and play .
(http://www.amazon.com/exec/obidos/ts/book-reviews/1565921976/103-5409830-17
11020)
Reading the books give  the impression that Python and C++ were meant to
live in peace and be  haply  working with each other. Unfortunately, my
personal experience realty proved the opposite. Python may work nicely with
C code, but who needs that 30 days before the year 2000 !!, Programmers
today need at least integration with existing C++ code and very soon they
need integration with Java code.
Our example only had  the simplest C++ class in the word, and Python could
not even use it correctly. Is  just un believable to me.
It is possible that we made stupid mistakes in the code, it is also possible
that we are bad programmers. Who knows?. We did  not hide our code. In fact,
I attached it to my original email.

Does anyone know how much Python is really  used  on Redhat Linux with the
conjunction with C++, especially the direction of Python calling C++
functions?

We have not given up yet but we are close. I would love to hear from the
group that I am wrong. That I was just blind and simply did not get it.
That calling C++ class from Python is really a piece of cake after all and
especially on RedHat 6.1 Linux with standard  installation components.

Thank you all for helping us.

We hope to stay in the group and contribute in the future, but it all depend
on what next for us in terms of using Python/C++.

Ephi.
----- Original Message -----
From: Dror, Ephi <ephi.dror at cis.canon.com>
To: <mhammond at skippinet.com.au>; <c++-sig at python.org>
Sent: Wednesday, November 24, 1999 10:46 AM
Subject: RE: [C++-SIG] Python calling C++ issues


> Hi Mark,
>
> Thanks a lot again. We tried it but unfortunately it did not help in our
> case.
>
> Does anyone know where can we download C++ based debug version of Python
> 1.52 for Linux?
>
> Our next step is to use Python 1.52 which is compiled with C++ and
possibly
> a debug version and then try our little sample.
>
> Thanks a lot.
>
> Ephi.
>
> -----Original Message-----
> From: Mark Hammond [mailto:mhammond at skippinet.com.au]
> Sent: Wednesday, November 24, 1999 12:53 AM
> To: 'Ephi Dror'; c++-sig at python.org
> Subject: RE: [C++-SIG] Python calling C++ issues
>
>
> > Unfortunately, we don't seems to understand what do you mean
> > by "Object". Is
> > it something we need to define or is it already defined in a
> > h file that we
> > forgot to include. We use Linux/gcc environment.
>
> Sorry - I meant "PyObject"
>
> > Also, in our sample, we did not use virtual function
>
> Hmmm - OK - my advice could well be wrong.  Unless your C compiler
> always adds a vfptr.
>
> I have seen this problem (when using virtuals) before and the symptoms
> sound almost identical.  Try the code snippet I posted, and see if the
> debugger does show the "PyObject *" and the "Spam *" as different
> values, even though they are pointing to the same object.
>
> Otherwise Im afraid I have no idea...
>
> Mark.
>
>
> _______________________________________________
> C++-SIG maillist  -  C++-SIG at python.org
> http://www.python.org/mailman/listinfo/c++-sig
>
> _______________________________________________
> C++-SIG maillist  -  C++-SIG at python.org
> http://www.python.org/mailman/listinfo/c++-sig
>




From morse at harborcom.net  Wed Dec  1 13:31:38 1999
From: morse at harborcom.net (Kevin Dahlhausen)
Date: Wed, 1 Dec 1999 07:31:38 -0500
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: <028301bf3c2b$16ca9a20$b7e856ce@london>
Message-ID: <199912011231.HAA10120@python.org>

Ephi,

I'd suggest you take a look at SWIG - the simplified wrapper interface 
generator.   I've had great success in using it to wrap the C++ cross-platform 
GUI FLTK.    If you need static constructors to run, I think you can use the 
%init statment somehow, even with a C-based Python interpreter.   

Python is all people say it is.  The only thing I don't like about it compared 
to straight C++ is that it is not easy to provide users with a single run-time 
executable - but then they should have Python installed anyway! :)



> Reading the books give  the impression that Python and C++ were meant to
> live in peace and be  haply  working with each other. 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sr Software Engineer, Morse Controls
http://www.geocities.com/SiliconValley/Peaks/4673
"Do or do not.  There is no 'Try.' "  -Yoda




From a.eyre at optichrome.com  Wed Dec  1 14:12:05 1999
From: a.eyre at optichrome.com (Adrian Eyre)
Date: Wed, 1 Dec 1999 13:12:05 -0000
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: <028301bf3c2b$16ca9a20$b7e856ce@london>
Message-ID: <000201bf3bfd$aa22b830$3acbd9c2@peridot.optichrome.com>

> Unfortunately, as of this moment, we have not yet successfully run our
> simple spam program.

You give up too easily... :)

Assuming you haven't changed the source from the original post:

The problem lies in the differences between Python extension modules and
Python extension types. You have written an extension module, but are
returning the values as if it were a type.

Here's how to fix it:

1. The file c_spam.cpp is pointless, so get rid of it.

2. In ext_spam.cxx:

static PyObject*
ext_new_Spam(PyObject* self, PyObject* args)
{
	int a, b;

	if (!PyArg_ParseTuple(args, "ii", &a, &b))
		return NULL;

	Spam* spam = new_Spam(a, b);

	return Py_BuildValue("O", spam);
}

'Spam' is a C++ object, not a python object, and you can't just cast it
to a PyObject* and expect it to work. One way to do this is to pass back
a pointer to the object as a long:

static PyObject*
ext_new_Spam(PyObject* self, PyObject* args)
{
	int a, b;

	if (!PyArg_ParseTuple(args, "ii", &a, &b))
		return NULL;

	Spam* spam = new Spam(a, b);
	return Py_BuildValue("l", (long) spam); // Or return PyInt_FromLong((long)
spam);
}

The Python script now stores this as a Python integer object.

3. In all other functions in ext_spam.cxx:

e.g.

static PyObject*
ext_Spam_Display(PyObject* self, PyObject* args)
{

	Spam* spam;

	if (!PyArg_ParseTuple(args, "O", &spam))
		return NULL;

	Spam_Display(spam);

	return Py_BuildValue("");
}

Use:

static PyObject*
ext_Spam_Display(PyObject* self, PyObject* args)
{
	Spam* spam;

	if (!PyArg_ParseTuple(args, "l", &spam))
		return NULL;

	Spam_Display(spam);

	Py_INCREF(Py_None);	// Use these lines when you don't
	return Py_None;		// want to return a value
}

That should just about fix it.

A couple of other things:

o If you don't already have Mark Lutz's book "Programming Python",
  get yourself a copy, and have a look through the chapters on
  embedding and extending Python.

o If you want a better response, try posting to the main Python
  mailing list, not the C++ binding one. This list is more to
  do with the procedures for automating the process of creating
  Python objects which mirror C++ objects.

o There are ways to do this sort of thing in a much more groovy way.
  Judging by the test script:

#!/usr/bin/python

import c_spam

myspam=c_spam.new_Spam(8, 5)
c_spam.Spam_Display(myspam)
c_spam.delete_Spam(myspam)

print "python end"


...you are using it like a module, not a type. There are ways to get
Python to make your Spam object usable like this:

import c_spam
myspam = c_spam.Spam(8, 5)
myspam.Display()
del myspam

I don't really want to give too much away, then you'll never learn
how to do things yourself, but if you need any more assistance, don't
hesistate to mail back.

Good luck.

--------------------------------------------
Adrian Eyre <mailto:a.eyre at optichrome.com>
Optichrome Computer Solutions Ltd
Maybury Road, Woking, Surrey, GU21 5HX, UK
Tel: +44 1483 740 233  Fax: +44 1483 760 644
http://www.optichrome.com
--------------------------------------------




From dgrisby at uk.research.att.com  Wed Dec  1 14:55:44 1999
From: dgrisby at uk.research.att.com (Duncan Grisby)
Date: Wed, 01 Dec 1999 13:55:44 +0000
Subject: [C++-SIG] Python calling C++ issues 
In-Reply-To: Your message of "Wed, 01 Dec 1999 13:12:05 GMT."
             <000201bf3bfd$aa22b830$3acbd9c2@peridot.optichrome.com> 
Message-ID: <199912011355.NAA07751@pineapple.cam-orl.co.uk>

On Wednesday 1 December, "Adrian Eyre" wrote:

[...]

> 'Spam' is a C++ object, not a python object, and you can't just cast it
> to a PyObject* and expect it to work. One way to do this is to pass back
> a pointer to the object as a long:
> 
> static PyObject*
> ext_new_Spam(PyObject* self, PyObject* args)
> {
> 	int a, b;
> 
> 	if (!PyArg_ParseTuple(args, "ii", &a, &b))
> 		return NULL;
> 
> 	Spam* spam = new Spam(a, b);
> 	return Py_BuildValue("l", (long) spam); // Or return PyInt_FromLong((long)

I haven't particularly followed the discussion, but I just want to
point out that doing what you suggest is extremely dangerous. There is
no guarantee that longs are the same size as pointers. There is also
nothing to stop the Python code from modifying the integer, causing
the C code to dereference arbitrary memory locations. It shouldn't be
possible for Python code to cause core dumps. You should use
PyCObjects instead.

Cheers,

Duncan.

-- 
 -- Duncan Grisby  \  Research Engineer  --
  -- AT&T Laboratories Cambridge          --
   -- http://www.uk.research.att.com/~dpg1 --



From a.eyre at optichrome.com  Wed Dec  1 15:21:49 1999
From: a.eyre at optichrome.com (Adrian Eyre)
Date: Wed, 1 Dec 1999 14:21:49 -0000
Subject: [C++-SIG] Python calling C++ issues 
In-Reply-To: <199912011355.NAA07751@pineapple.cam-orl.co.uk>
Message-ID: <000901bf3c07$67979c10$3acbd9c2@peridot.optichrome.com>

> I haven't particularly followed the discussion, but I just want to
> point out that doing what you suggest is extremely dangerous. There is
> no guarantee that longs are the same size as pointers.

There's no guarantee of most types in C++, only that a long is at least
1 byte. It's up to the programmer to make their C++ code platform
independent by using #ifdefs and typedefs, but I thought that much was
obvious. Besides which, the criteria IIRC was to run in on a PC on Linux,
for which the stated code will work.

> There is also nothing to stop the Python code from modifying the integer,
> causing the C code to dereference arbitrary memory locations.

Also true. However, if providing the Python programmer is aware of how the
extension works, and then chooses to modify the pointer, or whatever, then
they Deserve To Lose (tm).

> It shouldn't be possible for Python code to cause core dumps. You should
> use PyCObjects instead.

It shouldn't, no, but I present you with the following code:

#!/usr/bin/env python

l = []
m = []
l.append(m)
m.append(l)
print l == m

It was just a reponse to get the code to work. I mentioned that there were
better ways of doing it. Although interestingly enough, code almost
identical
to what I posted appears in Mark Lutz's Programming Python book, as a valid
way of achieve an extension type/class.

--------------------------------------------
Adrian Eyre <mailto:a.eyre at optichrome.com>
Optichrome Computer Solutions Ltd
Maybury Road, Woking, Surrey, GU21 5HX, UK
Tel: +44 1483 740 233  Fax: +44 1483 760 644
http://www.optichrome.com
--------------------------------------------




From beazley at cs.uchicago.edu  Wed Dec  1 15:22:44 1999
From: beazley at cs.uchicago.edu (David Beazley)
Date: Wed, 1 Dec 1999 08:22:44 -0600 (CST)
Subject: [C++-SIG] Python calling C++ issues 
References: <000201bf3bfd$aa22b830$3acbd9c2@peridot.optichrome.com>
	<199912011355.NAA07751@pineapple.cam-orl.co.uk>
Message-ID: <199912011422.IAA11646@gargoyle.cs.uchicago.edu>

Duncan Grisby writes:
> 
> I haven't particularly followed the discussion, but I just want to
> point out that doing what you suggest is extremely dangerous. There is
> no guarantee that longs are the same size as pointers. 

With the notable exception of pointers to members, since when has a
long never been large enough to hold a pointer value?  Can you name a
specific instance where this doesn't work?  (or has the C++ standards
committee finally decided to break all pointers just for the hell of
it?)   SWIG does this internally and has never had any problems.

> There is also
> nothing to stop the Python code from modifying the integer, causing
> the C code to dereference arbitrary memory locations. It shouldn't be
> possible for Python code to cause core dumps. You should use
> PyCObjects instead.

Just because you use a CObject doesn't mean you're going to eliminate
core dumps.  Of course, I personally believe that you should be able
to modify the pointer value if you really want to--but that's just me :-).

Cheers,

Dave




From beazley at cs.uchicago.edu  Wed Dec  1 15:37:20 1999
From: beazley at cs.uchicago.edu (David Beazley)
Date: Wed, 1 Dec 1999 08:37:20 -0600 (CST)
Subject: [C++-SIG] Python calling C++ issues 
References: <199912011355.NAA07751@pineapple.cam-orl.co.uk>
	<000901bf3c07$67979c10$3acbd9c2@peridot.optichrome.com>
Message-ID: <199912011437.IAA11853@gargoyle.cs.uchicago.edu>

> There's no guarantee of most types in C++, only that a long is at least
> 1 byte. 


From ians at amc.com  Wed Dec  1 15:36:47 1999
From: ians at amc.com (Ian Searle)
Date: Wed, 01 Dec 1999 06:36:47 -0800
Subject: [C++-SIG] Python calling C++ issues
References: <5865800428DFD21199020090274E8C9554B4D0@cisncdc.cissv.canon.com> <028301bf3c2b$16ca9a20$b7e856ce@london>
Message-ID: <3845327F.897EAC6F@amc.com>

Ephi Dror wrote:
> 
> 
> Does anyone know how much Python is really  used  on Redhat Linux with the
> conjunction with C++, especially the direction of Python calling C++
> functions?

We are developing version 2 of a large application that will be centered
around Python with several large, complex C++ extensions.  The
application is targeted at embedded software developers and provides
visibility into their embedded target via source code instrumentation,
and analysis of the resulting "tag stream" as their target runs.  The
product is called CodeTEST (www.amc.com).  We have been shipping a
"reduced" version of this for a year now that allows customers to script
tests for source code coverage with great success.  The only reason I am
providing this information is to add some context, and hopefully, some
believability to the claims that will follow.  The only difference is
that we ship a python we build with the product, as we cannot depend
upon our customers to already have it installed.

We ship on Windows NT, Solaris, and soon Linux.  So, the answer to your
question is yes, we use Python to supply an interface to our C++
libraries. To date it works quite well on all platforms that we develop
on.  This is not to say there have been no difficulties.  But, they have
not been insurmountable, and the payoff for using Python is too great to
let a few annoyances impede progress.  Besides, to date most of our
problems have been centered around C++ itself.  Our C++ components make
heavy use of STL, and getting them to work with both VC++ and g++ can be
a challenge.  Mostly due to the inadequate STL shipped with VC++ 6.0.
 
> We have not given up yet but we are close. I would love to hear from the
> group that I am wrong. That I was just blind and simply did not get it.
> That calling C++ class from Python is really a piece of cake after all and
> especially on RedHat 6.1 Linux with standard  installation components.

I believe you are wrong. Calling a C++ class is not too difficult.  I
will take an extension I wrote recently, and pair it down to something
very simple.  I think the thing you missed in your simple example is not
creating a new type.  Trying to pass C++ object pointers around as other
things is not the best road to travel in my opinion.  If you extend
Python with a new type, you can do type checking on the self argument,
and make your extension more robust in the presence of other
programmers.

I will mail the sample extension to this list in a few hours.

> 
> Thank you all for helping us.
> 
> We hope to stay in the group and contribute in the future, but it all depend
> on what next for us in terms of using Python/C++.
> 
> Ephi.

-Ian Searle
 Engineering Manager
 Software Analysis Tools
 Applied Microsystems Corporation
 http://www.amc.com



From a.eyre at optichrome.com  Wed Dec  1 15:43:58 1999
From: a.eyre at optichrome.com (Adrian Eyre)
Date: Wed, 1 Dec 1999 14:43:58 -0000
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: <199912011437.IAA11853@gargoyle.cs.uchicago.edu>
Message-ID: <000a01bf3c0a$80313030$3acbd9c2@peridot.optichrome.com>

> From K & R, 2nd ed., 2:2, verse 8, p. 36:
> 
> "Each compiler is free to choose appropriate sizes for its own
> hardware, subject only to the restriction that shorts and ints are at
> least 16 bits, longs are at least 32 bits, and short is no longer than
> int, which is no longer than long."

That's rather annoying. What if my processor only has 1-bit buses, and my
RAM only stores 10 words. I could never have any integers. :(

--------------------------------------------
Adrian Eyre <mailto:a.eyre at optichrome.com>
Optichrome Computer Solutions Ltd
Maybury Road, Woking, Surrey, GU21 5HX, UK
Tel: +44 1483 740 233  Fax: +44 1483 760 644
http://www.optichrome.com 
--------------------------------------------




From beazley at cs.uchicago.edu  Wed Dec  1 15:52:42 1999
From: beazley at cs.uchicago.edu (David Beazley)
Date: Wed, 1 Dec 1999 08:52:42 -0600 (CST)
Subject: [C++-SIG] Python calling C++ issues
References: <199912011437.IAA11853@gargoyle.cs.uchicago.edu>
	<000a01bf3c0a$80313030$3acbd9c2@peridot.optichrome.com>
Message-ID: <199912011452.IAA12074@gargoyle.cs.uchicago.edu>

Adrian Eyre writes:
> 
> That's rather annoying. What if my processor only has 1-bit buses, and my
> RAM only stores 10 words. I could never have any integers. :(
> 

What, you mean like if you're working on air-traffic control?   :-).

-- Dave





From skaller at maxtal.com.au  Wed Dec  1 16:21:40 1999
From: skaller at maxtal.com.au (skaller)
Date: Thu, 02 Dec 1999 02:21:40 +1100
Subject: [C++-SIG] Python calling C++ issues
References: <199912011355.NAA07751@pineapple.cam-orl.co.uk>
		<000901bf3c07$67979c10$3acbd9c2@peridot.optichrome.com> <199912011437.IAA11853@gargoyle.cs.uchicago.edu>
Message-ID: <38453D04.6B02D3A7@maxtal.com.au>

David Beazley wrote:
> 
> > There's no guarantee of most types in C++, only that a long is at least
> > 1 byte.
> 
> >From K & R, 2nd ed., 2:2, verse 8, p. 36:
> 
> "Each compiler is free to choose appropriate sizes for its own
> hardware, subject only to the restriction that shorts and ints are at
> least 16 bits, longs are at least 32 bits, and short is no longer than
> int, which is no longer than long."

Those limits do NOT apply to C++: there is NO requirement on the
number of bits, except that it be documented by the vendor.
[The comparative length requirements apply]

The conversion of pointers to ints is not supported in C++:
more precisely, the cast MAY be implemented by the vendor,
if so, the vendor must document it, and it MUST hold
the whole pointer so that:

	(void*)(long)(void*) p == p

For those using C Python sources with C++, there are two
gotchas.

	1)  Enumerations are NOT integers in C++
		[Luckily, Guido doesn't use them much]

	2) The very core of Python contains a serious
		breach of C++ rules: it casts functions
		pointers, which is NOT allowed AT ALL.

[I'm responsible for that rule!]

Luckily, this actually works on most C++ compilers.
To Guido's credit, it is the only breach of strict
ISO C++ Standards compliance I'm aware of.

[In fact, there is a way around it .. but Guido has
chosen not to use that technique]

If I can give a summary: the Python core language source
code is in EXCELLENT shape as a C++ program.

-- 
John Skaller, mailto:skaller at maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
voice: 61-2-9660-0850



From robind at earthling.net  Wed Dec  1 17:38:56 1999
From: robind at earthling.net (Robin Dunn)
Date: Wed, 1 Dec 1999 08:38:56 -0800
Subject: [C++-SIG] Python calling C++ issues
References: <5865800428DFD21199020090274E8C9554B4D0@cisncdc.cissv.canon.com> <028301bf3c2b$16ca9a20$b7e856ce@london>
Message-ID: <003101bf3c1a$92bd29b0$1a25d2d1@jenkondev.com>

>
> Does anyone know how much Python is really  used  on Redhat Linux with the
> conjunction with C++, especially the direction of Python calling C++
> functions?
>

It works wonderfully.  Using SWIG even makes it simple.  See
http://alldunn.com/wxPython/ for an example.

--
Robin Dunn
Software Craftsman
robin at AllDunn.com
http://AllDunn.com/robin/
http://AllDunn.com/wxPython/  Check it out!





From furnish at actel.com  Wed Dec  1 17:54:39 1999
From: furnish at actel.com (Geoffrey Furnish)
Date: Wed, 1 Dec 1999 08:54:39 -0800 (PST)
Subject: [C++-SIG] Python calling C++ issues 
In-Reply-To: <199912011422.IAA11646@gargoyle.cs.uchicago.edu>
References: <000201bf3bfd$aa22b830$3acbd9c2@peridot.optichrome.com>
	<199912011355.NAA07751@pineapple.cam-orl.co.uk>
	<199912011422.IAA11646@gargoyle.cs.uchicago.edu>
Message-ID: <14405.21199.700315.720894@gargle.gargle.HOWL>

David Beazley writes:
 > Duncan Grisby writes:
 > > 
 > > I haven't particularly followed the discussion, but I just want to
 > > point out that doing what you suggest is extremely dangerous. There is
 > > no guarantee that longs are the same size as pointers. 
 > 
 > With the notable exception of pointers to members, since when has a
 > long never been large enough to hold a pointer value?  Can you name a
 > specific instance where this doesn't work?  (or has the C++ standards
 > committee finally decided to break all pointers just for the hell of
 > it?)   SWIG does this internally and has never had any problems.

"Hasn't failed yet" is not the same as "works because it is intended
to work".  The right way to hold pointer values, is in objects of
pointer type.

-- 
Geoffrey Furnish            Actel Corporation        furnish at actel.com
Senior Staff Engineer      955 East Arques Ave       voice: 408-522-7528
Placement & Routing     Sunnyvale, CA   94086-4533   fax:   408-328-2303

"... because only those who write the code truly control the project."
						      -- Jamie Zawinski



From furnish at actel.com  Wed Dec  1 18:08:44 1999
From: furnish at actel.com (Geoffrey Furnish)
Date: Wed, 1 Dec 1999 09:08:44 -0800 (PST)
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: <38453D04.6B02D3A7@maxtal.com.au>
References: <199912011355.NAA07751@pineapple.cam-orl.co.uk>
	<000901bf3c07$67979c10$3acbd9c2@peridot.optichrome.com>
	<199912011437.IAA11853@gargoyle.cs.uchicago.edu>
	<38453D04.6B02D3A7@maxtal.com.au>
Message-ID: <14405.22044.983702.729937@gargle.gargle.HOWL>

skaller writes:
 > David Beazley wrote:
 > The conversion of pointers to ints is not supported in C++:

Right.  Pointer values must be held in objects of pointer type.  And
anticipating the following, I'll go ahead and state that pointer
values must be held in objects of pointer type which are of the same
type as the pointer in question (or which are accessible through valid 
pointer conversions).

 > For those using C Python sources with C++, there are two
 > gotchas.
 > 
 > 	1)  Enumerations are NOT integers in C++
 > 		[Luckily, Guido doesn't use them much]

Just to amplify, you can always make an int from an enum if you need
to.  A common case where this issue comes up, is in situations like
the following:

enum { This, That };

vector<int>::iterator it = find( v.begin(), v.end(), That );

which will not work (will not compile), but which can be repaired by
doing: 

vector<int>::iterator it = find( v.begin(), v.end(), int(That) );

 > 	2) The very core of Python contains a serious
 > 		breach of C++ rules: it casts functions
 > 		pointers, which is NOT allowed AT ALL.
 > 
 > [I'm responsible for that rule!]

Thank you!

 > Luckily, this actually works on most C++ compilers.

I disagree with this.  It does not work on /any/ "C++" compilers.
There are however, on the market, many compilers which support any of
a family of as yet undocumented languages, which seem all to be
similar to "C++" in many ways, but which evidently support various
"extension" constructs that are illegal in "C++", such as casting
pointers.

The right thing to do, is to avoid efforts to cast function pointers.

For the Python/C++ interface, this basically means you need a seperate 
extension type object for each C++ object you wish to register as a
Python extension.

Making this machinery work easily and maintainably seems to be
challenging.  I don't personally have much hope for doing it, except
through heavy use of template mechanisms, which have been the source
of much consternation on this sig.

-- 
Geoffrey Furnish            Actel Corporation        furnish at actel.com
Senior Staff Engineer      955 East Arques Ave       voice: 408-522-7528
Placement & Routing     Sunnyvale, CA   94086-4533   fax:   408-328-2303

"... because only those who write the code truly control the project."
						      -- Jamie Zawinski



From furnish at actel.com  Wed Dec  1 19:00:21 1999
From: furnish at actel.com (Geoffrey Furnish)
Date: Wed, 1 Dec 1999 10:00:21 -0800 (PST)
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: <028301bf3c2b$16ca9a20$b7e856ce@london>
References: <5865800428DFD21199020090274E8C9554B4D0@cisncdc.cissv.canon.com>
	<028301bf3c2b$16ca9a20$b7e856ce@london>
Message-ID: <14405.25141.297349.76968@gargle.gargle.HOWL>

Ephi Dror writes:
 > Unfortunately, as of this moment, we have not yet successfully run our
 > simple spam program.

I'd like to respond to some parts of your message.  I will confess up
front that I have not analyzed your recent posts personally in any
detail whatsoever, and have not followed the technical exchange.  I'm
wading in an aligator pond, fighting for my life right now in my day
job, and simply couldn't afford the distraction.  I've gotten the
vague impression that you were given good advice, but can easily
believe you haven't yet found all the necessary tricks of the trade.
So, with that background, I'd like to comment on the non-technical
aspects of your post.

 > It is partially worked but then we got the two unexplained strange
 > phenomena that as of now, nobody seems to understand why.  Also,
 > even if we got those problems solved using compiler xyz version 123
 > who knows what other problems are waiting for us in the next
 > corner.

This is an important reason why, imo, the focus of this sig should be
on constructing facilities that are correct according to a strict
reading of the ISO C++ language definition.  Compiler implementation
quality variations will always be a fact of life.  But there is an
important qualitative distinction between unsupported features, and
unsupportable features.  It may be desirable in the eyes of many to
limit use of the former, but it is /imperative/ to avoid use of the
later. 

 > I am personally very disappointed from all of this. I really truly
 > wanted to give python a chance. I got the impression by reading one
 > of the Python books back cover that what we want to do is common
 > practice and should be a plug and play .
 > (http://www.amazon.com/exec/obidos/ts/book-reviews/1565921976/103-5409830-1711020)
 > Reading the books give the impression that Python and C++ were
 > meant to live in peace and be haply working with each
 > other. Unfortunately, my personal experience realty proved the
 > opposite. 

Well, there are shades of truth on both sides.  C code is C++ code,
so of course, techniques which work with Python/C, will work with
Python/C++ too.  If you just stick to the part of C++ which happens to 
be equivalent to C, then you can certainly get the examples from the
books to work basically without change.

But if you do that, you aren't really "using C++".  And therein lies
the rub.  

 > Python may work nicely with C code, but who needs that 30
 > days before the year 2000 !!, Programmers today need at least
 > integration with existing C++ code and very soon they need
 > integration with Java code.  Our example only had the simplest C++
 > class in the word, and Python could not even use it correctly. Is
 > just un believable to me.  It is possible that we made stupid
 > mistakes in the code, it is also possible that we are bad
 > programmers. Who knows?. We did not hide our code. In fact, I
 > attached it to my original email.

Well, like I said above, I haven't analyzed your posts for technical
details, so I can't say whether you made avoidable mistakes.  But I
definitely do agree with you that it is roughly 100 times harder than
it needs to be, to use Python from C++.  The charter of this sig is to 
fix that, by developing the additional software that would allow
Python's compiled interface to be exploited from C++ "with ease".

The first and most basic issue, is compiling Python so it initializes
C++ global objects correctly.  There is a patch on the sig's www site
to help with that.

Beyond that, it should be possible to call C++ functions, both global
functions and member functions, easily.  Several of us have efforts in 
this area under way, but no consensus has really emerged yet.

Beyond that, it would be desirable to abolish the horid NULL checking
that is so endemic to Python C extension authorship.  PyExceptions
should be real C++ exceptions.  

Beyond that, you'd like to have a C++ friendly API for Python services.

Beyond that, you'd like to have classes to encapsulate the API
functional subsystems in the Python C API.  

At that point, programming Python from C++ would be "easy".  We're not 
there yet.

 > Does anyone know how much Python is really used on Redhat Linux
 > with the conjunction with C++, especially the direction of Python
 > calling C++ functions?
 > 
 > We have not given up yet but we are close. I would love to hear
 > from the group that I am wrong. 

I don't think you're wrong.  But I do think you may be missing the
purpose of the SIG.  The SIG isn't really here for user support--at
least that's not the primary mission.  The primary mission of this
SIG, at this point in time, is to FIX THE PROBLEM!!!  Python is
unnecessarily hard to extend.  That is not to say it is iimpossible,
or even that it is too hard to be worth it on an absolute scale.  Ian
gave a detailed description of one effort.  I have personal knowledge
of extensive extension effort underway in the DOE labs, at NASA, and
beyond, and these efforts have been featured at length in public talks
at Python workshops over the years.  There are doubtless hundreds more 
such development projects at institutions all over the world.  So it
can be done.  But it could be done much more easily, if the right
supporting infrastructure is developed.  At this point in time, that
is the primary mission of this sig.

At some future date, after we have reached a point where the SIG has
something tangible and generally agreed upon, to put forward to the
larger community for use, and hopefully ultimately for direct
incorporation into the Python core, then the focus of this SIG will
change to more of a user support organization.  But we have an immense 
amount of work to do before we get to that point.

 > That I was just blind and simply did not get it.  That calling C++
 > class from Python is really a piece of cake after all and
 > especially on RedHat 6.1 Linux with standard installation
 > components.

I would encourage you to view your recent experiences not as a cause
for dismissal of Python as a programming environment, but as a
personal challenge for involvement to improve the situation for the
common good.  Python is community devleopment project, just as most
free software is.  Someone was recently inquring to know when GCC was
ever going to be upgraded to suppor ISO C++, and someone (Nathan
Myers?) wrote back "Sooner if you help".

So.  When will Python be easy to use from C++?  Sonner if you help.

 > Thank you all for helping us.
 > 
 > We hope to stay in the group and contribute in the future, but it
 > all depend on what next for us in terms of using Python/C++.

Grab a keyboard, and get to work!  Write something that fixes some
part of the problems you've encountered, and lets get it posted for
public comment.

-- 
Geoffrey Furnish            Actel Corporation        furnish at actel.com
Senior Staff Engineer      955 East Arques Ave       voice: 408-522-7528
Placement & Routing     Sunnyvale, CA   94086-4533   fax:   408-328-2303

"... because only those who write the code truly control the project."
						      -- Jamie Zawinski



From dgrisby at uk.research.att.com  Wed Dec  1 19:24:32 1999
From: dgrisby at uk.research.att.com (Duncan Grisby)
Date: Wed, 01 Dec 1999 18:24:32 +0000
Subject: [C++-SIG] Python calling C++ issues 
In-Reply-To: Your message of "Wed, 01 Dec 1999 10:00:21 PST."
             <14405.25141.297349.76968@gargle.gargle.HOWL> 
Message-ID: <199912011824.SAA08715@pineapple.cam-orl.co.uk>

On Wednesday 1 December, Geoffrey Furnish wrote:

> The first and most basic issue, is compiling Python so it initializes
> C++ global objects correctly.  There is a patch on the sig's www site
> to help with that.

Unfortunately, I don't believe that that patch addresses the full
problem with global initialisers. It guarantees that global
initialisers will be called for object files linked with the actual
Python executable, since the main() function is C++. Unfortunately,
compiling main() with C++ gives no guarantees about initialisation of
shared libraries dynamically loaded at run time.

It is my understanding that a sizeable portion of C / C++ code called
from Python is accessed as extension modules loaded at run-time. The
code I've written certainly is. C++ as a language gives no guarantees
about how and whether static initialisers in extension modules are
handled. I don't think Python can do much to help with this -- it's
fundamentally a linker and operating system issue.

Fortunately, it seems that the majority of platforms do handle this
correctly. All ELF-based Unixes should do. Windows NT also does. The
only platform with a problem I have encountered so far is AIX, and
that can be fixed with a small patch to Python which uses an AIX
specific call rather than libdl.

I'm not saying that the patch to compile main() with C++ is wrong --
it's just not sufficient.

Cheers,

Duncan.

-- 
 -- Duncan Grisby  \  Research Engineer  --
  -- AT&T Laboratories Cambridge          --
   -- http://www.uk.research.att.com/~dpg1 --



From beazley at cs.uchicago.edu  Wed Dec  1 20:49:54 1999
From: beazley at cs.uchicago.edu (David Beazley)
Date: Wed, 1 Dec 1999 13:49:54 -0600 (CST)
Subject: [C++-SIG] Python calling C++ issues 
References: <000201bf3bfd$aa22b830$3acbd9c2@peridot.optichrome.com>
	<199912011355.NAA07751@pineapple.cam-orl.co.uk>
	<199912011422.IAA11646@gargoyle.cs.uchicago.edu>
	<14405.21199.700315.720894@gargle.gargle.HOWL>
Message-ID: <199912011949.NAA17313@gargoyle.cs.uchicago.edu>

Geoffrey Furnish writes:
> 
> "Hasn't failed yet" is not the same as "works because it is intended
> to work".  The right way to hold pointer values, is in objects of
> pointer type.
> 

<rant>
Yeah, yeah, whatever.  For all of the time that C++ facists spend
whining about "intended behavior" and the "right" way to do things, I
have yet to see a compiler with 13 bit integers, 77 bit longs, and 125
bit pointers, nor do I ever think I will see such a thing in my
lifetime.  Therefore, semantics aside, there is really nothing wrong
with opting for a simple approach if it works and you know what you
are doing.  Otherwise, I just don't see the point of needlessly making
things 10 times more complicated than it has to be.
</rant>

-- Dave






From guido at CNRI.Reston.VA.US  Wed Dec  1 23:03:46 1999
From: guido at CNRI.Reston.VA.US (Guido van Rossum)
Date: Wed, 01 Dec 1999 17:03:46 -0500
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: Your message of "Wed, 01 Dec 1999 13:49:54 CST."
             <199912011949.NAA17313@gargoyle.cs.uchicago.edu> 
References: <000201bf3bfd$aa22b830$3acbd9c2@peridot.optichrome.com> <199912011355.NAA07751@pineapple.cam-orl.co.uk> <199912011422.IAA11646@gargoyle.cs.uchicago.edu> <14405.21199.700315.720894@gargle.gargle.HOWL>  
            <199912011949.NAA17313@gargoyle.cs.uchicago.edu> 
Message-ID: <199912012203.RAA10759@eric.cnri.reston.va.us>

> Geoffrey Furnish writes:
> > 
> > "Hasn't failed yet" is not the same as "works because it is intended
> > to work".  The right way to hold pointer values, is in objects of
> > pointer type.

David Beazley <beazley at cs.uchicago.edu>:

> <rant>
> Yeah, yeah, whatever.  For all of the time that C++ facists spend
> whining about "intended behavior" and the "right" way to do things, I
> have yet to see a compiler with 13 bit integers, 77 bit longs, and 125
> bit pointers, nor do I ever think I will see such a thing in my
> lifetime.  Therefore, semantics aside, there is really nothing wrong
> with opting for a simple approach if it works and you know what you
> are doing.  Otherwise, I just don't see the point of needlessly making
> things 10 times more complicated than it has to be.
> </rant>

Rant aside, what compiler is going to support different datatype sizes
for stuff declared extern "C" than for native C++ types?  Get real.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From dubois1 at llnl.gov  Wed Dec  1 23:23:24 1999
From: dubois1 at llnl.gov (Paul F. Dubois)
Date: Wed, 1 Dec 1999 14:23:24 -0800
Subject: [C++-SIG] Python calling C++ issues
References: <199912012203.RAA10759@eric.cnri.reston.va.us>
Message-ID: <99120114285301.17182@almanac>

O
> 
> Rant aside, what compiler is going to support different datatype sizes
> for stuff declared extern "C" than for native C++ types?  Get real.
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> _______________________________________________
> C++-SIG maillist  -  C++-SIG at python.org
> http://www.python.org/mailman/listinfo/c++-sig

Gotcha.

IBM, that's who. It was IBM who insisted on the C++ standard specifiying that
a pointer to an extern "C" function and a pointer to a C++ function with the
same signature are type incompatible. Without this provision they claimed they 
could not implement C++ on one of their mainframes. I don't have the article 
in front of me but this point was the subject of a column in the C++ Report
(Aug 99, I think). 

I have some improvements to CXX thanks to Barry Scott but I am going to have to
figure out what this annoying change to the standard means before doing any
more.



From furnish at actel.com  Wed Dec  1 23:37:01 1999
From: furnish at actel.com (Geoffrey Furnish)
Date: Wed, 1 Dec 1999 14:37:01 -0800 (PST)
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: <199912012203.RAA10759@eric.cnri.reston.va.us>
References: <000201bf3bfd$aa22b830$3acbd9c2@peridot.optichrome.com>
	<199912011355.NAA07751@pineapple.cam-orl.co.uk>
	<199912011422.IAA11646@gargoyle.cs.uchicago.edu>
	<14405.21199.700315.720894@gargle.gargle.HOWL>
	<199912011949.NAA17313@gargoyle.cs.uchicago.edu>
	<199912012203.RAA10759@eric.cnri.reston.va.us>
Message-ID: <14405.41741.930376.14233@gargle.gargle.HOWL>

For the list: The most contentious aspects of this tiff are being
handled in private email.  However I would like to comment publicly on
one aspect of Guido's post:

Guido van Rossum writes:
 > Rant aside, what compiler is going to support different datatype sizes
 > for stuff declared extern "C" than for native C++ types?  Get real.

One of my personal beefs with the casting to PyCFunction, has nothing
to do with the (illegal according to strict C++ rules) presumptions
about function pointer compatibility.  There is also the issue of just
exactly what the function signature is.  The Python C implementation
actually casts all function pointers to PyCFunction, even if they have
different arg lists!  The fact that this is illegal by C++ language
rules is almost the least of all concerns!  The size of the pointer
type is almost irrelevant.  Here we have type information (the
function invocation arg list) being communicated through independent
parameters.

Okay, fine, there aren't a lot of choices in C.  Whatever.  In C++
though, the compiler actually supports a decent type system, and there 
really is no reason to attempt to subvert the C++ type system and
reincarnate its soul in the form of extraneous parameters, whose
definitions have to be predetermined and extensions to which cannot be 
properly anticipated by the implementor.

These sorts of problems can be solved in C++ by letting the compiler's 
type system do the work for you.  For this case in question, probably
its going to have to mean some template trampolines and all the
associated machinery.  And unfortunately, this reliance upon templates 
has caused a lot of fuss on this SIG.

Frnakly, I don't want to get into a C versus C++ style debate.  I
really couldn't care less how people write their C code.  Observe that
I have never posted to c.l.p suggesting that "Python's implementation
would be a lot nicer if it was swtiched from C to C++".  That does
happen to be my personal oppinion, but I don't advocate it publicly
because such advocacy isn't going to change the situation, and just
creates needless stir and resentment.

But by the same token, I really don't think it's fair for people with
an ax to grind against C++ to use the C++ SIG to vent about how
dogmatic the C++ camp is.

Seriously, couldn't we all just calm down and work on solving the
problem? 

-- 
Geoffrey Furnish            Actel Corporation        furnish at actel.com
Senior Staff Engineer      955 East Arques Ave       voice: 408-522-7528
Placement & Routing     Sunnyvale, CA   94086-4533   fax:   408-328-2303

"... because only those who write the code truly control the project."
						      -- Jamie Zawinski



From beazley at cs.uchicago.edu  Wed Dec  1 23:37:27 1999
From: beazley at cs.uchicago.edu (David Beazley)
Date: Wed, 1 Dec 1999 16:37:27 -0600 (CST)
Subject: [C++-SIG] Python calling C++ issues
References: <199912012203.RAA10759@eric.cnri.reston.va.us>
	<99120114285301.17182@almanac>
Message-ID: <199912012237.QAA21457@gargoyle.cs.uchicago.edu>

> 
> Gotcha.
> 
> IBM, that's who. It was IBM who insisted on the C++ standard specifiying that
> a pointer to an extern "C" function and a pointer to a C++ function with the
> same signature are type incompatible. Without this provision they claimed they 
> could not implement C++ on one of their mainframes. I don't have the article 
> in front of me but this point was the subject of a column in the C++ Report
> (Aug 99, I think). 
> 

Aiiiigh!!! Make it stop! Make it stop!

First off, I want to apologize for my out-of-line facist comment from
earlier--having spent far too much time trying to sort out changes in
the C++ spec recently has turned this into a hot-button for me (so
much so, I was about 30 seconds away from deleting all signs of C++
from my machines last month).

As for the casting to long issue, Geoffrey pointed out to me privately
that you can't do this if you care about inheritance which is, of
course, completely correct.   SWIG, although it does cast to a long at
some point in its internal pointer handling, also keeps track of type
information for this purpose.  For instance, depending on the type,
it is necessary to perform a casting operation to correctly adjust the
value of a pointer when cast from a derived class to a base class.
If you're extremely lucky, you can avoid this step and things will
work, but there's no guarantee that it will work everywhere and it
definitely won't work with multiple inheritance.

However, with that said, I'm still a big fan of keeping things as
simple as is reasonable.

As for the extern "C" issue, I would love to know how to officially
resolve that issue.  For now, I've solved it by simply slapping an
extern "C" around everything that gets hooked to Python (all wrapper
functions).  Still, it doesn't seem like a very satisfactory solution.
I have no idea what it implies for something like CXX.

Cheers,

Dave




From skaller at maxtal.com.au  Thu Dec  2 15:18:29 1999
From: skaller at maxtal.com.au (skaller)
Date: Fri, 03 Dec 1999 01:18:29 +1100
Subject: [C++-SIG] Python calling C++ issues
References: <199912011355.NAA07751@pineapple.cam-orl.co.uk>
		<000901bf3c07$67979c10$3acbd9c2@peridot.optichrome.com>
		<199912011437.IAA11853@gargoyle.cs.uchicago.edu>
		<38453D04.6B02D3A7@maxtal.com.au> <14405.22044.983702.729937@gargle.gargle.HOWL>
Message-ID: <38467FB5.10ECF94@maxtal.com.au>

Geoffrey Furnish wrote:
> The right thing to do, is to avoid efforts to cast function pointers.
> 
> For the Python/C++ interface, this basically means you need a seperate
> extension type object for each C++ object you wish to register as a
> Python extension.

	I don't believe this is the case. See below.
 
> Making this machinery work easily and maintainably seems to be
> challenging.  I don't personally have much hope for doing it, except
> through heavy use of template mechanisms, which have been the source
> of much consternation on this sig.

	I don't really understand this. It is very easy to do.
In some kind of minor simplification of python, exactly one abstract
base class is required. This class has a vtable (set of pure virtual
functions) which match the Python vtable.

	That's it: all extension type derive from this abstract
base, and they're all automatically valid python types.

	Now the gotcha: instead of a single linear vtable,
python has several subtables. The presence or absence of 
the subtable pointer (e.g the pointer to the sequence
subtable) has semantics. This complication may mean
some tricky design is needed; perhaps a minor
restriction of the extension type semantics 
(like: 'you can't be both a sequence and map').

	I will give an example, by taking
a tiny subset of the python virtual functions:
the len function, for example:

	class cpp_py_object {
	virtual int len() const = 0;
	...
	};


	Now, by using THIS class as the sole base
of all cpp extension types, you only need a single
CPython type, with a pointer to the C++ object
of type cpp_py_object.

	By FAR the most interesting aspect of this
is that you can dynamically load shared libraries
using dlopen (or windows equivalent), which have
factory functions in them which create extension types.

	These DLL's don't [necessarily] 
need to know anything about the Python wrapper objects which wrap them. 

-- 
John Skaller, mailto:skaller at maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
voice: 61-2-9660-0850



From skaller at maxtal.com.au  Thu Dec  2 15:41:37 1999
From: skaller at maxtal.com.au (skaller)
Date: Fri, 03 Dec 1999 01:41:37 +1100
Subject: [C++-SIG] Python calling C++ issues
References: <000201bf3bfd$aa22b830$3acbd9c2@peridot.optichrome.com> <199912011355.NAA07751@pineapple.cam-orl.co.uk> <199912011422.IAA11646@gargoyle.cs.uchicago.edu> <14405.21199.700315.720894@gargle.gargle.HOWL>  
	            <199912011949.NAA17313@gargoyle.cs.uchicago.edu> <199912012203.RAA10759@eric.cnri.reston.va.us>
Message-ID: <38468521.69CEDD30@maxtal.com.au>

Guido van Rossum wrote:

> Rant aside, what compiler is going to support different datatype sizes
> for stuff declared extern "C" than for native C++ types?  Get real.

Actually, there is a kind of understanding on the committee of the
concept
of 'compatible C and C++ compilers' aka C/C++ compiler. This arose
because

	1) many compilers are C/C++ compilers: same compiler, with a mode
switch.
	2) there is considerable difficulty expressing any requirements on
		what 'extern "C"' actually means, otherwise.

Also there is some reason for there to be a difference between C
and C++ on some systems, especially in the case of things like
function pointers.

But, there is another issue here: for function pointers,
what exactly does 'extern "C"' mean if the _parameters_
are, for example, an enum. Or another function pointer??

In my opinion, it is reasonable to target a product like
Python on a C/C++ compiler pair. It is a pragmatic choice,
excluding few platforms/systems (and even in these cases,
patches may provide a fix).

Just be wary of 'get reals': from being on the committee I
know there are a LOT of examples where people aren't doing
quite what you'd expect, claim to have a good reason for it,
and insist their implementation not be broken by a gratuitous rule :-)

Sigh. A final explanation. The ISO C++ Standard does NOT define
any C++ language. It defines 'conforming translator'. [This is not
the case for C]. Many different languages can be compiled by a
conforming translator, the specification is said to be
parameterized by certain rules and quantities which the
vendor is required to provide: i.e. a 'C++ compiler' is NOT
a piece of software. It is a piece of software AND some
documentation.

This is an excellent device, because it allows a software
vendor to say: 'this software requires a C++ compiler
satisfying the following additional constraints',
and it is up to the client to find such a product to
build the software.

-- 
John Skaller, mailto:skaller at maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
voice: 61-2-9660-0850



From skaller at maxtal.com.au  Thu Dec  2 15:51:39 1999
From: skaller at maxtal.com.au (skaller)
Date: Fri, 03 Dec 1999 01:51:39 +1100
Subject: [C++-SIG] Python calling C++ issues
References: <199912012203.RAA10759@eric.cnri.reston.va.us>
		<99120114285301.17182@almanac> <199912012237.QAA21457@gargoyle.cs.uchicago.edu>
Message-ID: <3846877B.67C633C2@maxtal.com.au>

David Beazley wrote:

> Aiiiigh!!! Make it stop! Make it stop!

	Try ocaml :-)
 
> However, with that said, I'm still a big fan of keeping things as
> simple as is reasonable.
> 
> As for the extern "C" issue, I would love to know how to officially
> resolve that issue.

	First you read the Standard. If you can't work out what it means,
you ask a committee delegate (such as me). If I can't figure it
out, which is likely, I can post the question directly on the committee
email reflector, and echo the responses back to you.

>  For now, I've solved it by simply slapping an
> extern "C" around everything that gets hooked to Python (all wrapper
> functions).  Still, it doesn't seem like a very satisfactory solution.
> I have no idea what it implies for something like CXX.

	You must assume your client (possibly you) has a 
C/C++ compiler pair, and Python was built with the C part of it
(except for main, which must be built with C++ to ensure the
run time is set up for exception handling and RTTI).

	When it comes to DLLS/shared libraries, you must
rely on vendor specs, and luck. The same applies to
threading, and calling ANY system function other than
those required by the Standard.

	That is, basically, Guido's "get real" approach
is pretty close to the right idea. :-)

-- 
John Skaller, mailto:skaller at maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
voice: 61-2-9660-0850



From guido at CNRI.Reston.VA.US  Thu Dec  2 16:31:13 1999
From: guido at CNRI.Reston.VA.US (Guido van Rossum)
Date: Thu, 02 Dec 1999 10:31:13 -0500
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: Your message of "Fri, 03 Dec 1999 01:41:37 +1100."
             <38468521.69CEDD30@maxtal.com.au> 
References: <000201bf3bfd$aa22b830$3acbd9c2@peridot.optichrome.com> <199912011355.NAA07751@pineapple.cam-orl.co.uk> <199912011422.IAA11646@gargoyle.cs.uchicago.edu> <14405.21199.700315.720894@gargle.gargle.HOWL> <199912011949.NAA17313@gargoyle.cs.uchicago.edu> <199912012203.RAA10759@eric.cnri.reston.va.us>  
            <38468521.69CEDD30@maxtal.com.au> 
Message-ID: <199912021531.KAA14331@eric.cnri.reston.va.us>

All I can say is, the more I hear about C++, the less likely it
becomes that Python 2.0 will be written in it.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From furnish at actel.com  Thu Dec  2 19:44:37 1999
From: furnish at actel.com (Geoffrey Furnish)
Date: Thu, 2 Dec 1999 10:44:37 -0800 (PST)
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: <38467FB5.10ECF94@maxtal.com.au>
References: <199912011355.NAA07751@pineapple.cam-orl.co.uk>
	<000901bf3c07$67979c10$3acbd9c2@peridot.optichrome.com>
	<199912011437.IAA11853@gargoyle.cs.uchicago.edu>
	<38453D04.6B02D3A7@maxtal.com.au>
	<14405.22044.983702.729937@gargle.gargle.HOWL>
	<38467FB5.10ECF94@maxtal.com.au>
Message-ID: <14406.48661.44091.470344@gargle.gargle.HOWL>

skaller writes:
 > Geoffrey Furnish wrote:
 > > For the Python/C++ interface, this basically means you need a seperate
 > > extension type object for each C++ object you wish to register as a
 > > Python extension.
 > 
 > 	I don't believe this is the case. See below.
 >  [...]
 > 	I will give an example, by taking
 > a tiny subset of the python virtual functions:
 > the len function, for example:
 > 
 > 	class cpp_py_object {
 > 	virtual int len() const = 0;
 > 	...
 > 	};
 > 
 > 
 > 	Now, by using THIS class as the sole base
 > of all cpp extension types, you only need a single
 > CPython type, with a pointer to the C++ object
 > of type cpp_py_object.

Okay, great.  So we can invoke the virtual function len() on an
extension object of unknown type.

Now what's the plan for invoking non virtual methods of C++ objects
without knowing their type?

-- 
Geoffrey Furnish            Actel Corporation        furnish at actel.com
Senior Staff Engineer      955 East Arques Ave       voice: 408-522-7528
Placement & Routing     Sunnyvale, CA   94086-4533   fax:   408-328-2303

"... because only those who write the code truly control the project."
						      -- Jamie Zawinski



From gregc at cgl.ucsf.EDU  Thu Dec  2 20:49:48 1999
From: gregc at cgl.ucsf.EDU (Greg Couch)
Date: Thu, 2 Dec 1999 11:49:48 -0800 (PST)
Subject: [C++-SIG] Re: Python calling C++ issues
Message-ID: <199912021949.LAA166506@socrates.cgl.ucsf.edu>

Guido van Rossum <guido at CNRI.Reston.VA.US>:

> Rant aside, what compiler is going to support different datatype sizes
> for stuff declared extern "C" than for native C++ types?  Get real.

My understanding is that the issue usually isn't datatype sizes, it's
the calling convention.  The ABI for C++ functions may be different than
for C functions.  extern "C" gives that the C++ compiler that clue.
Clueless compilers require extension keywords, such as __cdecl, for
similar effects.

	Greg Couch
	gregc at cgl.ucsf.edu
	(c++-sig digest reader)



From dheise at debitel.net  Thu Dec  2 23:08:18 1999
From: dheise at debitel.net (Dirk Heise)
Date: Thu, 2 Dec 1999 23:08:18 +0100
Subject: [C++-SIG] Python calling C++ issues
Message-ID: <19991202220926.16106.qmail@psmtp1.dnsg.net>

> Von: Guido van Rossum <guido at CNRI.Reston.VA.US>
> An: c++-sig at python.org
> Betreff: Re: [C++-SIG] Python calling C++ issues
> Datum: Donnerstag, 2. Dezember 1999 16:31
> 
> All I can say is, the more I hear about C++, the less likely it
> becomes that Python 2.0 will be written in it.

Come on! You don't have to use all sick features they built
into the standard. During my dull C++-9-to-5 job, i avoid
operator overloading, C++ references, "const" whereever
possible, STL and hell-knows-what and i'm having a great
time as a C++-10-percenter. Possibly not Geoffreys opinion,
but it pays the rent :-)

But one serious question: Do you consider prototyping
Python 2 in Python 1?


Dirk




From skaller at maxtal.com.au  Fri Dec  3 19:55:38 1999
From: skaller at maxtal.com.au (skaller)
Date: Sat, 04 Dec 1999 05:55:38 +1100
Subject: [C++-SIG] Python calling C++ issues
References: <199912011355.NAA07751@pineapple.cam-orl.co.uk>
		<000901bf3c07$67979c10$3acbd9c2@peridot.optichrome.com>
		<199912011437.IAA11853@gargoyle.cs.uchicago.edu>
		<38453D04.6B02D3A7@maxtal.com.au>
		<14405.22044.983702.729937@gargle.gargle.HOWL>
		<38467FB5.10ECF94@maxtal.com.au> <14406.48661.44091.470344@gargle.gargle.HOWL>
Message-ID: <3848122A.662E0418@maxtal.com.au>

Geoffrey Furnish wrote:

> Okay, great.  So we can invoke the virtual function len() on an
> extension object of unknown type.

	yes.
 
> Now what's the plan for invoking non virtual methods of C++ objects
> without knowing their type?

	That is impossible. It is not necessary to be able
to do this, since the Python parser cannot generate any
code which calls it. Bytecode can only call 'virtual functions'
on PyObjects.

	Perhaps, you mean, how do we do this:

	object.method(arguments)

And the answer is: the same way python does. It is the only way.
Python calls 'getattr' to fetch 'method' by name.
The resulting function fetched has a fixed calling sequence.
Otherwise the bytecode interpreter could never call it.
It accepts a pair of arguments, a tuple and a dictionary.

	[Actually, there is a complication with an obsoleted 
flag, determining if **kwds is support -- but lets ignore that for
clarity].

	In other words, it is impossible to call ANYTHING
from python OTHER than one of the virtual methods.
And so that is all that needs to be provided.

	Now, this is not to say that C++ cannot
call other C++ method directly. But you have to understand
that a C++ method like

	void method (int x, float y) const

cannot ever be called by Python. Here's the sequence of events:

	1) Python calls the C 'getattr' method of the
single, fixed C++ wrapper type: this is a C function.

	2) The C function casts the object pointer
to the abstract base type of the C++ heirarchy.

	3) Then, it calls the C++ virtual getattr method,
after suitably rearranging the arguments.

	4) The C++ getattr method has to return something
that can be converted into a function which Python can call;
that is, it has to return a C function which takes a tuple
and dictionary as an argument.

	It finds this function in a table. Because C is a bit
brain dead, there is no way to create a closure here. 
[This is possible using Viper, since the implementation
language, ocaml, supports them: generic extensions in Viper
are much easier than in C or C++ [but binding existing
C libraries is, conversely, harder]

	The C function takes the first element of the tuple
argument, which is the object, and casts it to the CONCRETE
type begin defined, and calls the corresponding C++ method,
munging arguments as required. [Smly, the return value
of the C++ method has to be converted to a PyObject for the C
function to return]

	There are many ways to do this; using SWIG is one;
but it is IRRELEVANT. You can write it by hand for example.
The point is that only ONE abstract C++ class is required
as a base, and only ONE Python type object is required to
wrap it, and this WILL handle every single C++ class
suitably derived from the abstract base.

	I hope this is clear: the lookup of specific
non-virtual method on a class is independent of the
_automatically_ polymorphic lookup of the various
python virtual methods, including getattr and,
as you mentioned, len.

	It makes sense, to have a specific derived
class, in which the virtual getattr is overriden,
and uses a particular data structure and algorithm
to do attribute lookup; for example, for the kind of
table SWIG or other automatic generator generates.

	If I were implementing a getattr method by
hand, I'd be tempted to us an STL map, myself,
since it is likely to MUCH faster than the standard
linear search typically used implementing
extension types.

	Anyhow, the point is that there is nothing
new here, this is no different to C: finding attributes
of objects, including methods, has nothing to do with
the Python interface to the type object: it is universally
achieved by a type specific getattr method.

-- 
John Skaller, mailto:skaller at maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
voice: 61-2-9660-0850



From furnish at actel.com  Fri Dec  3 20:25:21 1999
From: furnish at actel.com (Geoffrey Furnish)
Date: Fri, 3 Dec 1999 11:25:21 -0800 (PST)
Subject: [C++-SIG] Python calling C++ issues
In-Reply-To: <3848122A.662E0418@maxtal.com.au>
References: <199912011355.NAA07751@pineapple.cam-orl.co.uk>
	<000901bf3c07$67979c10$3acbd9c2@peridot.optichrome.com>
	<199912011437.IAA11853@gargoyle.cs.uchicago.edu>
	<38453D04.6B02D3A7@maxtal.com.au>
	<14405.22044.983702.729937@gargle.gargle.HOWL>
	<38467FB5.10ECF94@maxtal.com.au>
	<14406.48661.44091.470344@gargle.gargle.HOWL>
	<3848122A.662E0418@maxtal.com.au>
Message-ID: <14408.6433.161591.566420@gargle.gargle.HOWL>

I for one am losing track of the argument here.  Everything you say
below about the mechanism that Python uses for finding and invoking
functions of modules, is well known, and if you didn't read it
anywhere first, can be directly observed from the Python C source.
The part of your post which relates to a factual rendition of how
things works in CPython today, is not in contention.

Your claim that this can all be handled with a single base class
is--until you demonstrate it by posting ocde which works as you
posit--at best understood to be an hypothesis.  Several of us on this
list, including myself, Paul, Barry, and doubtless others, have worked 
out schemes largely similar to what you describe.  What has not yet
been done, is to field an implementation that both has the desired
features, works with exactly one base class for all extensions, and is 
correct according to the C++ language standard.  I have pursued a
slightly different approach which is correct according to the language 
standard, but uses templates to fabricate on demand, much of the
machinery which is neded to make all the funciton invocations.  I am
sorry that I have not yet been able to get this fully dusted off and
whipped into presentable form.  Still working on that.  Meanwhile,
Paul is pursing a system which does not have that characteristic, but
instead works more like what you describe.  But he is also not done,
and still working out final details.

I invite you to develop and post code which works as you claim can be
done.  We would all love to see a tangible incarnation of the design
ideas you espouse.  In other words, show us the beef.

skaller writes:
 > Geoffrey Furnish wrote:
 > 
 > > Okay, great.  So we can invoke the virtual function len() on an
 > > extension object of unknown type.
 > 
 > 	yes.
 >  
 > > Now what's the plan for invoking non virtual methods of C++ objects
 > > without knowing their type?
 > 
 > 	That is impossible. It is not necessary to be able
 > to do this, since the Python parser cannot generate any
 > code which calls it. Bytecode can only call 'virtual functions'
 > on PyObjects.
 > 
 > 	Perhaps, you mean, how do we do this:
 > 
 > 	object.method(arguments)
 > 
 > And the answer is: the same way python does. It is the only way.
 > Python calls 'getattr' to fetch 'method' by name.
 > The resulting function fetched has a fixed calling sequence.
 > Otherwise the bytecode interpreter could never call it.
 > It accepts a pair of arguments, a tuple and a dictionary.
 > 
 > 	[Actually, there is a complication with an obsoleted 
 > flag, determining if **kwds is support -- but lets ignore that for
 > clarity].
 > 
 > 	In other words, it is impossible to call ANYTHING
 > from python OTHER than one of the virtual methods.
 > And so that is all that needs to be provided.
 > 
 > 	Now, this is not to say that C++ cannot
 > call other C++ method directly. But you have to understand
 > that a C++ method like
 > 
 > 	void method (int x, float y) const
 > 
 > cannot ever be called by Python. Here's the sequence of events:
 > 
 > 	1) Python calls the C 'getattr' method of the
 > single, fixed C++ wrapper type: this is a C function.
 > 
 > 	2) The C function casts the object pointer
 > to the abstract base type of the C++ heirarchy.
 > 
 > 	3) Then, it calls the C++ virtual getattr method,
 > after suitably rearranging the arguments.
 > 
 > 	4) The C++ getattr method has to return something
 > that can be converted into a function which Python can call;
 > that is, it has to return a C function which takes a tuple
 > and dictionary as an argument.
 > 
 > 	It finds this function in a table. Because C is a bit
 > brain dead, there is no way to create a closure here. 
 > [This is possible using Viper, since the implementation
 > language, ocaml, supports them: generic extensions in Viper
 > are much easier than in C or C++ [but binding existing
 > C libraries is, conversely, harder]
 > 
 > 	The C function takes the first element of the tuple
 > argument, which is the object, and casts it to the CONCRETE
 > type begin defined, and calls the corresponding C++ method,
 > munging arguments as required. [Smly, the return value
 > of the C++ method has to be converted to a PyObject for the C
 > function to return]
 > 
 > 	There are many ways to do this; using SWIG is one;
 > but it is IRRELEVANT. You can write it by hand for example.
 > The point is that only ONE abstract C++ class is required
 > as a base, and only ONE Python type object is required to
 > wrap it, and this WILL handle every single C++ class
 > suitably derived from the abstract base.
 > 
 > 	I hope this is clear: the lookup of specific
 > non-virtual method on a class is independent of the
 > _automatically_ polymorphic lookup of the various
 > python virtual methods, including getattr and,
 > as you mentioned, len.
 > 
 > 	It makes sense, to have a specific derived
 > class, in which the virtual getattr is overriden,
 > and uses a particular data structure and algorithm
 > to do attribute lookup; for example, for the kind of
 > table SWIG or other automatic generator generates.
 > 
 > 	If I were implementing a getattr method by
 > hand, I'd be tempted to us an STL map, myself,
 > since it is likely to MUCH faster than the standard
 > linear search typically used implementing
 > extension types.
 > 
 > 	Anyhow, the point is that there is nothing
 > new here, this is no different to C: finding attributes
 > of objects, including methods, has nothing to do with
 > the Python interface to the type object: it is universally
 > achieved by a type specific getattr method.
 > 
 > -- 
 > John Skaller, mailto:skaller at maxtal.com.au
 > 10/1 Toxteth Rd Glebe NSW 2037 Australia
 > homepage: http://www.maxtal.com.au/~skaller
 > voice: 61-2-9660-0850
 > 



From skaller at maxtal.com.au  Fri Dec  3 21:17:34 1999
From: skaller at maxtal.com.au (skaller)
Date: Sat, 04 Dec 1999 07:17:34 +1100
Subject: [C++-SIG] Python calling C++ issues
References: <199912011355.NAA07751@pineapple.cam-orl.co.uk>
		<000901bf3c07$67979c10$3acbd9c2@peridot.optichrome.com>
		<199912011437.IAA11853@gargoyle.cs.uchicago.edu>
		<38453D04.6B02D3A7@maxtal.com.au>
		<14405.22044.983702.729937@gargle.gargle.HOWL>
		<38467FB5.10ECF94@maxtal.com.au>
		<14406.48661.44091.470344@gargle.gargle.HOWL>
		<3848122A.662E0418@maxtal.com.au> <14408.6433.161591.566420@gargle.gargle.HOWL>
Message-ID: <3848255E.48F14E1C@maxtal.com.au>

Geoffrey Furnish wrote:
> 
> I for one am losing track of the argument here.  Everything you say
> below about the mechanism that Python uses for finding and invoking
> functions of modules, is well known, and if you didn't read it
> anywhere first, can be directly observed from the Python C source.
> The part of your post which relates to a factual rendition of how
> things works in CPython today, is not in contention.

	Good.

> Your claim that this can all be handled with a single base class
> is--until you demonstrate it by posting ocde which works as you
> posit--at best understood to be an hypothesis. 

	Fine. It seems like a good hypothesis. I have built
some of the code. I know it can work, since I know the Python
object model reasonably well, and I know C++ backwards,
forwards, and inside out. :-)

> Several of us on this list, including myself, Paul, Barry, 
> and doubtless others, have worked
> out schemes largely similar to what you describe.

	What problems were encountered?

> I invite you to develop and post code which works as you claim can be
> done.  We would all love to see a tangible incarnation of the design
> ideas you espouse.  In other words, show us the beef.

	I have another more pressing project. What I can tell you
is that I did indeed begin construction of a standard C++ wrapper,
but I didn't bother to complete it because I found a better way to 
achieve my goals. What I'm saying is that there is a few days work
involved, and I won't gain anything out of it, because I'm not that
interested in extending CPython using C++ any more.

	I'm still on this mailing list because I am interested
in what others are doing and I'm willing to provide what assistance
I can. I can give a sketch of the required architecture.
I would be very interested in talking about some of the 
detail problems, such as the one I mentioned about subtables.

	But I have no comprehension of how it is not
completely obvious what path must be taken. You made a comment
something like: 'So now we can call the virtual len, what about
the non-virtual methods'.

	And my point is: that is an implementation detail
of the specific getattr hook a particular subclass of the abstract
base provides, and those details aren't relevant to writing
the C/C++ wrapper.

	It is quite true, that there are multiple ways to do this.
[That is, lookup attributes]

	But it can be done USING POLYMORPHISM. 
So it can be separated out, utterly and totally. 
Exactly as there is more than one way to determine the 'len'
of an object. It depends on the derived type, and the derived
type provides the answer by overriding the virtual len.

In other words, to the point where such
specific functions are dynamically loadable.
[This is the true test of correct abstraction: when you can dynamically
load a class, known ONLY by it's abstract base. Ignorance of 
any other structure is enforced by the very fact of dynamic loading:
it just isn't possible to call a method not in the abstract base,
even if you get the run-time type of the object, you don't know the 
static type.]

	Consider the 'len' function. Clearly, we need a
C++ class like:

	class Cpp_PyObject {
		virtual int len() const =0;
		..
	};

	How is this called? Well, the python
type struct has a pointer in it of type

	Cpp_PyObject *

and there is a single C wrapper function:

	int len (PyObject *object) {
		return ((C_Cpp_PyObject *) object)->len()
	}

which is in the 'len' slot in the CPython virtual table.
in this example, the C struct C_Cpp_Pyobject contains
a pointer to a Cpp_PyObject.

From bill.shui at eBioinformatics.com  Tue Dec 21 05:23:09 1999
From: bill.shui at eBioinformatics.com (Bill Shui)
Date: Tue, 21 Dec 1999 15:23:09 +1100
Subject: [C++-SIG] beginner
Message-ID: <385F00AD.3C485FEB@ebioinformatics.com>

Hi there,
	I'm an beginner-intermediate c++ programmer, and I've also been
programming in Python for over a year also . 
	I have also read through the mailing list archive.

	I'm working on a project as a hobbie, it also will help me to gain more
appreciation of C/C++ and Python.

	some friends of mine have written (not 100% completed yet) a
DBMS(engine, parser, the lot), and I've been thinking of writing api for
it and port it to python.

	but in order to do that, I'll need a better understanding of Python and
C++, I assume.

	so I was wondering, what are the essentials that I need to know in
order to be able to write C++ codes which can be imported by Python and
vice versa?


Your helps are much appreciated.


cheers
Bill Shui



From a.eyre at optichrome.com  Wed Dec 22 11:32:13 1999
From: a.eyre at optichrome.com (Adrian Eyre)
Date: Wed, 22 Dec 1999 10:32:13 -0000
Subject: [C++-SIG] beginner
In-Reply-To: <385F00AD.3C485FEB@ebioinformatics.com>
Message-ID: <000101bf4c67$cf21e230$3acbd9c2@peridot.optichrome.com>

> 	so I was wondering, what are the essentials that I need to know in
> order to be able to write C++ codes which can be imported by Python and
> vice versa?

Have a read through the Programming Python book on extending/embedding
Python. See also the examples in the source:

Demo/embed/*
Demo/extend/*
Modules/xxmodule.c

You might get a better response posting this to comp.lang.python...

--------------------------------------------
Adrian Eyre <mailto:a.eyre at optichrome.com>
Optichrome Computer Solutions Ltd
Maybury Road, Woking, Surrey, GU21 5HX, UK
Tel: +44 1483 740 233  Fax: +44 1483 760 644
http://www.optichrome.com 
--------------------------------------------