From farber at cpan.org  Wed Mar  1 01:40:16 2000
From: farber at cpan.org (Alex Farber)
Date: Wed, 01 Mar 2000 01:40:16 +0100
Subject: [C++-SIG] Reimplementing Lib/codeop.py in C fails...
Message-ID: <38BC66F0.A5518428@cpan.org>

Dear colleagues,

I am trying to create a custom "readline" function in C. I have looked 
at Modules/readline.c, Python/pythonrun.c and Parser/myreadline.c. I've 
also read the http://www.python.org/doc/current/api/api.html and many 
postings in Deja.com... I would very appreciate any hints.

Here is a simple C code reimplementing Lib/codeop.py for test purposes:

  int main(int argc, char* argv[])
  {
    char* line;
    PyObject *globals, *source;

    Py_Initialize();
    globals = PyDict_New();
    PyDict_SetItemString(globals, "__builtins__", PyEval_GetBuiltins());

  /*
    while (1) {
      line = readline(">>> ");
  */
      printf("\nCompiling <for i in [1,2,3]:>\n");
      source = Py_CompileString("for i in [1,2,3]:", "<stdin>", Py_file_input);
      if (source == NULL) PyErr_Print();

      printf("\nCompiling <for i in [1,2,3]:\\n>\n");
      source = Py_CompileString("for i in [1,2,3]:\n", "<stdin>", Py_file_input);
      if (source == NULL) PyErr_Print();

      printf("\nCompiling <for i in [1,2,3]:\\n\\n>\n");
      source = Py_CompileString("for i in [1,2,3]:\n\n", "<stdin>", Py_file_input);
      if (source == NULL) PyErr_Print();

  /*
      free(line);
    }
  */

    Py_DECREF(globals);
    Py_Finalize();
    exit(0);
  }

Which produces the following output:

  Compiling <for i in [1,2,3]:>
    File "<string>", line 1
      for i in [1,2,3]:
                     ^
  SyntaxError: unexpected EOF while parsing

  Compiling <for i in [1,2,3]:\n>
    File "<string>", line 1
      for i in [1,2,3]:
                      ^
  SyntaxError: unexpected EOF while parsing

  Compiling <for i in [1,2,3]:\n\n>
   File "<string>", line 2
     
     ^
  SyntaxError: invalid syntax

However the comment in Lib/codeop.py suggests, that the errors in 
the first and the second case should be different. How can I see 
this difference in my C code? Shouldn't the second string compile?

Regards
Alex

PS: The background is that I am programming a 3D-viewer in C++
    which uses some finite elements library (written in my university;
    not thread-safe), Qt (not thread-safe) and which embeds Python.

    Because of the not-thread-safeness I have to run FE, Qt and Python 
    in the same thread. But then I can not run PyRun_InteractiveLoop() 
    or PyRun_InteractiveOne() - the Qt would "freeze".

    So I am running a readline in another thread and send received
    strings to a UNIX-pipe. Those strings should be read by the first
    thread and executed with PyRun_SimpleString().

    Now I get the problem with the strings like "for i in [1,2,3]:" - 
    the Python needs more input. To detect such cases I was hoping to 
    mimic the Lib/codeop.py and then either save the received string 
    (if more input is needed) or run it with PyEval_EvalCode().



From pauldubois at home.com  Fri Mar  3 23:35:40 2000
From: pauldubois at home.com (Paul F. Dubois)
Date: Fri, 3 Mar 2000 14:35:40 -0800
Subject: [C++-SIG] CXX at SourceForge
In-Reply-To: <199712171747.JAA03008@curlew.geog.ubc.ca>
Message-ID: <NDBBIEFMILBFPMDHJIMFGEIECDAA.pauldubois@home.com>

I have CXX under CVS at sourceforge: see http://cxx.sourceforge.net.

This is version 4 which contains Barry Scott's rewrite of the extension
module/object portion. It is ill tested and less ported. It isn't backward
compatible in this area, beware. I will make a downloadable release next
week but for now you can get it by CVS.




From Thomas.Malik at t-online.de  Sat Mar  4 00:53:18 2000
From: Thomas.Malik at t-online.de (Thomas Malik)
Date: Sat, 4 Mar 2000 00:53:18 +0100
Subject: [C++-SIG] just to introduce me and some comments
Message-ID: <005401bf856b$ad522560$dfd79e3e@mjrtom>

Hello, since i am new to this group, ok, i'm Thomas.
I've taken the CXX package 3 months ago (don't know which version it was)
and changed many things to suit my needs. My requirements were:
- simple, homogeneous look of my wrapper classes, since i had to write a
whole lot of them
- use of inheritance in the wrapper classes, since the class library i had
to incorporate into python, had it, and i didn't want to do mass copies of
source code (every base class method in every subclass)
- the whole thing has to run rock solid, since it's going to be used for
interconnecting stock trading applications in a financial institute.
- I wanted to rely on C++ exceptions as error notification mechanism.

What i did:
I moved the inheritance of the PythonExtension template from PyObject 'one
level up', by introducing an intermediate class
CXXPyObject with a virtual destructor:

// The purpose of this is only to get a virtual destructor into a PyObject
and to
// be able to use dynamic_cast on it. PythonExtension inherits virtually
from this.
class CXXPyObject : public PyObject {
  protected:
    CXXPyObject() {}
  public:
    virtual ~CXXPyObject() {}
};

CXXObject is a virtual base class for PythonExtension.
Now, i can use dynamic_cast even in situations, where i inherit from
PythonExtension multiple times, see below.
Another thing i changed was to eliminate the template argument from
PythonType. It's only purpose was to get the Object size, but you can pass
this easily as a constructor argument, or even pass the size as a template
argument. The downside is, you have to keep PythonExtension as a template
class, since the constructor cannot determine the object size at
construction time.

To do easy, safe casting in the static member functions (Python entry
points) i introduced a template function
T python_cast<class T>(PyObject *). This does first a static_cast to the
CXXObject*, and then a dynamic_cast to the object i'm actually in. Of
course, this does NOT lead me magically into the paths of the right member
functions of the right class, but i'm using dynamic_cast & the virtual
function table for type checking, which gives me a fairly efficient type
safe conversion.

template<class T>
T python_cast(PyObject *p) {
  T result = dynamic_cast<T>(static_cast< CXXPyObject* >(p));
  if (result == NULL) {
    cerr << "!shit! cast failed!" << endl;
    throw bad_cast();
  }
  return result;
}

Lastly, i've added some 'standard' functionality into PythonType &
PythonExtension, such as str(), repr(), getattr().
I strictly obey the following rule in my source:
- Python entry points always have a name starting with 's_' (such as
s_str(PyObject* self, PyObject* a)  )
- the first lines look like
MyClassName* self=python_cast<MyClassName*>(s); // looks damn familiar,
doesn't it ?
Tuple args(a);

so, here's some chunk out of the PythonExtension template:
// since CXXPyObject is a virtual Baseclass of PythonExtension, you can
// safely use multiple inheritance with the following Scheme:
// class A : public PythonExtension<A> {
// };
// class B : public A, public PythonExtension<B> {
// }
//
// in A's an B's methods you may use the python_cast<A*>() thing below
// to get a A* or B* out of a PyObject *
template<class T>
class PythonExtension: public virtual CXXPyObject
{
  private:
    static void extension_object_deallocator(PyObject* t) {
      CXXPyObject* o=static_cast<CXXPyObject*>(t);
      delete o;
      // delete t;
    }

    explicit PythonExtension(const PythonExtension<T>& other);
    void operator=(const PythonExtension<T>& rhs);
  protected:
    explicit PythonExtension() {
      ob_refcnt = 1;
      ob_type = type_object();
    }
  public:
    virtual ~PythonExtension() {}

    static PythonType& behaviors() {
      static PythonType* p;
      if(!p) {
         p = new PythonType(sizeof(T));
         p->dealloc(extension_object_deallocator);
         p->str(s_str);
         p->repr(s_repr);
      }
...
}

If someone's interested; i can send the whole story(code).
BTW, I've also implemented some huge macros for easy (and safe!) use of
exceptions within C++ Modules. Note that you CAN'T throw exceptions
'through' the python interpreter. Consider this:

PyObject* mainEventLoop()
{
    PyObject* o=someObject;
    try {
        while (...)
            // some crufty event magic going on
            if (eventOccured())
                Py_CallMethod(o,"crashbang",...);
    }
    catch (const Exception&) {
    }
}

and 'crashbang()' calls a C++ method from within your module, which throws
an exception. My experience was that the process will go wreck, with the
default execption handler aborting the program. What i did is to write a
macro to catch these exceptions and to return Null instead.

Ok, that's it for today. Comments are welcome.

Thomas Malik






From vanandel at atd.ucar.edu  Mon Mar  6 22:52:35 2000
From: vanandel at atd.ucar.edu (Joe Van Andel)
Date: Mon, 06 Mar 2000 14:52:35 -0700
Subject: [C++-SIG] just to introduce me and some comments
References: <005401bf856b$ad522560$dfd79e3e@mjrtom>
Message-ID: <38C428A3.7CC62806@atd.ucar.edu>

Thanks for your work on CXX, since it sounds quite interesting.

(I've attempted to use CXX in the past, and run into problems with the
GNU compiler.  I'm told these should be fixed by now.)

Could you see how your version of CXX compares to the one just posted by
Paul Dubois to
cxx.sourceforge.net?  I'd love to see us have a single version of CXX
with the best ideas of everyone who has worked on it.

I'd sure like to see your safe casting and other fixes incorporated into
an "official" release.

Are you by any chance willing to "adopt" CXX, since I understand Paul is
not using C++ in his current job?



-- 
Joe VanAndel  	          
National Center for Atmospheric Research
http://www.atd.ucar.edu/~vanandel/
Internet: vanandel at ucar.edu



From pauldubois at home.com  Tue Mar  7 01:10:57 2000
From: pauldubois at home.com (Paul F. Dubois)
Date: Mon, 6 Mar 2000 16:10:57 -0800
Subject: [C++-SIG] CXX at Sourceforge
In-Reply-To: <200003030747.XAA18457@web2.i.sourceforge.net>
Message-ID: <NDBBIEFMILBFPMDHJIMFAEJGCDAA.pauldubois@home.com>

Gentleman,

I now have CXX-4.2 up at SourceForge. The web page is cxx.sourceforge.net.
This leads you to the project page where you can get at the CVS tree. I am
looking for one or more of you to take over as lead on this open source
project since I do not use CXX in my work since I changed jobs last year. It
is better to have someone who has to live with the consequences of their
actions.

4.2 has Barry's extension stuff. I think there isn't a great deal of
controversy about the CXX_Objects part, and Daniel has some fresh stuff to
contribute. But I would think a good first step is to get what is there out
by anonymous cvs or by downloading the release and see what changes we need
for compilation on various platforms. Once we have a stable and portable
base we can move on to the interesting questions about CXX_Extension.

I have sent this to each of you because I know that you use, have used, or
have developed variants on, CXX. Perhaps there are more people who could
contribute, so I will cc this to the c++ sig.

There are no ends of ideas and styles being put forward. Rather than just
talking about the way each of you sees it, I would like to focus things by
looking at what is there now and seeing if we understand exactly its
strengths and weaknesses. Opinions about whether something will or won't
compile or bloat or act badly should be backed up by actual data, not just
speculation. Maybe that way we can keep down the noise level.

Excuse this previous paragraph but several people have complained to me
about the signal to noise ratio in our discussions and I think it is
important to keep things focused.

Best regards,

Paul





From poinot at onera.fr  Tue Mar  7 12:05:00 2000
From: poinot at onera.fr (Marc Poinot)
Date: Tue, 07 Mar 2000 12:05:00 +0100
Subject: [C++-SIG] CXX at Sourceforge
References: <NDBBIEFMILBFPMDHJIMFAEJGCDAA.pauldubois@home.com>
Message-ID: <38C4E25C.5AC80FB5@onera.fr>

"Paul F. Dubois" wrote:
> 
> There are no ends of ideas and styles being put forward. Rather than just
> talking about the way each of you sees it, I would like to focus things by
> looking at what is there now and seeing if we understand exactly its
> strengths and weaknesses. Opinions about whether something will or won't
> compile or bloat or act badly should be backed up by actual data, not just
> speculation. Maybe that way we can keep down the noise level.
> 
The actual change of version for our CXX extension to CXX 4.2 has 
raised *no problem* (2 hours with GNU emacs). 
This almost was a syntactic translation, and
This new version is very nice, as we avoid now the C layer for argument
translation and initialization. Thanks to all developpers of CXX.

I hope you will not see this as noise, here's the list of things I
had to do to change our code, for about 35 classes of extensions.
Short and simplified examples (actual data?) are coming from our 
CGNS (Computational Fluid Dynamics data representation) extension class.
--------------------------------------------------------------
[1] Change the way to initialize the module
a- For Extension MODULE: Now change it to have a derived 
   class from ExtensionModule<T>, change init methods, etc...
   In other words make it as a class.

   class CGNS_module : public ExtensionModule<CGNS_module>
   {
    public:
     CGNS_module();
    private:
     Py::Object new_ExtCGNS(const Tuple &t) ;
   };

b- Use class-based method for initialization of the Extension CLASS
   but also for repr, getattr...
   * Make init_type() as static, otherwise you cannot reach it without
     an object instance.
  class ExtCGNS: public PythonExtension<ExtCGNS> 
  {
    public:
     ExtCGNS(STD::string wname);
    ~ExtCGNS();
     static void init(); // should be static
     ...

c- Move creation of new objects (new MyClass) into Extension MODULE
   See above new_ExtCGNS method.

[2] Remove all C stuff in the objects
a- change args with Tuple, remove tuple translation. We had the
   following macro, and the following use of the macro

#define  E_PYTHON_TO_CXX(aaa,eee,ttt) \
  Object * o##eee = new Object( aaa ); \
  ttt * eee ; eee = static_cast< ttt *>( o##eee ->ptr());

int
ExtCGNS_setattr(PyObject* self, char *name, PyObject *op)
{
  E_PYTHON_TO_CXX(self,x,ExtCGNS);
  
    if(STD::string(name)=="init") 
    {
      Array w(op);
      x->writeRefState(w);
    }
  return 0;
}

-----------------------------------
The code is now:

int ExtCGNS::setattr(const char *name, const Py::Object &o)
{
  if(STD::string(name)=="init") 
  {
    Array w(o);
    writeRefState(w);
  }
  return 0;
}

Great ! :) :)

b- returns with Py::Object
c- setattr/getattr/repr... now class methods
   * Why can't we have strings for setattr/getattr parameters?
   So that the code can be:

int ExtCGNS::setattr(const STD::string name, const Py::Object &o)
{
  if(name=="init") 
  {
    Array w(o);
    writeRefState(w);
  }
  return 0;
}

[3] Build
a- Add *simple* makefile (see end of this file)
b- compiler complains (warning) about CXX_Extensions.h, Line = 107 and 108
   * remove  ExtensionModuleBase:: qualifiers which are not necessary.
c- Use an extern "C" for init declaration of module.
   This is specific, as I put the init function into a C++ compiled file.


Sample makefile (SGI) for building a librarie with CXX

# ----------------------------------------------------
#
INCS=-I./Include -I/usr/local/include/python1.5
#
CCC=CC -64 -LANG:std
#
# use CC ar to instanciate templates (pre-linker)
AR=CC -ar -o
#
all:
	$(CCC) $(INCS) -c Src/cxx_extensions.cxx
	$(CCC) $(INCS) -c Src/cxxextensions.c
	$(CCC) $(INCS) -c Src/cxxsupport.cxx
	$(AR) libCXX.a *.o

clean:
	rm -f libCXX.a *.o
# ----------------------------------------------------


Marcvs [alias Long life CXX extensions...]



From Thomas.Malik at t-online.de  Tue Mar  7 19:19:39 2000
From: Thomas.Malik at t-online.de (Thomas Malik)
Date: Tue, 7 Mar 2000 19:19:39 +0100
Subject: [C++-SIG] just to introduce me and some comments
References: <005401bf856b$ad522560$dfd79e3e@mjrtom> <38C428A3.7CC62806@atd.ucar.edu>
Message-ID: <005601bf8861$b3fef8e0$0a1d9fc1@mjrtom>

-----Urspr?ngliche Nachricht-----
Von: Joe Van Andel <vanandel at atd.ucar.edu>
An: Thomas Malik <Thomas.Malik at t-online.de>
Betreff: Re: [C++-SIG] just to introduce me and some comments


[...}
> Could you see how your version of CXX compares to the one just posted by
> Paul Dubois to
> cxx.sourceforge.net?  I'd love to see us have a single version of CXX
> with the best ideas of everyone who has worked on it.
>
> I'd sure like to see your safe casting and other fixes incorporated into
> an "official" release.
>
> Are you by any chance willing to "adopt" CXX, since I understand Paul is
> not using C++ in his current job?
>
So, i do and will use the CXX package a lot, because it's what my current
work is based upon. I like it very much, because i think  it's the 'better'
way to integrate C++ APIs into Python. I mean, if one extends the
capabilities of a specific C++ API in a 'Python - compatible' manner, one
can take advantage from both: the dynamic typing and ease of use from
Python, and the efficiency and 'lower level' capabilities of C++.
So, yes, i am going to look into the newest CXX version and see how i can
integrate my changes into the 'baseline'.
Mainly, what i did, was this 'python_cast' stuff, so that you are able to
use inheritance and better type - safety, and a greater number of numeric
conversions in the Int, Long & Float classes from CXX_Objects.h.
The other things were this CATCH_ALL_EXCEPTIONS cpp Macro & a new Macro for
Argument checking (which gives you more Python - conformant error messages).

Thomas




From darylm at mail.mnet.com  Fri Mar 10 16:46:18 2000
From: darylm at mail.mnet.com (Daryl V. McDaniel)
Date: Fri, 10 Mar 2000 07:46:18 -0800
Subject: [C++-SIG] An introduction
Message-ID: <002601bf8aa7$cfb6d2e0$05ba8e0a@mnet.com>

Hello,

I would like to take this opportunity to introduce myself and briefly describe the project I am currently working on and its relevance to the C++ SIG.

A quick summary of my idealized goals is to achieve seamless, bi-directional, integration between the C++ and Python portions of our product allowing Python to be used as the product's scripting language and as a prototyping language during development.  And, to do this in such a way that my programmers don't have to write special interface code.

Now for some background.  I began programming in 1968 in SOL and Fortran.  LISP, Cobol, and several flavors of assembler followed in the '70s.  Somewhere around 1978 I discovered C and ended up writing a C compiler for the National 16032 processor family sometime around 1983.  In the late '80s I found out about C++ and began using it professionally in 1990.

About 2 months before Python 1.5.2 was released, I began looking for a scripting language to include in our product.  I started the search as a TCL advocate but ended up selecting Python, primarily because of its power and usability by non-programmers.  Python's modularity, embedding, and extension features were also prime decision points.

The product I am responsible for is actually just a small part of our client's complete product.  Its target environment is a distributed system of 1 to 64 nodes acting as a federation of autonomous intelligent agents.  Each node consists of an embedded processor running VxWorks with Python and the C++ application.  There is no disk, no virtual memory, and only a small amount of RAM (32MB).

This is a large, fast track project (which one isn't these days?) that had, in previous generations, been implemented entirely in C++.  The new incarnation is a 90% rewrite due to a vastly different operating environment and system requirements.  We essentially have to deliver about 800,000 new lines of working and tested code by December of this year.

The scripting component of the product must allow non-programmer users (Manufacturing and Service technicians), as well as software-savvy engineers, to write scripts to automate operations, debug systems, or modify operational procedures.  In order to do this, the Python components need access to the low-level C++ implementation.

[So far, I don't think I have said anything really new to anyone, its just background]

I read the section on embedding and extending in the Python documentation as well as the help and hints articles on python.org.  This led me to some of the wrapper packages and eventually to CXX, scxx, SWIG, sip, and the Extension Classes in Zope.  Of all of these, I like CXX best.  SWIG would meet all of my requirements except that the current "stable" version doesn't handle C++ well enough.  I haven't had time to reverse engineer the makefiles so that I could build one of the newer versions -- that is forthcoming.

So, I am currently hoping that between CXX and some documentation I have yet to write, we will be able to write and wrap our C++ code such that the C++ objects are available in Python, as true Python classes, and Pure Python functions and scripts will be available to the C++ code.

This is rapidly becoming more important as we begin prototyping in Python.  It now looks like the code we have to deliver in April will be mostly Python.  The July release to engineering will still be about 80% Python.  So, you can see that Python is taking on a larger role than "just an embedded scripting language".

What I currently believe would be "perfect" would be a merge of SWIG and CXX.  I am not going to be able to do this and won't suggest it be done.  Both SWIG and CXX are fine on their own rights and I don't want to start any wars.

CXX has taught me that I don't know C++, especially templates, as well as I should.  It is providing me with a very practical example to learn from.  It is also allowing my team to "write Python in C++".  The synergy is producing much more powerful software with less code.

Now we get to my possible contributions to Python and the SIG.

At a minimum, I believe the documentation I produce will be useful to anyone using CXX.  I may be able to expand upon the current work in CXX to make it easier to produce extension modules and classes.  I, of course, would make any CXX extensions I produce available to the community.

I hope that what I produce will be able to benefit the Python community.  To that end, I will probably be asking lots of questions.

Disclaimer:  I can't guarantee that the client's legal department agrees with any of my views.  They are currently examining the Python, Pyro, CXX, IDLE, JPython, and SWIG licenses to see if they will allow us to use any of them in a shippable product.  8-(

Sincerely,
--------
Daryl V. McDaniel               darylm at python.mnet.com    Micronetics
                                    (503) 621-0234        Aloha Research Group
                                                          Portland, Oregon  USA
   "Aloha Research Group", "Mnet", and "mnet"  are trademarks of Micronetics




From barry at scottb.demon.co.uk  Thu Mar  9 00:07:15 2000
From: barry at scottb.demon.co.uk (Barry Scott)
Date: Wed, 8 Mar 2000 23:07:15 -0000
Subject: [C++-SIG] RE: CXX at Sourceforge
In-Reply-To: <NDBBIEFMILBFPMDHJIMFAEJGCDAA.pauldubois@home.com>
Message-ID: <001901bf8953$0b8798f0$060210ac@private>

Paul,

	I am happy to take over CXX, as you say I am using CXX actively. It would
	be good to share the CXX work however.

	At the moment I'm working mostly in the Windows world and not doing much
	on Unix. It would be better to have someone working actively in the Unix
	world involved.

		Barry




From dan at cgsoftware.com  Sat Mar 11 21:49:40 2000
From: dan at cgsoftware.com (Daniel Berlin)
Date: Sat, 11 Mar 2000 12:49:40 -0800 (PST)
Subject: [C++-SIG] RE: CXX at Sourceforge
In-Reply-To: <001901bf8953$0b8798f0$060210ac@private>
Message-ID: <Pine.LNX.4.10.10003111249010.4348-100000@propylaea.anduin.com>

I'd happily be that person.
I use FreeBSD, as well as BeOS.
I use CXX very actively.
> Paul,
> 
> 	I am happy to take over CXX, as you say I am using CXX actively. It would
> 	be good to share the CXX work however.
> 
> 	At the moment I'm working mostly in the Windows world and not doing much
> 	on Unix. It would be better to have someone working actively in the Unix
> 	world involved.
> 
> 		Barry
> 




From gregc at cgl.ucsf.EDU  Mon Mar 13 04:46:10 2000
From: gregc at cgl.ucsf.EDU (Greg Couch)
Date: Sun, 12 Mar 2000 19:46:10 -0800 (PST)
Subject: [C++-SIG] ANN: wrappy -- a Python C++ interface generator
Message-ID: <200003130346.TAA508375@socrates.cgl.ucsf.edu>

I've finally packaged up wrappy into a form that can be redistributed.
Comments, contributions, questions, etc. are all welcome.

Wrappy takes a C++ header file and generates a Python module that
provides access to the classes and functions in the header file.
The goal is to write straight C++ and have wrappy take care of
generating the Python C interface.  Wrappy doesn't use CXX in the
generated code, but it might in the future.

If you wish to download wrappy, the url is:

	http://www.cgl.ucsf.edu/otf/wrappy/

The wrappy talk and associated paper from the 1999 O'Reilly Open
Source Conference is available from:

	http://conferences.oreilly.com/cd/python/presentations/gcouch/
or:
	http://www.cgl.ucsf.edu/home/gregc/oss99/

---
Greg Couch
UCSF Computer Graphics Labortory
gregc at cgl.ucsf.edu