[Python-Dev] Gradual migration

[M.-A. Lemburg]

Paul Prescod wrote:
> 
> "M.-A. Lemburg" wrote:
> >
> > ...
> >
> > >  * implementation language and Python inheritance semantics are almost
> > > completely distinct. After all, we have Python implementations in a
> > > variety of languages with minor incompatibilities. Python could have a
> > > proper type/class merging even if it were written in assembly language.
> > > So let's leave implementation language aside.
> >
> > Hey, think of it as opportunity: we can reuse much of C++'s
> > optimizations and the integration of Python and C++ applications
> > will get *much* easier.
> 
> We could make integrating C++ and Python easier through CXX or Py_Cpp.
> Perhaps we should ship one of those with Python 2.1. Anyhow, integrating
> C++ and Python is not really so hard, in my experience.

Ok, you can use SWIG to get most of the work done, but the interface
between the C++ object world and the Python object world is one
big mess -- all implementations I've seen use shadow objects
or proxies to interface from one to the other... with lots of
temporary objects used for the linkup.

Having Python itself written in C++ we could do *much* better.

But that's only a nice side-effect. The real argument here is
that we can push the type logic one layer further down. Ideal would
be a level of integration such as the one implemented in JPython
or Jython.
 
> > A rewrite shouldn't scare anyone away -- much of the existing
> > code can be reused since only the Python C APIs of the various
> > types will have to be rewritten, not the logic behind the types.
> 
> I wasn't really addressing the issue of backwards compatibility of
> extensions -- I was talking about Python programs. Nevertheless, I can't
> resist:
> 
> Porting your app to the Python APIs is often the majority of the work in
> a particular extension. A lot of Python extensions consist only of API
> glue!

True, but we're lucky, since we could provide a compatibility
layer on top of the new API.

BTW, I think I now know what your main concern is: the Python
level compatibility. I was talking of what goes on under the
hood and still think that Py3K should be used as a chance to
simplify the Python backend. As simplification often means
generalization, we'll open up new doors for future developments
along the way.

> > > I don't know what you mean by a "low-level strategy" for classes. It's
> > > not like we can somehow use C++ vtables without giving up Python's
> > > dynamicity. Python's class handling is about as low-level as it can get
> > > in my humble opinion.
> >
> > With "low-level" I meant trying to build Python classes and
> > instances on top of a very thin layer on top of C++ classes,
> > e.g. all slots could be implemented using true C++ methods with
> > additional logic to override them using dynamically bound
> > Python method objects.
> 
> I don't think that there is really interesting magic in a C++ compiler.
> A vtable is an array of pointers to functions. We've already implemented
> the equivalent in ANSI C. C++ exceptions, constructors, destriuctors and
> smart pointers are a little more interesting from a code maintenance and
> simplicity point of view. But even then I think that we could get to a
> C++ implementation through incremental rewrites.

Naa, the whole type slot interface is one big mess (sorry, Guido :-).
some slots are packaged, some are not, some are NULL, some are not,
there are oodles of sometimes weird dependencies between the
slots which are not really documented, etc. etc.

The slot design has serious drawbacks and should be replaced by
something more reliable, preferably C++ methods. That way, we'll
get some more "type" safety into Python and its extensions.

Note that porting old extensions won't be all that hard: 
a class reusing the existing slot functions as methods should
suffice in many cases.

> > Why spoil all the fun when we haven't even started thinking
> > about all the possibilities we could use to make Py3k a
> > success ?
> 
> Asking for a map of where we are going and how we will get here is
> "spoiling all of the fun?" I'm not sure what you are reacting to -- I
> didn't advise one implementation strategy or another -- I just said that
> we should employ a strategy that minimizes sudden and surprising code
> breakage.

Ok, we've been talking about different things here: with "spoiling
the fun" I meant putting ropes on possible changes to the C 
backend. 

You are talking about the frontend and I agree with you
that there should be a clear upgrade path from the 2.x series
to Py3K w/r to the Python side of things.

So I guess it's time for some PEPs now... the upgrade path
PEP and the fluffy clouds PEP.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/