[pypy-dev] question about core implementation language

[Christian Tismer]

Edward K. Ream wrote:
> I have started some prototyping in Python, and I now
> remember some words of Guido, that Python isn't specially
> well suited for such a task. I admit, there is some
> truth in it.
> 
> 
> I don't understand Guido's comment.  I personally think Python is absolutely
> superb for any prototyping endeavor.

It is not superb if you want to directly map stuff
from a completely different language, that C is.

But as I said, this is only tempting.
My conclusions are quite different, tho.

...

> Perhaps I am confusing levels of discussion or implementation.  However, the
> way I see it, the most important thing is to do experimentation with
> algorithms.  To do that, the essential thing is to compare various ways of
> doing things, at as high a level as possible.  In other words, a
> psyco_switch routine provides a good enough model for the task at hand.
> Ditto for any other construct you want.

Yes, I agree. That's not the problem.
The problem is simply this:
The algorithms written in C are written very
very well. This is highly tested code, evolved
for years, and it is as good as thinkable, given
the known restrictions, like that it has to be C.

My (our) problem is now, that we have to transliterate
this quality code into Python, hopefully not destroying
anything. Alongside, whe have to invent lots of new
objects on-the-fly, as there appear so many structs
and things, while you try to write a pythonic Python
implementation of a C module.
I tried this today with a module which I felt urgent
to try to define for this project: frameobject.c .
This module is less than 1000 lines, and it is one of
the C modules which I know the best, due to my stackless
work.
I tried to map this module on a 3.5 hour yourney from
Kiel to Berlin, and I had one fourth done by an hour.
Nevertheless, I got into trouble, just by comparing its
implementation differences between 2.2.2 and 2.3a.
Then I dropped that and decided that this is the wrong way.

It is not a simple task to do a good mapping into Python,
although I know exactly how I would map this and that.
But only the fact that the python-dev crew is working
hard on all the C code makes them into our biggest enemies:
The are changing the C code all the time, in all places,
getting us into the role to program after them all the time.

Between the lines, I recognized that I still would know
the mapping that I wanted to apply from C to Python.
I just hate to write this by hand, for similar reasons
why I abandones Stackless 1.0 long time ago, simply
since keeping track of changes is a PITA, unless *your*
code is in the core.

So what I am thinking to start instead is quite different,
with the same target, but one level higher.
I will post that after first experiments.

> Actually, for experimentation (perhaps leading to inspiration) I wouldn't
> necessarily confine myself to any particular level of implementation or
> design or whatever.  I would simply blast away using whatever language tools
> appear to be most useful.  Armin has talked about doing high-level work with
> discovering list optimizations, for example.  I don't understand what the
> lack of a Python switch statement has to do with such matters.

Simply try to implement something.
Yes, it is fine. Try to re-implement ceval.c in an afternoon.
It is 3900 lines of code, well-written C code, a huge switch
statement, and you find yourself writing all those case cases
as tiny functions inside a class. Sure it is possible.
What the matter is? I have to change something substantial,
there is no trivial mapping, but a more complicated one.

> In short, I would hardly bother at all with questions of the level at which
> Python is modeling some implementation.  I would use whatever works in
> Python and worry about mapping that back to a final implementation only
> after it is clear that Python in Python will be a big win.  At that time we
> will have lots more data, energy and incentive to do the needed grunt work.
> Until then, I wouldn't let implementation problems get in the way of
> invention.

I had the same POV before I started to try coding.
You are absolutely right, the implementation does
not matter so much.
In extent, if it doesn't matter too much, then there
is no need to do this by hand.

Please give me two days to try this, then I will
hopefully show a way how to do it *not by hand*.

all the best -- chris   (too old to do this by hand)