[pypy-dev] stdobjspace summary?

[Armin Rigo]

Hello everybody,

As I won't read my mail during the whole next week I thought it might be
helpful if I attempted some kind of summary. It collects numerous ideas posted
here (recently or not) and a bit of personal feelings. I hope it might help in
getting some distance over what we are actually trying to do with our
"standard object space". I am afraid it is all very vague, but should be kept
in mind before we actually design something new there.


The std/ directory is the "space of compliant Python object implementations",
as opposed to non-compliant object spaces which do some more funny things
while the interpreter follows the bytecode. It collects various
implementations for the same user types, which are based on some lower-level
abstractions. The abstractions that are used are explicitely described in each
file, for example: N-bits or machine-sized signed or unsigned integers; memory
blocks with explicit management; bitfield objects; various hints (e.g. "this
implementation should not be used with refcounts because it has many
circularities"); or even which "RPython level" we use (if we develop more and
more complex translators), including what basic type operations we allow.

Then each low-level abstraction itself has a reference implementation in
Python, like Christian's r_int; this should be put in a file that can easily
be associated to alternate implementations of the same concept, the ones that
can be used by the translator (like the description of the C 'int' type and
the tedious complexities of overflow detection in C). These low-level
implementations can also explicitely depend on other low-level
implementations: for example, "C lists with stored length" could be
implemented in term of malloc-style memory blocks -- as opposed to "lists as
Java arrays" which don't go lower-level because the length is always stored in
Java arrays.

"Above" all these files, we have several compliant object spaces, whose
purpose is to link a selection of implementations together. For testing
purposes object spaces should be easy to build and run in Python, thanks to
the Python reference implementations of low-level abstractions. On the other
hand, the "C object space" links some or all the files which are (indirectly)
based on C-implemented low-level abstractions only; this is what the
translator will input.

Which files an object space depends on should be specified declaratively, in
such a way that we can actually use this information to implement the "wrap",
"newlist", "newstring",... methods of the object space. These are the
functions that build W_XxxObject instances of the basic types from scratch.

Let me stress again that the maximum flexibility seems to be to allow each
concept to be implemented in possibly several ways above lower-level concepts,
including new ones if the existing ones don't suit you. The arguments on the
mailing list show nicely enough that there is just no single better way to
implement things, so we should just allow them all to coexist[1]. After that,
*other* files are responsible for putting the pieces together -- explicitely
(listing the files to include), implicitely (automatically searching for
implementations that meet some requirements), or even dynamically at run-time
to optimize the performances (not easy, but at least it should be *possible*)!


[1] is a departure from one of Python's moto: "there is one and only one
obvious way to do something". Well, I assume that I won't shock people around
here if I say I never liked this one :-)


A bientôt,

Armin.