[pypy-svn] rev 1976 - pypy/trunk/doc/funding

[arigo at codespeak.net]

Author: arigo
Date: Wed Oct 15 16:09:04 2003
New Revision: 1976

Modified:
   pypy/trunk/doc/funding/B1.0_objectives.txt
Log:
Rewritten and reordered, please review


Modified: pypy/trunk/doc/funding/B1.0_objectives.txt
==============================================================================
--- pypy/trunk/doc/funding/B1.0_objectives.txt	(original)
+++ pypy/trunk/doc/funding/B1.0_objectives.txt	Wed Oct 15 16:09:04 2003
@@ -89,37 +89,43 @@
 Interpreters Modularity
 ++++++++++++++++++++++++ 
 
-In its basics, the approach of writing a language interpreter in the
-language itself (a subset thereof) and then producing a running
-low-level code version trough a translation process has already been
-taken, e.g. for the Scheme (Scheme 48) [K97]_ and the Smalltalk
-(Squeak) [IKM97]_ languages. These approaches are typically based on
-translation from high-level source code parsed into ASTs to
-C. Obtaining in this way an interpreter comparable with current C
-Python implementation is within current state of the art, but we plan
-to exploit the gained flexibility much further, separating concerns
-between the translator, the code and Object Spaces encapsulating the
-core language and its semantics and further pluggable modules for
-both. We intend to use abstract interpretation as our analysis tool
-making the Object Space operations the semantics units also for
-translation, leveraging the coherence of our architecture and gaining
-independence from surface issues.
-
 Haskell monadic modular interpreters [LHJ95]_ [H98]_ are a
 researched attempt at achieving modularity for interpreters. They have
-not been tried on something as large as Python but in the context of DSLs, 
-however, as the approach is hard
+not been tried on something as large as Python but in the context of
+Domain-Specific Languages (DSLs). However, the approach is hard
 to relate to for programmers accustomed to more conventional Object-Oriented 
 (OO) programming and requires sophisticated partial evaluation to remove the
-monadic overhead. Monad transformers can also be used to modularize
-continuation passing, exceptions and other control flow aspects.
+monadic overhead. On the positive side, monad transformers are powerful enough
+to modularize continuation passing, exceptions and other control flow aspects.
 
 Some of the kind of modularity we are interested in for interpreters -
 subsetting, choice among implementation of basics aspects (memory
-management,...), feature selection - has some similarity with recent research on OS
-modularity and software composition, e.g the Flux Group OSKit project,
-see [RFSLE00]_ about their composition tool Knit.
- 
+management,...), feature selection - has some similarity with recent research
+on OS modularity and software composition, e.g. the Flux Group OSKit project
+and their composition tool Knit [RFSLE00]_.
+
+In its basics, the approach of writing a language interpreter in the
+language itself (a subset thereof) and then producing a running
+low-level code version through a translation process has already been
+taken e.g. for the Scheme (Scheme 48) [K97]_ and the Smalltalk
+(Squeak) [IKM97]_ languages. These approaches are typically based on
+translation from high-level source code (or parsed source code) into
+C. Obtaining in this way an interpreter comparable with the current C
+Python implementation would be within current state of the art; but note
+that our approach differs in three respects (as explained in more
+details in the next section): what we will translate is a dynamically loaded
+and initialized program state instead of the static source code; the translator
+works by abstract interpretation (also known as symbolic interpretation),
+which makes it independent from the actual source or bytecode; and we will
+put the key modularity and flexibility concerns into the translator.
+
+We plan to exploit the gained flexibility of the translator much further.
+It will enable separation on concerns between the translator, the
+core interpreter, and the Object Spaces, in the spirit of Aspect
+Oriented Programming (AOP) as developed in [KLM97]_. AOP separate cross-cutting
+aspects into separate components. This approach has however not been used on
+interpreters for large practical languages.
+
 
 JITs and Dynamic Optimisation Complexity
 ++++++++++++++++++++++++++++++++++++++++++
@@ -172,56 +178,60 @@
 Our architecture is based on Object Spaces and translation.  The
 interpreter's main dispatch loop handles control flow and supporting
 data structures (frame stack, bytecode objects...); each individual
-operation on objects is dispatched to the object space.
-
-Unlike the related work previously cited, we don't expect to encode a
-fixed single interpreter in all its details in a subset of our VHLL
-(Python). We plan to exploit the flexibility and abstraction gained by
-using the VHLL and -- most importantly -- the indirectness of translation,
-in order to weave aspect such as memory management, object layout,
-threading model etc. at translation time. 
-
-Many of these aspects are really cross-cutting concerns in the
-original Aspect Oriented Programming (AOP) sense. E.g. we expect to
-code addition between Python integers in high-level way independent
-memory management and of a concrete unboxing implementation. In
-general our approach relates to the seminal AOP ideas in [KLM97]_,
-although they have not been used on interpreters for large practical
-languages. The subset of Python in which we express the core
-interpreter and Object Spaces is the component language in the
-terminology of the paper, while the translator is a weaver.  For
-describing aspects and at system definition time we will be able to
-use the full dynamism of Python.
-
-In general we will explore for each feature and extension how to best
-implement it by separating concerns between OO-customized Object
-Spaces, other modules also translated, or the pluggable behavior of
-the translation itself.
+operation on objects is dispatched to the Object Space.
 
 Object Spaces, in contrast to monadic interpreters, will allow
 customization with OO techniques, as they encapsulate the object
 operation semantics, creation and global state of all objects. As
 mentioned above, monad transformers can be used to modularize
 continuation passing, exceptions and other control flow aspects. We
-expect to encapsulate those in an interpreter loop to be also
-translated or to implement them as aspects of the translation process,
-for example, producing continuation-passing code.
-
-For debugging and comprehensibility we expect the core interpreter and
-Object Spaces to be runnable as a normal Python program on the
-reference CPython interpreter.
+expect to encapsulate each of those either in Object Spaces, in the interpreter
+loop, or as aspects of the translation process (which would then generate
+for example continuation-passing code).
+
+This latter point is in contrast to the related work previously cited:
+we don't expect to encode a fixed single interpreter in all its details in a
+subset of our VHLL (Python). We plan to exploit the flexibility and abstraction
+gained by using the VHLL and -- most importantly -- the indirectness of
+translation in order to "weave" aspects such as continuation passing, memory
+management, object layout, threading model etc. at translation time.
+
+Many of these aspects are really cross-cutting concerns in the
+original Aspect Oriented Programming (AOP) sense. E.g. we expect to
+code addition between Python integers in a high-level way independent
+of memory management and of boxing issues. In
+general our approach relates to the seminal AOP ideas of [KLM97]_:
+the subset of Python in which we express the core
+interpreter and Object Spaces is the *component language* in the
+terminology of the paper, while the translator is a *weaver* written
+with the full dynamism of Python.
+
+In summary, we will explore for each feature and extension how to best
+implement it by separation of concerns: either in OO-customized Object
+Spaces, or in the core or in modules to be translated, or as a pluggable
+behavior of the translation itself.
 
 The front-end of the translator itself will be innovative in that it is
 based on abstract (symbolic) interpretation. The translation of code
 from our Python subset (in particular the source of PyPy itself) will
 thus be driven by PyPy's own Python interpreter, made symbolic by
-plugging a custom Object Space. Weaving custom aspects will be done
-mainly as intermediate stages, and by customizing the back-end to
-generate low-level code in various styles as needed. For example, it can
-be in continuation passing style (CPS), or it can target altogether
-different runtime environments like Java -- the latter is a cheap way to
-regenerate and subsequently maintain a replacement for Jython, the
-Python interpreter written in Java.
+plugging a custom Object Space. This turns the Object Space operations
+into the semantics units of translation as well, leveraging the coherence
+of our architecture and gaining independence from surface issues including
+the details of the bytecode itself.
+Moreover, we can use the full dynamism of Python at system definition time,
+i.e. when PyPy loads, until we reach a "stable-and-ready" state that the
+translator can freeze into a snapshot containing exactly the features and
+modules that we need. In previous work, the translator only operated on the
+static source code.
+
+During the translation process, weaving custom aspects will be done mainly
+as intermediate stages, while the customizable back-end generates low-level
+code in various styles as needed. For example, it can be in continuation
+passing style (CPS), or it can target altogether
+different runtime environments like Java -- providing a cheap way to
+regenerate a replacement for Jython, the Python interpreter written in Java,
+without the burden of having to maintain it synchronized with CPython.
 
 Another key innovative aspect of our project is to generate the JIT
 compiler instead of writing it manually. Indeed, although Psyco was