[pypy-svn] r10781 - pypy/dist/pypy/documentation
hpk at codespeak.net
hpk at codespeak.net
Sun Apr 17 19:09:04 CEST 2005
Author: hpk
Date: Sun Apr 17 19:09:04 2005
New Revision: 10781
Removed:
pypy/dist/pypy/documentation/restrictedpy.txt
pypy/dist/pypy/documentation/testdesign.txt
Modified:
pypy/dist/pypy/documentation/annotateobjspace.txt
pypy/dist/pypy/documentation/coding-style.txt
pypy/dist/pypy/documentation/redirections
Log:
first steps refactoring according to 'newstructure'
restrictedpy and testdesign are merged with coding-style.txt
Modified: pypy/dist/pypy/documentation/annotateobjspace.txt
==============================================================================
--- pypy/dist/pypy/documentation/annotateobjspace.txt (original)
+++ pypy/dist/pypy/documentation/annotateobjspace.txt Sun Apr 17 19:09:04 2005
@@ -23,4 +23,4 @@
------------------------------
.. _StandardObjectSpace: stdobjspace.html
-.. _RestrictedPython: restrictedpy.html
+.. _RestrictedPython: coding-style.html
Modified: pypy/dist/pypy/documentation/coding-style.txt
==============================================================================
--- pypy/dist/pypy/documentation/coding-style.txt (original)
+++ pypy/dist/pypy/documentation/coding-style.txt Sun Apr 17 19:09:04 2005
@@ -1,10 +1,251 @@
=====================================
-Coding Style
+PyPy Coding Style
=====================================
+.. contents::
+.. sectnum::
-Naming and environment
------------------------------
+Restricted Python
+==================
+
+We are writing a Python interpreter in Python, using Python's well known ability
+to step behind the algorithmic problems as language. At first glance, one might
+think this achieves nothing but a better understanding for everybody how the
+interpreter works. This alone would make it worth doing, but we have much larger
+goals.
+
+
+CPython vs. PyPy
+-------------------
+
+Compared to the CPython implementation, Python takes the role of the C Code. So
+actually, we describe something by Python, which has been coded in C already,
+with all the restrictions that are implied by C. We are not trying to make the
+structures of the CPython interpreter more flexible by rewriting things in C,
+but we want to use Python to give an alternative description of the interpreter.
+
+The clear advantage is that this description is probably shorter and simpler to
+read, and many implementation details vanish. The drawback of this approach is
+that this interpreter will be unbearably slow.
+
+To get to a useful interpreter again, we need to apply some mappings to the
+implementation, later. One rather straight-forward is to do a whole program
+analysis of the PyPy interpreter and create a C source, again. There are many
+other ways, but let's stick with the easiest approach, first.
+
+In order to make a C code generator simple, we restrict ourselves to a subset of
+the Python language, and we adhere to some rules, which make code generation
+obvious and easy.
+
+
+Restricted Python is Runtime Python
+-------------------------------------
+
+Restricted Python describes a runnable Python interpreter implementation. This
+is a quite static object that can be suitably described by RPython. But the
+restrictions do not apply during the startup phase.
+
+
+PyPy Bootstrap
+-------------------
+
+When the PyPy interpreter is started as a CPython program, it can use all of
+CPython for a while, until it reaches runtime. That is, all executable code will
+be executed using the full power of Python.
+
+An example can be found in the implementation, which is quite elegant: For the
+definition of all the opcodes of the Python interpreter, the module dis is
+imported and used. This saves us from adding extra modules to PyPy. The import
+code is run at startup time, and we are allowed to use the CPython builtin
+import function.
+
+When the startup code is done, all resulting objects, functions, code blocks
+etc. must adhere to the runtime restrictions. All initialized modules are
+written out in a persistent manner. Our current idea is to emit a huge C source
+file which contains everything created so far. During this process, a whole
+program analysis is performed, which makes use of the restrictions defined in
+RPython. This enables the code generator to emit efficient replacements for pure
+integer objects, for instance.
+
+
+RPython Definition
+--------------------
+
+It might make sense to define a sub-language of Python called RPython, with the
+restrictions depicted below. This is an evolving topic, and we're just
+collecting things which come up during trying to code the interpreter, so this
+is no language at all, but an arbitrary set of rules, which are about to be
+changed all day.
+
+
+Object restrictions
+-------------------------
+
+We are using
+
+**variables**
+
+ the same variable in the same context can receive values of different types,
+ at a possible overhead cost. For example, a variable that can contain a
+ wrapped object or None is efficiently implemented as a PyObject* pointer that
+ can be NULL, but a variable that can contain either an integer or a float must
+ be implemented as a union with a type tag in C.
+
+**constants**
+
+ all module globals are considered constants.
+
+**integer, float, string, boolean**
+
+ avoid string methods and complex operations like slicing with a step
+
+**tuples**
+
+ no variable-length tuples; use them to store or return pairs or n-tuples of
+ values
+
+**lists**
+
+ lists are used as an allocated array; list.append() does naive resizing, so as
+ far as possible use list comprehensions (see below). list.extend() or the +=
+ operator are allowed and efficient. Unless there is really a use case for it,
+ repetition is limited to initialization purposes: '[single_value] * length'.
+
+**dicts**
+
+ dicts with string keys only (preferrably the kind of strings that are usually
+ interned in CPython, i.e. short strings that look like identifiers). The
+ implementation could safely decide that all dict keys should be interned.
+
+**control structures**
+
+ all allowed
+
+**list comprehensions**
+
+ may be used to create allocated, initialized array. the array size must be
+ computable in advance, which implies that we don't allow an if clause.
+
+**functions**
+
++ statically called functions may use defaults and a variable number of
+ arguments (which may be passed as a list instead of a tuple, so write code that
+ does not depend on it being a tuple).
+
++ dynamic dispatch enforces use of very simple signatures, equal for all
+ functions to be called in that context. At the moment, this occurs in the opcode
+ dispatch, only.
+
+**builtin functions**
+
+ A few builtin functions will be used, while this set is not defined
+ completely, yet. Some builtin functions are special forms:
+
+**range**
+
+ does not create an array. It is only allowed in for loops. The step argument
+ must be a constant.
+
+**len**
+
++ may be used with basic types that have a length. But len is a special form
+ that is recognized by the compiler.
+
++ If a certain structure is never touched by len, the compiler might save the
+ length field from the underlying structure.
+
+``int, float, ord, chr``... are available as simple conversion functions.
+
+``int, float, str``... have a special meaning as a type inside of isinstance only.
+
+**classes**
+
++ methods and other class attributes do not change after startup
++ inheritance is supported
++ classes are first-class objects too
+
+**exceptions**
+
++ fully supported
++ see below for restrictions on exceptions raised by built-in operations
+
+**objects**
+
+ wrapped objects are borrowed from the object space. Just like in CPython, code
+ that needs e.g. a dictionary can use a wrapped dict and the object space
+ operations on it.
+
+This layout makes the number of types to take care about quite limited.
+
+
+Example: Integer Types
+-------------------------
+
+While implementing the integer type, I (Chris) stumbled over the problem, that
+integers are quite in flux in CPython right now. Depending on the version,
+integers either raise an overflow exception or mutate into longs on overflow.
+Also, shifting right now truncates (up to 2.3) but is going to extend to longs
+as well. In order to enable us to run the restricted Python stuff in CPython, I
+needed to use a derived class r_int(int), which always behaves the same: Never
+leaving its domain, but always staying an integer.
+
+The r_int type is implemented in a pervasive way: Every operation that involves
+an r_int creates an r_int as the result. Therefore, the implementation is not
+cluttered with special type casts. Just the initial value of an emulated
+integer's intval field is initialized by obj.intval = r_int(val) . This way, the
+r_int type propagates itself through all operations without extra effort of the
+programmer.
+
+This concept looks promising, and since we will need unsigned integers which do
+not overflow as well, I also created r_uint. It is always a word-sized unsigned
+integer and never overflows. This will be a building block for things like
+computing hash values, where wrap-around effects are intended and should be
+easily coded without lots of explicit mask shuffling.
+
+Now I think to extend this even more and build a full set of primitive types,
+which are intended to
+
++ define the semantics of the primitive type
++ give a working implementation for unrestricted Python
+
+These primitive types can later easily be augmented with methods to emit C code instead of executing. I guess this would be implemented in an extra ObjectSpace.
+
+
+Exception rules
+---------------------
+
+Exceptions are by default not generated for simple cases.::
+
+ #!/usr/bin/python
+
+ x = 5
+ x = x + 1 # this code is not checked for overflow
+
+ try:
+ x = x + y
+ except OverflowError:
+ # implement using longs
+
+
+Code with no exception handlers does not raise exceptions. By supplying an
+exception handler, you ask for error checking. Without, you assure the system
+that the operation cannot overflow.
+
+Exceptions explicitly raised will always be generated.
+
+PyPy is debuggable on top of CPython
+------------------------------------
+
+PyPy has the advantage that it is runnable on standard
+CPython. That means, we can run all of PyPy with all exception
+handling enabled, so we might catch cases where we failed to
+adhere to our implicit assertions.
+
+Naming and development conventions
+==================================
+
+Naming
+------
- directories/modules/namespaces are always **lowercase**
@@ -22,6 +263,17 @@
- it's appreciated if you manage to name files in a directory
so that tab-completion on the shell level is as easy as possible.
+- objectspace classes are always spelled "ObjSpace". e.g.
+
+ - StdObjSpace
+ - FlowObjSpace
+
+- at interpreter level and in ObjSpace all boxed values
+ have a leading ``w_`` to indicate "wrapped values". This
+ includes w_self. Don't use ``w_`` in application level
+ python only code.
+
+
Committing
----------
@@ -33,18 +285,42 @@
that the property 'svn:eol-style' is set to native which
allows checkin/checkout in native line-ending format.
-Object Spaces
----------------
-- objectspace classes are always spelled "ObjSpace". e.g.
+Test Design
+=============
- - StdObjSpace
- - FlowObjSpace
+Our tests are based on the new `py.test`_ tool which lets you write
+unittests without boilerplate. All tests of modules
+in a directory usually reside in a subdirectory **test**. There are
+basically two types of unit tests:
-- at interpreter level and in ObjSpace all boxed values
- have a leading ``w_`` to indicate "wrapped values". This
- includes w_self. Don't use ``w_`` in application level
- python only code.
+- **Interpreter Level tests**. They run at the same level as PyPy's
+ interpreter.
+
+- **Application Level tests**. They run at application level which means
+ that they look like straight python code but they are interpreted by PyPy.
+
+Both types of tests need an objectspace they can run with (the interpreter
+dispatches operations on objects to an objectspace). If you run a test you
+can usually give the '-o' switch to select an object space. E.g. '-o thunk'
+will select the thunk object space. The default is the "Standard Object Space"
+which aims to implement unmodified Python semantics.
+
+.. _`py.test`: http://codespeak.net/py/current/doc/test.html
+
+Writing a test
+--------------
+
+Currently the best reference is to go to some test files and look how they are done.
+
+Command line tool test_all
+--------------------------
+
+You can run almost all of PyPy's tests by invoking::
+
+ python test_all.py
+
+For switches to modify test execution invoke "python test_all.py -h".
Test conventions
----------------
@@ -60,7 +336,3 @@
- each test directory needs a copy of pypy/tool/autopath.py which
upon import will make sure that sys.path contains the directory
where 'pypy' is in.
-
-- see some more information about tests at the test-design_ document.
-
-.. _test-design: testdesign.html
Modified: pypy/dist/pypy/documentation/redirections
==============================================================================
--- pypy/dist/pypy/documentation/redirections (original)
+++ pypy/dist/pypy/documentation/redirections Sun Apr 17 19:09:04 2005
@@ -1,5 +1,6 @@
# please make sure this is evaluatable
{
- 'aaa.html': 'architecture.html',
+ 'restrictedpy.html': 'coding-style.html',
+ 'testdesign.html' : 'coding-style.html',
}
Deleted: /pypy/dist/pypy/documentation/restrictedpy.txt
==============================================================================
--- /pypy/dist/pypy/documentation/restrictedpy.txt Sun Apr 17 19:09:04 2005
+++ (empty file)
@@ -1,238 +0,0 @@
-==================
-Restricted Python
-==================
-
-We are writing a Python interpreter in Python, using Python's well known ability
-to step behind the algorithmic problems as language. At first glance, one might
-think this achieves nothing but a better understanding for everybody how the
-interpreter works. This alone would make it worth doing, but we have much larger
-goals.
-
-
-CPython vs. PyPy
--------------------
-
-Compared to the CPython implementation, Python takes the role of the C Code. So
-actually, we describe something by Python, which has been coded in C already,
-with all the restrictions that are implied by C. We are not trying to make the
-structures of the CPython interpreter more flexible by rewriting things in C,
-but we want to use Python to give an alternative description of the interpreter.
-
-The clear advantage is that this description is probably shorter and simpler to
-read, and many implementation details vanish. The drawback of this approach is
-that this interpreter will be unbearably slow.
-
-To get to a useful interpreter again, we need to apply some mappings to the
-implementation, later. One rather straight-forward is to do a whole program
-analysis of the PyPy interpreter and create a C source, again. There are many
-other ways, but let's stick with the easiest approach, first.
-
-In order to make a C code generator simple, we restrict ourselves to a subset of
-the Python language, and we adhere to some rules, which make code generation
-obvious and easy.
-
-
-Restricted Python is Runtime Python
--------------------------------------
-
-Restricted Python describes a runnable Python interpreter implementation. This
-is a quite static object that can be suitably described by RPython. But the
-restrictions do not apply during the startup phase.
-
-
-PyPy Bootstrap
--------------------
-
-When the PyPy interpreter is started as a CPython program, it can use all of
-CPython for a while, until it reaches runtime. That is, all executable code will
-be executed using the full power of Python.
-
-An example can be found in the implementation, which is quite elegant: For the
-definition of all the opcodes of the Python interpreter, the module dis is
-imported and used. This saves us from adding extra modules to PyPy. The import
-code is run at startup time, and we are allowed to use the CPython builtin
-import function.
-
-When the startup code is done, all resulting objects, functions, code blocks
-etc. must adhere to the runtime restrictions. All initialized modules are
-written out in a persistent manner. Our current idea is to emit a huge C source
-file which contains everything created so far. During this process, a whole
-program analysis is performed, which makes use of the restrictions defined in
-RPython. This enables the code generator to emit efficient replacements for pure
-integer objects, for instance.
-
-
-RPython Definition
---------------------
-
-It might make sense to define a sub-language of Python called RPython, with the
-restrictions depicted below. This is an evolving topic, and we're just
-collecting things which come up during trying to code the interpreter, so this
-is no language at all, but an arbitrary set of rules, which are about to be
-changed all day.
-
-
-Object restrictions
--------------------------
-
-We are using
-
-**variables**
-
- the same variable in the same context can receive values of different types,
- at a possible overhead cost. For example, a variable that can contain a
- wrapped object or None is efficiently implemented as a PyObject* pointer that
- can be NULL, but a variable that can contain either an integer or a float must
- be implemented as a union with a type tag in C.
-
-**constants**
-
- all module globals are considered constants.
-
-**integer, float, string, boolean**
-
- avoid string methods and complex operations like slicing with a step
-
-**tuples**
-
- no variable-length tuples; use them to store or return pairs or n-tuples of
- values
-
-**lists**
-
- lists are used as an allocated array; list.append() does naive resizing, so as
- far as possible use list comprehensions (see below). list.extend() or the +=
- operator are allowed and efficient. Unless there is really a use case for it,
- repetition is limited to initialization purposes: '[single_value] * length'.
-
-**dicts**
-
- dicts with string keys only (preferrably the kind of strings that are usually
- interned in CPython, i.e. short strings that look like identifiers). The
- implementation could safely decide that all dict keys should be interned.
-
-**control structures**
-
- all allowed
-
-**list comprehensions**
-
- may be used to create allocated, initialized array. the array size must be
- computable in advance, which implies that we don't allow an if clause.
-
-**functions**
-
-+ statically called functions may use defaults and a variable number of
- arguments (which may be passed as a list instead of a tuple, so write code that
- does not depend on it being a tuple).
-
-+ dynamic dispatch enforces use of very simple signatures, equal for all
- functions to be called in that context. At the moment, this occurs in the opcode
- dispatch, only.
-
-**builtin functions**
-
- A few builtin functions will be used, while this set is not defined
- completely, yet. Some builtin functions are special forms:
-
-**range**
-
- does not create an array. It is only allowed in for loops. The step argument
- must be a constant.
-
-**len**
-
-+ may be used with basic types that have a length. But len is a special form
- that is recognized by the compiler.
-
-+ If a certain structure is never touched by len, the compiler might save the
- length field from the underlying structure.
-
-``int, float, ord, chr``... are available as simple conversion functions.
-
-``int, float, str``... have a special meaning as a type inside of isinstance only.
-
-**classes**
-
-+ methods and other class attributes do not change after startup
-+ inheritance is supported
-+ classes are first-class objects too
-
-**exceptions**
-
-+ fully supported
-+ see below for restrictions on exceptions raised by built-in operations
-
-**objects**
-
- wrapped objects are borrowed from the object space. Just like in CPython, code
- that needs e.g. a dictionary can use a wrapped dict and the object space
- operations on it.
-
-This layout makes the number of types to take care about quite limited.
-
-
-Example: Integer Types
--------------------------
-
-While implementing the integer type, I (Chris) stumbled over the problem, that
-integers are quite in flux in CPython right now. Depending on the version,
-integers either raise an overflow exception or mutate into longs on overflow.
-Also, shifting right now truncates (up to 2.3) but is going to extend to longs
-as well. In order to enable us to run the restricted Python stuff in CPython, I
-needed to use a derived class r_int(int), which always behaves the same: Never
-leaving its domain, but always staying an integer.
-
-The r_int type is implemented in a pervasive way: Every operation that involves
-an r_int creates an r_int as the result. Therefore, the implementation is not
-cluttered with special type casts. Just the initial value of an emulated
-integer's intval field is initialized by obj.intval = r_int(val) . This way, the
-r_int type propagates itself through all operations without extra effort of the
-programmer.
-
-This concept looks promising, and since we will need unsigned integers which do
-not overflow as well, I also created r_uint. It is always a word-sized unsigned
-integer and never overflows. This will be a building block for things like
-computing hash values, where wrap-around effects are intended and should be
-easily coded without lots of explicit mask shuffling.
-
-Now I think to extend this even more and build a full set of primitive types,
-which are intended to
-
-+ define the semantics of the primitive type
-+ give a working implementation for unrestricted Python
-
-These primitive types can later easily be augmented with methods to emit C code instead of executing. I guess this would be implemented in an extra ObjectSpace.
-
-
-Exception rules
----------------------
-
-Exceptions are by default not generated for simple cases.::
-
-
- #!/usr/bin/python
-
- x = 5
- x = x + 1 # this code is not checked for overflow
-
- try:
- x = x + y
- except OverflowError:
- # implement using longs
-
-
-Code with no exception handlers does not raise exceptions. By supplying an
-exception handler, you ask for error checking. Without, you assure the system
-that the operation cannot overflow.
-
-Exceptions explicitly raised will always be generated.
-
-
-Testing
-------------
-
-Besides extra tests which have to be written, PyPy has the advantage that it is
-runnable on standard CPython. That means, we can run all of PyPy with all
-exception handling enabled, so we might catch cases where we failed to adhere to
-our implicit assertions.
Deleted: /pypy/dist/pypy/documentation/testdesign.txt
==============================================================================
--- /pypy/dist/pypy/documentation/testdesign.txt Sun Apr 17 19:09:04 2005
+++ (empty file)
@@ -1,36 +0,0 @@
-=============
-Test Design
-=============
-
-Our tests are based on the new `py.test`_ tool which lets you write
-unittests without boilerplate. All tests of modules
-in a directory usually reside in a subdirectory **test**. There are
-basically two types of unit tests:
-
-- **Interpreter Level tests**. They run at the same level as PyPy's
- interpreter.
-
-- **Application Level tests**. They run at application level which means
- that they look like straight python code but they are interpreted by PyPy.
-
-Both types of tests need an objectspace they can run with (the interpreter
-dispatches operations on objects to an objectspace). If you run a test you
-can usually give the '-o' switch to select an object space. E.g. '-o thunk'
-will select the thunk object space. The default is the "Standard Object Space"
-which aims to implement unmodified Python semantics.
-
-.. _`py.test`: http://codespeak.net/py/current/doc/test.html
-
-Writing a test
---------------
-
-Currently the best reference is to go to some test files and look how they are done.
-
-Command line tool test_all
---------------------------
-
-You can run almost all of PyPy's tests by invoking::
-
- python test_all.py
-
-For switches to modify test execution invoke "python test_all.py -h".
More information about the Pypy-commit
mailing list