[pypy-svn] r12434 - pypy/dist/pypy/documentation
hpk at codespeak.net
hpk at codespeak.net
Wed May 18 14:12:26 CEST 2005
Author: hpk
Date: Wed May 18 14:12:25 2005
New Revision: 12434
Modified:
pypy/dist/pypy/documentation/coding-guide.txt
Log:
issue51 in-progress
A whole new chapter with subchapters dealing with "Modules in PyPy".
Missing is how PyPy interacts with CPython's sys.path etc.pp. I am
not sure that i am sure about this myself.
Modified: pypy/dist/pypy/documentation/coding-guide.txt
==============================================================================
--- pypy/dist/pypy/documentation/coding-guide.txt (original)
+++ pypy/dist/pypy/documentation/coding-guide.txt Wed May 18 14:12:25 2005
@@ -403,20 +403,63 @@
We are thinking about replacing ``OperationError`` with a family of common exception classes (e.g. ``AppKeyError``, ``AppIndexError``...) so that we can more easily catch them. The generic ``AppError`` would stand for all other application-level classes.
+Modules in PyPy
+===============
-Naming and directory layout
-===========================
+Modules visible from application programs are imported from
+interpreter or application level files. PyPy reuses almost all python
+modules of CPython's standard library, currently from version 2.3.4. We
+sometimes need to `modify modules`_ and - more often - regression tests
+because they rely on implementation details of CPython.
+
+If we don't just modify an original CPython module but need to rewrite
+it from scratch we put it into `pypy/lib`_ as a pure application level
+module.
+
+When we need access to interpreter-level objects we put the module into
+`pypy/module`_. Such modules use a `mixed module mechanism`_
+which makes it convenient to use both interpreter- and applicationlevel
+parts for the implementation. Note that there is no extra facility for
+pure-interpreter level modules because we haven't needed it so far.
+
+.. _`lib-python`: http://codespeak.net/svn/pypy/dist/lib-python/
+.. _`pypy/module`: http://codespeak.net/svn/pypy/dist/pypy/module/
+.. _`pypy/lib`: http://codespeak.net/svn/pypy/dist/pypy/lib/
+
+Determining the location of a module implementation
+---------------------------------------------------
+
+You can interactively find out where a module comes from,
+here are examples for the possible locations::
+
+ >>>> import sys
+ >>>> sys.__file__
+ '/home/hpk/pypy-dist/pypy/module/sys2/__init__.pyc'
+
+ >>>> import operator
+ >>>> operator.__file__
+ '/home/hpk/pypy-dist/pypy/lib/operator.py'
+
+ >>>> import types
+ t>>>> types.__file__
+ '/home/hpk/pypy-dist/lib-python/modified-2.3.4/types.py'
+
+ >>>> import os
+ faking <type 'posix.stat_result'>
+ faking <type 'posix.statvfs_result'>
+ >>>> os.__file__
+ '/home/hpk/pypy-dist/lib-python/2.3.4/os.py'
+ >>>>
+
+Module directories / Import order
+---------------------------------
-Modifying/Hacking PyPy's standard library
---------------------------------------------
+Here is the order in which PyPy looks up Python modules:
-PyPy reuses the Python implementations of CPython's standard
-library, currently from version 2.3.4. Sometimes
-we need to modify modules and - more often -
-regression tests because they rely on implementation
-details of CPython. Therefore we have a strict scheme on
-how to deal with these issues. Here is the order
-in which PyPy looks up Python modules:
+*pypy/modules*
+
+ mixed interpreter/app-level builtin modules, such as
+ the sys and builtin module.
*pypy/lib/*
@@ -432,14 +475,126 @@
*lib-python/2.3.4/*
- The unmodified CPython library. **Never checkin anything here**.
+ The unmodified CPython library. **Never ever checkin anything here**.
+
+.. _`modify modules`:
+
+Modifying a CPython library module or regression test
+-------------------------------------------------------
+
+Although PyPy is very compatible to CPython we sometimes need
+to change modules contained in our copy of the standard library,
+often due to the fact that PyPy works with all new-style classes
+by default and CPython has a number of places where it relies
+on some classes being old-style.
+
+If you want to change a module or test contained in `lib-python/2.3.4`
+then make sure that you copy the file to our `lib-python/modified-2.3.4`
+directory first. In subversion commandline terms this reads:
+
+ svn cp lib-python/2.3.4/somemodule.py lib-python/modified-2.3.4/
+
+and subsequently you edit and commit ``lib-python/modified-2.3.4/somemodule.py``.
+These copying operation is important because it keeps the original
+CPython tree clean and makes it obvious what we had to change.
+
+.. _`mixed module mechanism`:
+
+Implementing a mixed interpreter/application level Module
+---------------------------------------------------------
+
+If a module needs to access PyPy's interpreter level
+then it is implemented as a mixed module.
+
+Mixed modules are directories in `pypy/module`_ with an `__init__.py`
+file containing specifications where each name in a module comes from.
+Only specified names will be exported to a Mixed Module's applevel
+namespace.
+
+application level definitions
+.............................
+
+Application level specifiations are found in the `appleveldefs`
+dictionary found in `__init__.py` files of directories in `pypy/module`.
+For example, in `pypy/module/builtin/__init__.py`_ you find the following
+entry specifying where `__builtin__.locals` comes from::
+
+ ...
+ 'locals' : 'app_inspect.locals',
+ ...
+
+The `app_` prefix indicates that the submodule `app_inspect` is
+interpreted at application level and the function value for `locals`
+will be extracted accordingly.
+
+.. _`pypy/module/builtin/__init__.py`: http://codespeak.net/svn/pypy/dist/pypy/module/builtin/__init__.py
-If you e.g. need to change a standard module or package then
-you would issue::
+interpreter level definitions
+.............................
- svn cp lib-python/2.3.4/standardmodule.py lib-python/modified-2.3.4
+Interpreter level specifiations are found in the `interpleveldefs`
+dictionary found in `__init__.py` files of directories in `pypy/module`.
+For example, in `pypy/module/builtin/__init__.py`_ the following
+entry specifies where `__builtin__.len` comes from::
-and subsequently edit and commit ``lib-python/modified-2.3.4/somemodule.py``.
+ ...
+ 'len' : 'operation.len',
+ ...
+
+The `operation` submodule lives at interpreter level and `len`
+is expected to be exposable to application level. Here is
+the definition for `operation.len()`::
+
+ def len(space, w_obj):
+ "len(object) -> integer\n\nReturn the number of items of a sequence or mapping."
+ return space.len(w_obj)
+
+Exposed interpreter level functions usually take a `space` argument
+and some wrapped values (see `wrapping rules`_) .
+
+You can also use a convenient shortcut in `interpleveldefs` dictionaries:
+namely an expression in parentheses to specify an interpreter level
+expression directly (instead of pulling it indirectly from a file)::
+
+ ...
+ 'None' : '(space.w_None)',
+ 'False' : '(space.w_False)',
+ ...
+
+The interpreter level expression has a `space` binding when
+it is executed.
+
+
+Testing modules in `pypy/lib`
+-----------------------------
+
+You can go to the `pypy/lib/test2`_ directory and invoke the testing tool
+("py.test" or "python ../../test_all.py") to run tests against the
+pypy/lib hierarchy. Note, that tests in `pypy/lib/test2`_ are allowed
+and encouraged to let their tests run at interpreter level although
+`pypy/lib` modules eventually live at PyPy's application level.
+This allows us to quickly test our python-coded reimplementations
+against CPython.
+
+.. _`pypy/lib/test2`: http://codespeak.net/svn/pypy/dist/pypy/lib/test2
+
+Testing modules in `pypy/module`
+----------------------------------
+
+Simply change to `pypy/module` and run the tests as usual.
+
+
+Testing modules in `lib-python`
+-----------------------------------
+
+In order to let CPython's regression tests run against PyPy
+you can switch to the `lib-python`_ directory and run
+the testing tool in order to start compliance tests.
+(XXX ensure windows compatibility for producing test
+reports).
+
+Naming and directory layout
+===========================
Directory and File Naming
-------------------------
More information about the Pypy-commit
mailing list