From tismer at codespeak.net Tue Sep 2 11:48:41 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 2 Sep 2003 11:48:41 +0200 (MEST) Subject: [pypy-svn] rev 1270 - pypy/trunk/doc Message-ID: <20030902094841.730CC5C072@thoth.codespeak.net> Author: tismer Date: Tue Sep 2 11:48:40 2003 New Revision: 1270 Modified: pypy/trunk/doc/readme.txt Log: just broke down long lines into short ones Modified: pypy/trunk/doc/readme.txt ============================================================================== --- pypy/trunk/doc/readme.txt (original) +++ pypy/trunk/doc/readme.txt Tue Sep 2 11:48:40 2003 @@ -39,9 +39,12 @@ even more code still more code -End of the "block" occurs whenever you unindent back to the same level as the text with the ``::`` at the end. +End of the "block" occurs whenever you unindent back to the same level as the +text with the ``::`` at the end. -Using an underscore after a word like this_ will make reST think you want a hyperlink. To avoid that (especially with things like ``wrap_``), you can use the `` back quote `` to mark it as plain text. +Using an underscore after a word like this_ will make reST think you want a hyperlink. +To avoid that (especially with things like ``wrap_``), you can use the `` back quote `` +to mark it as plain text. You can get more info on reST markup at http://docutils.sourceforge.net/docs/rst/quickref.html @@ -49,11 +52,16 @@ Checking your work ------------------------ -In order to make sure that what you commit looks reasonably pretty (or at least not entirely broken), you'll need to run the ``docutils`` parser on it. Unless you've installed it in the past, you probably don't have it installed. Open IDLE (or any Python interactive environment) and try "import docutils". If it imports, hooray! Otherwise, you'll need to download it. +In order to make sure that what you commit looks reasonably pretty (or at least not +entirely broken), you'll need to run the ``docutils`` parser on it. Unless you've +installed it in the past, you probably don't have it installed. Open IDLE (or any +Python interactive environment) and try "import docutils". If it imports, hooray! +Otherwise, you'll need to download it. Go to sourceforge and download the ``snapshot`` version. Install it. -*Note to Debian users:* Be sure you installed ``python2.2-dev``, which includes ``distutils``, before trying to install ``docutils``. +*Note to Debian users:* Be sure you installed ``python2.2-dev``, which includes ``distutils``, +before trying to install ``docutils``. Once you have ``docutils`` installed, you can use it go to your shell and use it like this:: @@ -62,9 +70,15 @@ ::: Processing .txt: howtosvn.txt ::: Processing .txt: index.txt -**WARNING** This will process **all** text documents in the directory and any subdirectories. I prefer to work on text in a separate directory, run the ``docutils`` parser to see what it looks like, then copy the .txt file over to my local /doc checkouts to commit it. - -Use a browser menu to go to ``File: Open: filename.html`` then you can see what it looks like. Look at the command shell to see what errors you've got on which lines and fix it in your textfile. You can then re-run the buildhtml.py script and see what errors you get. After it's fixed, you can commit the .txt file and it'll automagically be turned into html viewable on the website. +**WARNING** This will process **all** text documents in the directory and any subdirectories. +I prefer to work on text in a separate directory, run the ``docutils`` parser to see what it +looks like, then copy the .txt file over to my local /doc checkouts to commit it. + +Use a browser menu to go to ``File: Open: filename.html`` then you can see what it looks +like. Look at the command shell to see what errors you've got on which lines and fix it +in your textfile. You can then re-run the buildhtml.py script and see what errors you get. +After it's fixed, you can commit the .txt file and it'll automagically be turned into html +viewable on the website. Here are some sample reST textfiles to see what it looks like: @@ -77,3 +91,4 @@ .. _this: http://docutils.sourceforge.net/docs/rst/quickref.html .. _ObjectSpace: /pypy/doc/objspace/objspace.txt .. _ObjectSpaceInterface: /pypy/doc/objspace/objspaceinterface.txt + From arigo at codespeak.net Sun Sep 7 18:49:17 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Sep 2003 18:49:17 +0200 (MEST) Subject: [pypy-svn] rev 1271 - pypy/trunk/doc/EU_funding Message-ID: <20030907164917.A19535A144@thoth.codespeak.net> Author: arigo Date: Sun Sep 7 18:49:16 2003 New Revision: 1271 Added: pypy/trunk/doc/EU_funding/plan.txt Log: plan - draft Added: pypy/trunk/doc/EU_funding/plan.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/EU_funding/plan.txt Sun Sep 7 18:49:16 2003 @@ -0,0 +1,190 @@ +========================= +Draft of a PyPy work plan +========================= + + +1. The PyPy Interpreter +----------------------- + +The goal is to make a complete Python interpreter that runs under any +existing Python implementation. + + a) develop and complete the PyPy interpreter itself, as a regular +Python program, until it contains all the parts of CPython that we don't +want to move to (b). Further investigate the unorthodox multimethod +concepts that the standard object space is based on, and how to hook in +the bytecode compiler. + + b) translate all other parts of CPython into regular Python libraries. +These ones should also work without PyPy, being just plain-Python +replacements for existing CPython functionality. This includes the +bytecode compiler. + + +2. Translation of RPython +------------------------- + +The goal is to be able to translate arbitrary RPython source code (e.g. +the one produced in 1a) into low-level code (C, Pyrex, Java, others). +This includes making a stand-alone, not-PyPy-related tool for general +optimization of arbitrary but suitably restricted Python application or +parts thereof. + + a) analyse code to produce the relevant typing information. Investigate +if we can use the annotation object space only or if additional +AST-based control flow analysis is needed. + + b) produce low-level code out of the data gathered in (a). Again +investigate how this is best done (AST-guided translation or +reverse-engeneering of the low-level control flow gathered by the +annotation object space). Compare different low-level environment that +we could target (C, Pyrex, others?). + + +3. Bootstrapping PyPy +--------------------- + +The goal is to put (1) and (2) together. + + a) investigate the particular problems specific to the global +translation of PyPy, as opposed to general to any RPython program. +According to the requirements and insights of (2) we will probably have +to redesign specific parts of PyPy, e.g. make the various +app-level/interp-level interface designs converge. + + b) build the low-level-specific run-time components of PyPy, most +notably the object layout, the memory management, possibly threading +support, and multimethod dispatch. Here, if we target C code, important +parts can be directly re-used from CPython. + + +4. High-performance PyPy-Python +------------------------------- + +The goal is to optimize (3) in possibly various ways, building on its +flexibility to go beyond CPython. + + a) develop several object implementations for the same types, as +explicitely allowed by the standard object space, and develop heuristics +to switch between implementations during execution. + + b) identify which optimizations would benefit from support from the +translator (2). These are the optimizations not easily available to +CPython because they would require large-scale code rewrites. + + c) for each issue, work on several solutions when no one is obviously +better than the other ones. The meta-programming underlying (b) -- +namely the work on the translator instead of on the resulting code -- is +what gives us the possibility of actually implementing several very +different schemes. + + d) integrate existing technology that traditionally depended on closely +following CPython's code base, notably Psyco and Stackless. Rewrite each +one as a meta-component that hooks into the translator (2) plus a +dedicated run-time component (3b). Further develop these technologies +based on the results gathered in (c), e.g. identify when these +technologies would guide specific choices among the solutions developed +in (a) and (b). + + +Annex to (a) +~~~~~~~~~~~~ + +Some major uses for several implementations of the built-in types: + + * dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, +or with string-only or integer-only keys. Dictionaries with specific +support for "on-change" callbacks (useful for Psyco). + + * strings as plain immutable memory buffers vs. immutable but more +complex data structures (see functional languages) vs. internally +mutable data structures (e.g. Psyco's concatenated strings) + + * ints as machine words vs. two machine words vs. internal longs vs. +external bignum library (investigate if completely unifying ints and +longs is possible in the Python language at this stage). + + * etc. (lists as range() or chained lists, ...) + +The above are mostly independent from any particular low-level run-time +environment. + + +Annex to (b) +~~~~~~~~~~~~ + +Here are some of the main issues and tricks. Note that compatibility +with legacy C extensions can be acheived by choosing, for each of the +following issues, the same one as CPython did. + + * object layout and memory management strategy or strategies, e.g. +reference counting vs. Boehm garbage collection vs. our own. Includes +speed vs. data size trade-offs. + + * code size vs. speed trade-offs (e.g. whether the final interpreter +should still include compact precompiled bytecode or be completely +translated into C). + + * the complex issue of threading (global interpreter lock vs. +alternatives). + + * multimethod dispatching + + * pointer tagging, e.g. encoding an integer object as a pointer with a +special value instead of a real pointer to a data structure representing +the integer. + +The above are mostly specific to a particular low-level run-time. + + +5. Low-level targets, tools and releases +---------------------------------------- + +The goal is to identify, among those low-level targets that are in +widespread use (e.g. workstation usage vs. web server vs. +high-performance computing vs. memory-starved hand-held device; C/Unix +vs. Java vs. .NET environment), which ones would benefit most from a +high-performance Python interpreter. For each of these, focus will be +given to: + + a) develop the translation process, run-time and those optimizations +that depend on low-level details. + + b) design interfaces for extension modules. Some can be very general +(e.g. a pure Python one that should allow generic third-party code to +hook into the PyPy interpreter source code without worrying about the +translation process). Others depend on the low-level environment and on +the choices made for the issues of (4). + + c) combine different solutions for the different issues discussed in +(4). Gather statistics with real-work Python application. Compare the +results. This is where the flexibility of the whole project is vital. +Typically, very different trade-offs need to be made on different +environments. + + d) most importantly, develop tools to easily allow third-parties to +repeat (c) in their own domain and build their own tailored versions of +PyPy. + + e) release a few official versions pre-tailored for various common +environments. Develop in particular a version whose goal is to simulate +the existing CPython interpreter to support legacy extension modules. +Investigate if the PyPy core can make internal choices that are very +different from CPython's without sacrifying legacy extension modules +compatibility. + + +6. Infrastructure +----------------- + +The goal is to address the development and maintenance issues. + + a) PyPy's own development needs an infrastructure that must +continuously be kept up-to-date and further developed. + + b) write tests. All parts of PyPy should be extensively covered by +stress tests. Investigate the use of test-coverage analysers. + + c) investigate means of keeping PyPy in sync with the future +developments of CPython, e.g. ways to relate pieces of PyPy source and +pieces of CPython source. Look for existing solutions. From arigo at codespeak.net Sun Sep 7 20:19:50 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Sep 2003 20:19:50 +0200 (MEST) Subject: [pypy-svn] rev 1272 - pypy/trunk/doc/EU_funding Message-ID: <20030907181950.671385A144@thoth.codespeak.net> Author: arigo Date: Sun Sep 7 20:19:49 2003 New Revision: 1272 Modified: pypy/trunk/doc/EU_funding/plan.txt Log: added a section Modified: pypy/trunk/doc/EU_funding/plan.txt ============================================================================== --- pypy/trunk/doc/EU_funding/plan.txt (original) +++ pypy/trunk/doc/EU_funding/plan.txt Sun Sep 7 20:19:49 2003 @@ -188,3 +188,29 @@ c) investigate means of keeping PyPy in sync with the future developments of CPython, e.g. ways to relate pieces of PyPy source and pieces of CPython source. Look for existing solutions. + + +7. Extension of PyPy +-------------------- + +The goal is to add functionalities in PyPy that are not present in +existing Python implementations. This is an open goal. We only list a +few promizing directions: + + a) build alternate object spaces provides features that are essentially +language-transparent, e.g. distributed computing (via a network proxy +object space), compatibility layers (e.g. a Python-1.5.2-compliant +object space), persistance (via a persistant object space). + + b) build language features that rely on translator support (2), i.e. +which can be turned on or off during the production of individual +versions of PyPy, e.g. Stackless and continuations. + + c) work on the interaction between the compiler and the main loop to +allow custom opcodes to be defined, generated by the compiler, and +interpreted by the main loop, thus allowing syntactic extension of the +language by user code + + d) conversely, develop interfaces to use object spaces without the main +loop to provide Python-like object semantics to other programming +languages, using their own syntax and execution environment, e.g. Java. From tismer at codespeak.net Sun Sep 7 23:43:14 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 7 Sep 2003 23:43:14 +0200 (MEST) Subject: [pypy-svn] rev 1273 - pypy/trunk/doc/EU_funding Message-ID: <20030907214314.E7F5A5A144@thoth.codespeak.net> Author: tismer Date: Sun Sep 7 23:43:14 2003 New Revision: 1273 Modified: pypy/trunk/doc/EU_funding/plan.txt Log: small format corrections. The numberi g btw. seems to get messy in reStructuredText Modified: pypy/trunk/doc/EU_funding/plan.txt ============================================================================== --- pypy/trunk/doc/EU_funding/plan.txt (original) +++ pypy/trunk/doc/EU_funding/plan.txt Sun Sep 7 23:43:14 2003 @@ -1,90 +1,94 @@ -========================= +Wiki-Safetybelt: 1062963471.68 +Type: msgrstprelinkfitissue +Log: + +========================== Draft of a PyPy work plan -========================= +========================== -1. The PyPy Interpreter ------------------------ +1. The !PyPy Interpreter +--------------------------- The goal is to make a complete Python interpreter that runs under any existing Python implementation. - a) develop and complete the PyPy interpreter itself, as a regular -Python program, until it contains all the parts of CPython that we don't -want to move to (b). Further investigate the unorthodox multimethod -concepts that the standard object space is based on, and how to hook in -the bytecode compiler. - - b) translate all other parts of CPython into regular Python libraries. -These ones should also work without PyPy, being just plain-Python -replacements for existing CPython functionality. This includes the -bytecode compiler. +a) develop and complete the !PyPy interpreter itself, as a regular + Python program, until it contains all the parts of CPython that we don't + want to move to b). Further investigate the unorthodox multimethod + concepts that the standard object space is based on, and how to hook in + the bytecode compiler. + +b) translate all other parts of !CPython into regular Python libraries. + These ones should also work without !PyPy, being just plain-Python + replacements for existing !CPython functionality. This includes the + bytecode compiler. -2. Translation of RPython -------------------------- +2. Translation of !RPython +------------------------------ The goal is to be able to translate arbitrary RPython source code (e.g. the one produced in 1a) into low-level code (C, Pyrex, Java, others). -This includes making a stand-alone, not-PyPy-related tool for general +This includes making a stand-alone, not-!PyPy-related tool for general optimization of arbitrary but suitably restricted Python application or parts thereof. - a) analyse code to produce the relevant typing information. Investigate -if we can use the annotation object space only or if additional -AST-based control flow analysis is needed. - - b) produce low-level code out of the data gathered in (a). Again -investigate how this is best done (AST-guided translation or -reverse-engeneering of the low-level control flow gathered by the -annotation object space). Compare different low-level environment that -we could target (C, Pyrex, others?). +a) analyse code to produce the relevant typing information. Investigate + if we can use the annotation object space only or if additional + AST-based control flow analysis is needed. + +b) produce low-level code out of the data gathered in (a). Again + investigate how this is best done (AST-guided translation or + reverse-engeneering of the low-level control flow gathered by the + annotation object space). Compare different low-level environment that + we could target (C, Pyrex, others?). -3. Bootstrapping PyPy ---------------------- +3. Bootstrapping !PyPy +-------------------------- The goal is to put (1) and (2) together. - a) investigate the particular problems specific to the global -translation of PyPy, as opposed to general to any RPython program. -According to the requirements and insights of (2) we will probably have -to redesign specific parts of PyPy, e.g. make the various -app-level/interp-level interface designs converge. - - b) build the low-level-specific run-time components of PyPy, most -notably the object layout, the memory management, possibly threading -support, and multimethod dispatch. Here, if we target C code, important -parts can be directly re-used from CPython. - - -4. High-performance PyPy-Python -------------------------------- - -The goal is to optimize (3) in possibly various ways, building on its -flexibility to go beyond CPython. - - a) develop several object implementations for the same types, as -explicitely allowed by the standard object space, and develop heuristics -to switch between implementations during execution. - - b) identify which optimizations would benefit from support from the -translator (2). These are the optimizations not easily available to -CPython because they would require large-scale code rewrites. - - c) for each issue, work on several solutions when no one is obviously -better than the other ones. The meta-programming underlying (b) -- -namely the work on the translator instead of on the resulting code -- is -what gives us the possibility of actually implementing several very -different schemes. - - d) integrate existing technology that traditionally depended on closely -following CPython's code base, notably Psyco and Stackless. Rewrite each -one as a meta-component that hooks into the translator (2) plus a -dedicated run-time component (3b). Further develop these technologies -based on the results gathered in (c), e.g. identify when these -technologies would guide specific choices among the solutions developed -in (a) and (b). +a) investigate the particular problems specific to the global + translation of !PyPy, as opposed to general to any !RPython program. + According to the requirements and insights of (2) we will probably have + to redesign specific parts of !PyPy, e.g. make the various + app-level/interp-level interface designs converge. + +b) build the low-level-specific run-time components of !PyPy, most + notably the object layout, the memory management, possibly threading + support, and multimethod dispatch. Here, if we target C code, important + parts can be directly re-used from !CPython. + + +4. High-performance !PyPy-Python +----------------------------------- + +The goal is to optimize `3. Bootstrapping !PyPy`_ in possibly various ways, building on its +flexibility to go beyond !CPython. + +a) develop several object implementations for the same types, as + explicitely allowed by the standard object space, and develop heuristics + to switch between implementations during execution. + +b) identify which optimizations would benefit from support from the + translator (2). These are the optimizations not easily available to + !CPython because they would require large-scale code rewrites. + +c) for each issue, work on several solutions when no one is obviously + better than the other ones. The meta-programming underlying (b) -- + namely the work on the translator instead of on the resulting code -- is + what gives us the possibility of actually implementing several very + different schemes. + +d) integrate existing technology that traditionally depended on closely + following !CPython's code base, notably Psyco and Stackless. Rewrite each + one as a meta-component that hooks into the translator (2) plus a + dedicated run-time component (3b). Further develop these technologies + based on the results gathered in (c), e.g. identify when these + technologies would guide specific choices among the solutions developed + in (a) and (b). Annex to (a) @@ -92,19 +96,19 @@ Some major uses for several implementations of the built-in types: - * dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, -or with string-only or integer-only keys. Dictionaries with specific -support for "on-change" callbacks (useful for Psyco). - - * strings as plain immutable memory buffers vs. immutable but more -complex data structures (see functional languages) vs. internally -mutable data structures (e.g. Psyco's concatenated strings) - - * ints as machine words vs. two machine words vs. internal longs vs. -external bignum library (investigate if completely unifying ints and -longs is possible in the Python language at this stage). +* dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, + or with string-only or integer-only keys. Dictionaries with specific + support for "on-change" callbacks (useful for Psyco). + +* strings as plain immutable memory buffers vs. immutable but more + complex data structures (see functional languages) vs. internally + mutable data structures (e.g. Psyco's concatenated strings) + +* ints as machine words vs. two machine words vs. internal longs vs. + external bignum library (investigate if completely unifying ints and + longs is possible in the Python language at this stage). - * etc. (lists as range() or chained lists, ...) +* etc. (lists as range() or chained lists, ...) The above are mostly independent from any particular low-level run-time environment. @@ -115,30 +119,30 @@ Here are some of the main issues and tricks. Note that compatibility with legacy C extensions can be acheived by choosing, for each of the -following issues, the same one as CPython did. +following issues, the same one as !CPython did. - * object layout and memory management strategy or strategies, e.g. -reference counting vs. Boehm garbage collection vs. our own. Includes -speed vs. data size trade-offs. +* object layout and memory management strategy or strategies, e.g. + reference counting vs. Boehm garbage collection vs. our own. Includes + speed vs. data size trade-offs. - * code size vs. speed trade-offs (e.g. whether the final interpreter -should still include compact precompiled bytecode or be completely -translated into C). +* code size vs. speed trade-offs (e.g. whether the final interpreter + should still include compact precompiled bytecode or be completely + translated into C). - * the complex issue of threading (global interpreter lock vs. -alternatives). +* the complex issue of threading (global interpreter lock vs. + alternatives). - * multimethod dispatching +* multimethod dispatching - * pointer tagging, e.g. encoding an integer object as a pointer with a -special value instead of a real pointer to a data structure representing -the integer. +* pointer tagging, e.g. encoding an integer object as a pointer with a + special value instead of a real pointer to a data structure representing + the integer. The above are mostly specific to a particular low-level run-time. 5. Low-level targets, tools and releases ----------------------------------------- +-------------------------------------------- The goal is to identify, among those low-level targets that are in widespread use (e.g. workstation usage vs. web server vs. @@ -147,70 +151,70 @@ high-performance Python interpreter. For each of these, focus will be given to: - a) develop the translation process, run-time and those optimizations -that depend on low-level details. +a) develop the translation process, run-time and those optimizations + that depend on low-level details. - b) design interfaces for extension modules. Some can be very general -(e.g. a pure Python one that should allow generic third-party code to -hook into the PyPy interpreter source code without worrying about the -translation process). Others depend on the low-level environment and on -the choices made for the issues of (4). - - c) combine different solutions for the different issues discussed in -(4). Gather statistics with real-work Python application. Compare the -results. This is where the flexibility of the whole project is vital. -Typically, very different trade-offs need to be made on different -environments. - - d) most importantly, develop tools to easily allow third-parties to -repeat (c) in their own domain and build their own tailored versions of -PyPy. - - e) release a few official versions pre-tailored for various common -environments. Develop in particular a version whose goal is to simulate -the existing CPython interpreter to support legacy extension modules. -Investigate if the PyPy core can make internal choices that are very -different from CPython's without sacrifying legacy extension modules -compatibility. +b) design interfaces for extension modules. Some can be very general + (e.g. a pure Python one that should allow generic third-party code to + hook into the !PyPy interpreter source code without worrying about the + translation process). Others depend on the low-level environment and on + the choices made for the issues of (4). + +c) combine different solutions for the different issues discussed in + (4). Gather statistics with real-work Python application. Compare the + results. This is where the flexibility of the whole project is vital. + Typically, very different trade-offs need to be made on different + environments. + +d) most importantly, develop tools to easily allow third-parties to + repeat (c) in their own domain and build their own tailored versions of + !PyPy. + +e) release a few official versions pre-tailored for various common + environments. Develop in particular a version whose goal is to simulate + the existing !CPython interpreter to support legacy extension modules. + Investigate if the !PyPy core can make internal choices that are very + different from !CPython's without sacrifying legacy extension modules + compatibility. 6. Infrastructure ------------------ +--------------------- The goal is to address the development and maintenance issues. - a) PyPy's own development needs an infrastructure that must -continuously be kept up-to-date and further developed. +a) !PyPy's own development needs an infrastructure that must + continuously be kept up-to-date and further developed. - b) write tests. All parts of PyPy should be extensively covered by -stress tests. Investigate the use of test-coverage analysers. +b) write tests. All parts of !PyPy should be extensively covered by + stress tests. Investigate the use of test-coverage analysers. - c) investigate means of keeping PyPy in sync with the future -developments of CPython, e.g. ways to relate pieces of PyPy source and -pieces of CPython source. Look for existing solutions. +c) investigate means of keeping !PyPy in sync with the future + developments of !CPython, e.g. ways to relate pieces of !PyPy source and + pieces of !CPython source. Look for existing solutions. -7. Extension of PyPy --------------------- +7. Extension of !PyPy +------------------------- -The goal is to add functionalities in PyPy that are not present in +The goal is to add functionalities in !PyPy that are not present in existing Python implementations. This is an open goal. We only list a few promizing directions: - a) build alternate object spaces provides features that are essentially -language-transparent, e.g. distributed computing (via a network proxy -object space), compatibility layers (e.g. a Python-1.5.2-compliant -object space), persistance (via a persistant object space). - - b) build language features that rely on translator support (2), i.e. -which can be turned on or off during the production of individual -versions of PyPy, e.g. Stackless and continuations. - - c) work on the interaction between the compiler and the main loop to -allow custom opcodes to be defined, generated by the compiler, and -interpreted by the main loop, thus allowing syntactic extension of the -language by user code - - d) conversely, develop interfaces to use object spaces without the main -loop to provide Python-like object semantics to other programming -languages, using their own syntax and execution environment, e.g. Java. +a) build alternate object spaces provides features that are essentially + language-transparent, e.g. distributed computing (via a network proxy + object space), compatibility layers (e.g. a Python-1.5.2-compliant + object space), persistance (via a persistant object space). + +b) build language features that rely on translator support (2), i.e. + which can be turned on or off during the production of individual + versions of !PyPy, e.g. Stackless and continuations. + +c) work on the interaction between the compiler and the main loop to + allow custom opcodes to be defined, generated by the compiler, and + interpreted by the main loop, thus allowing syntactic extension of the + language by user code + +d) conversely, develop interfaces to use object spaces without the main + loop to provide Python-like object semantics to other programming + languages, using their own syntax and execution environment, e.g. Java. From tismer at codespeak.net Sun Sep 7 23:44:17 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 7 Sep 2003 23:44:17 +0200 (MEST) Subject: [pypy-svn] rev 1274 - pypy/trunk/doc/EU_funding Message-ID: <20030907214417.87B8B5A144@thoth.codespeak.net> Author: tismer Date: Sun Sep 7 23:44:17 2003 New Revision: 1274 Modified: pypy/trunk/doc/EU_funding/plan.txt Log: small format corrections. The numberi g btw. seems to get messy in reStructuredText Modified: pypy/trunk/doc/EU_funding/plan.txt ============================================================================== --- pypy/trunk/doc/EU_funding/plan.txt (original) +++ pypy/trunk/doc/EU_funding/plan.txt Sun Sep 7 23:44:17 2003 @@ -1,7 +1,3 @@ -Wiki-Safetybelt: 1062963471.68 -Type: msgrstprelinkfitissue -Log: - ========================== Draft of a PyPy work plan ========================== From arigo at codespeak.net Mon Sep 8 14:50:30 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 8 Sep 2003 14:50:30 +0200 (MEST) Subject: [pypy-svn] rev 1275 - pypy/trunk/doc/EU_funding Message-ID: <20030908125030.1623E5A4D1@thoth.codespeak.net> Author: arigo Date: Mon Sep 8 14:50:29 2003 New Revision: 1275 Modified: pypy/trunk/doc/EU_funding/plan.txt Log: Samuele's comment Modified: pypy/trunk/doc/EU_funding/plan.txt ============================================================================== --- pypy/trunk/doc/EU_funding/plan.txt (original) +++ pypy/trunk/doc/EU_funding/plan.txt Mon Sep 8 14:50:29 2003 @@ -32,7 +32,8 @@ a) analyse code to produce the relevant typing information. Investigate if we can use the annotation object space only or if additional - AST-based control flow analysis is needed. + AST-based control flow analysis is needed. Give a formal definition + of RPython. b) produce low-level code out of the data gathered in (a). Again investigate how this is best done (AST-guided translation or From arigo at codespeak.net Mon Sep 8 20:53:16 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 8 Sep 2003 20:53:16 +0200 (MEST) Subject: [pypy-svn] rev 1276 - in pypy/branch/builtinrefactor/pypy: interpreter module objspace Message-ID: <20030908185316.1DE5A5A4D1@thoth.codespeak.net> Author: arigo Date: Mon Sep 8 20:53:13 2003 New Revision: 1276 Added: pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py pypy/branch/builtinrefactor/pypy/interpreter/miscutils.py (contents, props changed) - copied, changed from rev 1267, pypy/branch/builtinrefactor/pypy/interpreter/executioncontext.py Removed: pypy/branch/builtinrefactor/pypy/interpreter/appfile.py pypy/branch/builtinrefactor/pypy/interpreter/opcode_app.py pypy/branch/builtinrefactor/pypy/interpreter/threadlocals.py Modified: pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py pypy/branch/builtinrefactor/pypy/interpreter/error.py pypy/branch/builtinrefactor/pypy/interpreter/executioncontext.py pypy/branch/builtinrefactor/pypy/interpreter/function.py pypy/branch/builtinrefactor/pypy/interpreter/gateway.py pypy/branch/builtinrefactor/pypy/interpreter/interactive.py pypy/branch/builtinrefactor/pypy/interpreter/main.py pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py pypy/branch/builtinrefactor/pypy/interpreter/pycode.py pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py pypy/branch/builtinrefactor/pypy/interpreter/unittest_w.py pypy/branch/builtinrefactor/pypy/module/builtin.py pypy/branch/builtinrefactor/pypy/module/sysmodule.py pypy/branch/builtinrefactor/pypy/objspace/trivial.py Log: getting closer, some tests start passing again Deleted: /pypy/branch/builtinrefactor/pypy/interpreter/appfile.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/interpreter/appfile.py Mon Sep 8 20:53:13 2003 +++ (empty file) @@ -1,69 +0,0 @@ -import os - -class AppFile: - """Dynamic loader of a set of Python functions and objects that - should work at the application level (conventionally in .app.py files)""" - - # absolute name of the parent directory - ROOTDIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) - DEFAULT_PATH_EXT = [('appspace', '.py')] - LOCAL_PATH = [] - - def __init__(self, modulename, localpath=[]): - "Load and compile the helper file." - # XXX looking for a pre-compiled file here will be quite essential - # when we want to bootstrap the compiler - - # 'modulename' could be 'package.module' if passed in as __name__ - # we ignore that package part - modulename = modulename.split('.')[-1] - path_ext = [(path, '_app.py') for path in localpath + self.LOCAL_PATH] - for path, ext in path_ext + self.DEFAULT_PATH_EXT: - dirname = os.path.join(self.ROOTDIR, path.replace('.', os.sep)) - filename = os.path.join(dirname, modulename+ext) - if os.path.exists(filename): - break - else: - raise IOError, "cannot locate helper module '%s' in %s" % ( - modulename, path_ext) - f = open(filename, 'r') - src = f.read() - f.close() - #print filename - self.bytecode = compile(src, filename, 'exec') - - -class Namespace: - - def __init__(self, space, w_namespace=None): - self.space = space - ec = space.getexecutioncontext() - if w_namespace is None: - w_namespace = ec.make_standard_w_globals() - self.w_namespace = w_namespace - - def get(self, objname): - "Returns a wrapped copy of an object by name." - w_name = self.space.wrap(objname) - w_obj = self.space.getitem(self.w_namespace, w_name) - return w_obj - - def call(self, functionname, args): - "Call a module function." - w_func = self.get(functionname) - w_args = self.space.newtuple(args) - w_keywords = self.space.newdict([]) - return self.space.call(w_func, w_args, w_keywords) - - def runbytecode(self, bytecode): - # initialize the module by running the bytecode in a new - # dictionary, in a new execution context - from pypy.interpreter.gateway import ScopedCode - scopedcode = ScopedCode(self.space, bytecode, self.w_namespace) - scopedcode.eval_frame() - -class AppHelper(Namespace): - - def __init__(self, space, appfile, w_namespace=None): - Namespace.__init__(self, space, w_namespace) - self.runbytecode(appfile.bytecode) Modified: pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py Mon Sep 8 20:53:13 2003 @@ -1,6 +1,6 @@ -from pypy.interpreter.executioncontext import ExecutionContext, Stack +from pypy.interpreter.executioncontext import ExecutionContext from pypy.interpreter.error import OperationError -from pypy.interpreter import threadlocals +from pypy.interpreter.miscutils import Stack, getthreadlocals __all__ = ['ObjSpace', 'OperationError', 'NoValue'] @@ -17,7 +17,7 @@ class ObjSpace: """Base class for the interpreter-level implementations of object spaces. - XXX describe here in more details what the object spaces are.""" + http://codespeak.net/moin/pypy/moin.cgi/ObjectSpace""" def __init__(self): "Basic initialization of objects." @@ -31,8 +31,9 @@ from pypy.module import builtin self.builtin = builtin.__builtin__(self) - self.w_builtin = self.builtin._wrapped - self.w_builtins = self.getattr(self.w_builtin, self.wrap("__dict__")) + self.w_builtin = self.wrap(self.builtin) + #self.w_builtins = self.getattr(self.w_builtin, self.wrap("__dict__")) + self.w_builtins = self.builtin.w_dict for name, value in self.__dict__.items(): if name.startswith('w_'): @@ -42,14 +43,14 @@ #print "setitem: space instance %-20s into builtins" % name self.setitem(self.w_builtins, self.wrap(name), value) - self.sys._setmodule(self.builtin) + self.sys._setmodule(self.w_builtin) def make_sys(self): assert not hasattr(self, 'sys') from pypy.module import sysmodule - self.sys = sysmodule.sys(self) - self.w_sys = self.sys._wrapped - self.sys._setmodule(self.sys) + self.sys = sysmodule.Sys(self) + self.w_sys = self.wrap(self.sys) + self.sys._setmodule(self.w_sys) # XXX get rid of this. def get_builtin_module(self, w_name): @@ -66,7 +67,7 @@ def getexecutioncontext(self): "Return what we consider to be the active execution context." - ec = threadlocals.getlocals().executioncontext + ec = getthreadlocals().executioncontext if ec is None: ec = self.createexecutioncontext() return ec Modified: pypy/branch/builtinrefactor/pypy/interpreter/error.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/error.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/error.py Mon Sep 8 20:53:13 2003 @@ -69,7 +69,7 @@ tb.reverse() print >> file, "Traceback (application-level):" for f, i in tb: - co = f.bytecode + co = f.code lineno = offset2lineno(co, i) fname = co.co_filename if fname.startswith('\n'): Modified: pypy/branch/builtinrefactor/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/executioncontext.py Mon Sep 8 20:53:13 2003 @@ -1,4 +1,4 @@ -from pypy.interpreter import threadlocals +from pypy.interpreter.miscutils import getthreadlocals, Stack class ExecutionContext: """An ExecutionContext holds the state of an execution thread @@ -10,13 +10,14 @@ self.framestack = Stack() def enter(self, frame): - locals = threadlocals.getlocals() + locals = getthreadlocals() self.framestack.push(frame) previous_ec = locals.executioncontext locals.executioncontext = self return previous_ec def leave(self, previous_ec): + locals = getthreadlocals() locals.executioncontext = previous_ec self.framestack.pop() @@ -49,37 +50,3 @@ if frame.last_exception is not None: return frame.last_exception return None - - -class Stack: - """Utility class implementing a stack.""" - - def __init__(self): - self.items = [] - - def clone(self): - s = self.__class__() - for item in self.items: - try: - item = item.clone() - except AttributeError: - pass - s.push(item) - return s - - def push(self, item): - self.items.append(item) - - def pop(self): - return self.items.pop() - - def top(self, position=0): - """'position' is 0 for the top of the stack, 1 for the item below, - and so on. It must not be negative.""" - return self.items[~position] - - def depth(self): - return len(self.items) - - def empty(self): - return not self.items Added: pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py ============================================================================== --- (empty file) +++ pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py Mon Sep 8 20:53:13 2003 @@ -0,0 +1,35 @@ +""" + +Helpers to build extension modules. + +""" + +from pypy.interpreter.gateway import DictProxy +from pypy.interpreter.miscutils import InitializedClass +from pypy.interpreter.baseobjspace import Wrappable + + +class ExtModule(Wrappable): + """An empty extension module. + Non-empty extension modules are made by subclassing ExtModule.""" + + def __init__(self, space): + self.space = space + self.w_dict = self._appdict.makedict(space, self) + + __metaclass__ = InitializedClass + def __initclass__(cls): + # make sure that the class has its own appdict + if '_appdict' not in cls.__dict__: + cls._appdict = DictProxy(implicitspace=True, + implicitself=True) + # automatically publish all functions from this class + cls._appdict.exportall(cls.__dict__) + + def __getattr__(self, attr): + # XXX temporary, for use by objspace.trivial and + # objspace.std.cpythonobject + try: + return self.__class__._appdict.content[attr] + except KeyError: + raise AttributeError, attr Modified: pypy/branch/builtinrefactor/pypy/interpreter/function.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/function.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/function.py Mon Sep 8 20:53:13 2003 @@ -6,7 +6,9 @@ attribute. """ -class Function: +from error import OperationError + +class Function(object): """A function is a code object captured with some environment: an object space, a dictionary of globals, default arguments, and an arbitrary 'closure' passed to the code object.""" @@ -53,14 +55,15 @@ # put as many positional input arguments into place as available args_w = space.unpacktuple(w_args) scope_w = args_w[:co_argcount] + input_argcount = len(scope_w) # check that no keyword argument conflicts with these - for name in argnames[:len(scope_w)]: + for name in argnames[:input_argcount]: w_name = space.wrap(name) if space.is_true(space.contains(w_kwargs, w_name)): self.raise_argerr_multiple_values(name) - if len(scope_w) < co_argcount: + if input_argcount < co_argcount: # not enough args, fill in kwargs or defaults if exists def_first = co_argcount - len(self.defs_w) for i in range(input_argcount, co_argcount): @@ -112,7 +115,7 @@ else: msg1 = "at least" n -= defcount - if kws: + if kwargname is not None: msg2 = "non-keyword " else: msg2 = "" @@ -126,7 +129,7 @@ n, msg2, plural, - len(args)) + nargs) raise OperationError(self.space.w_TypeError, msg) def raise_argerr_multiple_values(self, argname): @@ -148,3 +151,21 @@ self.func_code.co_name, nkwds) raise OperationError(self.space.w_TypeError, msg) + + + def __get__(self, inst, cls=None): + # for TrivialObjSpace only !!! + # use the mecanisms of gateway.py otherwise + import sys, new + assert 'pypy.objspace.trivial' in sys.modules, ( + "don't try to __get__() Function instances out of classes") + self.__name__ = self.func_code.co_name + return new.instancemethod(self, inst, cls) + + def __call__(self, *args, **kwds): + # for TrivialObjSpace only !!! + # use the mecanisms of gateway.py otherwise + import sys, new + assert 'pypy.objspace.trivial' in sys.modules, ( + "don't try to __call__() Function instances directly") + return self.call(args, kwds) Modified: pypy/branch/builtinrefactor/pypy/interpreter/gateway.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/gateway.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/gateway.py Mon Sep 8 20:53:13 2003 @@ -1,14 +1,22 @@ """ Gateway between app-level and interpreter-level: -* BuiltinCode (calling interp-level code from app-level) -* code2interp (embedding a code object into an interpreter-level callable) -* app2interp (embedding an app-level function's code object in the same way) +* BuiltinCode (call interp-level code from app-level) +* app2interp (embed an app-level function into an interp-level callable) +* interp2app (publish an interp-level object to be visible from app-level) +* publishall (mass-call interp2app on a whole list of objects) """ +# +# XXX warning, this module is a bit scary in the number of classes that +# all play a similar role but in slightly different contexts +# + +import types from pypy.interpreter import eval, pycode -from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter.baseobjspace import Wrappable, ObjSpace +from pypy.interpreter.function import Function class BuiltinCode(eval.Code): @@ -17,23 +25,47 @@ # When a BuiltinCode is stored in a Function object, # you get the functionality of CPython's built-in function type. - def __init__(self, func): + def __init__(self, func, **argflags): # 'implfunc' is the interpreter-level function. - # note that this uses a lot of (construction-time) introspection. + # See below for 'argflags'. + # Note that this uses a lot of (construction-time) introspection. eval.Code.__init__(self, func.__name__) self.func = func + self.argflags = argflags # extract the signature from the (CPython-level) code object tmp = pycode.PyCode(None) tmp._from_code(func.func_code) self.sig = tmp.signature() + if isinstance(func, types.MethodType) and func.im_self is not None: + argnames, varargname, kwargname = self.sig + argnames = argnames[1:] # implicit hidden self argument + self.sig = argnames, varargname, kwargname self.nargs = len(self.getvarnames()) + def bind_code(self, instance): + """Create another version of this code object that calls 'func' + as a method, with implicit first argument 'instance'.""" + return BuiltinCode(self.func.__get__(instance, instance.__class__), + **self.argflags) + def create_frame(self, space, w_globals, closure=None): return BuiltinFrame(space, self, w_globals, numlocals=self.nargs) def signature(self): return self.sig +# An application-level function always implicitely expects wrapped arguments, +# but not an interpreter-level function not. The extra keywords given to the +# constructor of BuiltinCode describes what kind of arguments 'func' expects. +# +# Default signature: +# def func(space, w_arg1, w_arg2...) <- plain functions +# def func(self, space, w_arg1, w_arg2...) <- methods +# +# Flags: (XXX need more) +# implicitspace=True method with no 'space' arg. We use 'self.space' +# implicitself=True the app-level doesn't see any 'self' argument + class BuiltinFrame(eval.Frame): "Frame emulation for BuiltinCode." @@ -42,8 +74,11 @@ # via the interface defined in eval.Frame. def run(self): + argarray = self.fastlocals_w + if not self.code.argflags.get('implicitspace'): + argarray = [space] + argarray return call_with_prepared_arguments(self.space, self.code.func, - self.fastlocals_w) + argarray) def call_with_prepared_arguments(space, function, argarray): @@ -56,33 +91,47 @@ # you don't need to look at it :-) keywords = {} co = function.func_code - if co.flags & 8: # CO_VARKEYWORDS + if co.co_flags & 8: # CO_VARKEYWORDS w_kwds = argarray[-1] for w_key in space.unpackiterable(w_kwds): keywords[space.unwrap(w_key)] = space.getitem(w_kwds, w_key) argarray = argarray[:-1] - if co.flags & 4: # CO_VARARGS + if co.co_flags & 4: # CO_VARARGS w_varargs = argarray[-1] argarray = argarray[:-1] + space.unpacktuple(w_varargs) return function(*argarray, **keywords) -class code2interp(object): +class Gateway(object): # General-purpose utility for the interpreter-level to create callables # that transparently invoke code objects (and thus possibly interpreted # app-level code). - def __init__(self, code, staticglobals, staticdefaults=[]): - self.code = code - self.staticglobals = staticglobals # a StaticGlobals instance - self.staticdefaults = staticdefaults + # 'argflags' controls how the Gateway instance should decode its + # arguments. It only influences calls made from interpreter-level. + # It has the same format as BuiltinCode.argflags. - def make_function(self, space): - assert self.staticglobals.is_frozen(), ( - "gateway not callable before the StaticGlobals is frozen") - w_globals = space.wrap(self.staticglobals) - defs_w = [space.wrap(def_value) for def_value in self.staticdefaults] - return Function(space, self.code, w_globals, defs_w) + def __init__(self, code, staticglobals, staticdefs=[], **argflags): + self.code = code + self.staticglobals = staticglobals # a DictProxy instance + self.staticdefs = staticdefs + self.argflags = argflags + + def make_function(self, space, bind_instance=None, w_globals=None): + if w_globals is None: + w_globals = space.wrap(self.staticglobals) + defs_w = [space.wrap(def_value) for def_value in self.staticdefs] + code = self.code + if self.argflags.get('implicitself') and isinstance(code, BuiltinCode): + assert bind_instance is not None, ("built-in function can only " + "be used as a method") + code = code.bind_code(bind_instance) + return Function(space, code, w_globals, defs_w) + + def __wrap__(self, space): + # to wrap a Gateway, we first make a real Function object out of it + # and the result is a wrapped version of this Function. + return space.wrap(self.make_function(space)) def __call__(self, space, *args, **kwds): wrap = space.wrap @@ -92,325 +141,151 @@ fn = self.make_function(space) return fn.call(w_args, w_kwds) + def __get__(self, obj, cls=None): + if obj is None: + return self + else: + return BoundGateway(self, obj) + -class StaticGlobals(Wrappable): - # This class captures a part of the content of an interpreter module - # or of a class definition, to be exposed at app-level with a read-only - # dict-like interface. - - def __init__(self, content=None): - self.content = None - if content is not None: - self.freeze(content) - - def freeze(self, content): - # Freeze the object to the value given by 'content': - # either a dictionary or a (new-style) class - assert self.content is None, "%r already frozen" % self - if isinstance(content, dict): - content = content.copy() +class BoundGateway(object): + + def __init__(self, gateway, obj): + self.gateway = gateway + self.obj = obj + + def __call__(self, *args, **kwds): + if self.gateway.argflags.get('implicitspace'): + # we read 'self.space' from the object we are bound to + space = self.obj.space else: - mro = list(content.__mro__) - mro.reverse() - content = {} - for c in mro: - content.update(c.__dict__) - self.content = content + space = args[0] # explicit 'space' as a first argument + args = args[1:] + if not isinstance(space, ObjSpace): + raise TypeError, "'space' expected as first argument" + if self.gateway.argflags.get('implicitself'): + pass # app-space gets no 'self' argument + else: + args = (space.wrap(self.obj),) + args # insert 'w_self' + return self.gateway(space, *args, **kwds) - def is_frozen(self): - return self.content is not None - def app2interp(self, app_f): - "Build a code2interp gateway that calls 'app_f' at app-level." - code = pycode.PyCode(None) - code._from_code(app_f.func_code) - return code2interp(code, self, list(app_f.func_defaults or ())) +class DictProxy(Wrappable): + # This class exposes at app-level a read-only dict-like interface. + # The items in the DictProxy are not wrapped (they are independent + # of any object space, and are just interpreter-level objects) until + # app-level code reads them. + + # Instances of DictProxy play the role of the 'globals' for app-level + # helpers. This is why app2interp and interp2app are methods of + # DictProxy. Calling them on the same DictProxy for several functions + # gives all these functions the same 'globals', allowing them to see + # and call each others. + + # XXX a detail has been (temporarily?) changed because too many + # places (notably in pyopcode.py) assume that the globals should + # satisfy 'instance(globals, dict)'. Now wrapping a DictProxy + # gives you a real dict. + + def __init__(self, basedict=None, **defaultargflags): + if basedict is None: + self.content = {} + else: + self.content = basedict.content.copy() + self.defaultargflags = defaultargflags - def __getitem__(self, key): - # XXX is only present for today's stdobjspace.cpythonobject wrapper - return self.content[key] - -noglobals = StaticGlobals({}) - - -##class app2interp(object): -## """ this class exposes an app-level method at interpreter-level. - -## Note that the wrapped method must *NOT* use a 'self' argument. -## Assumption: the instance on which this method is bound to has a -## 'space' attribute. -## """ -## def __init__(self, appfunc): -## self.appfunc = appfunc - -## def __get__(self, instance, cls=None): -## return InterpretedFunction(instance.space, self.appfunc) - -##class InterpretedFunctionFromCode(ScopedCode): -## def __init__(self, space, cpycode, w_defs, w_globals=None, closure_w=()): -## ScopedCode.__init__(self, space, cpycode, w_globals, closure_w) -## self.w_defs = w_defs -## self.simple = cpycode.co_flags & (CO_VARARGS|CO_VARKEYWORDS)==0 -## self.func_code = cpycode - -## def parse_args(self, frame, w_args, w_kwargs): -## """ parse args and kwargs and set fast scope of frame. -## """ -## space = self.space -## loc_w = None -## if self.simple and (w_kwargs is None or not space.is_true(w_kwargs)): -## try: -## loc_w = space.unpacktuple(w_args, self.cpycode.co_argcount) -## except ValueError: -## pass -## if loc_w is None: -## #print "complicated case of arguments for", self.cpycode.co_name, "simple=", self.simple -## w_loc = self.parse_args_complex(self.w_code, w_args, w_kwargs, self.w_defs) -## loc_w = space.unpacktuple(w_loc) -## loc_w.extend([_NULL] * (self.cpycode.co_nlocals - len(loc_w))) - -## # make nested cells -## if self.cpycode.co_cellvars: -## varnames = list(self.cpycode.co_varnames) -## for name in self.cpycode.co_cellvars: -## i = varnames.index(name) -## w_value = loc_w[i] -## loc_w[i] = _NULL -## frame.closure_w += (Cell(w_value),) - -## assert len(loc_w) == self.cpycode.co_nlocals, "local arguments not prepared correctly" -## frame.setfastscope(loc_w) - -## def create_frame(self, w_args, w_kwargs): -## """ parse arguments and execute frame """ -## from pyframe import PyFrame -## frame = PyFrame() -## frame.initialize(self) -## self.parse_args(frame, w_args, w_kwargs) -## return frame - -## def app_parse_args_complex(cpycode, args, kwargs, defs): -## """ return list of initial local values parsed from -## 'args', 'kwargs' and defaults. -## """ -## #if cpycode.co_name == 'failUnlessRaises': -## # print "co_name", cpycode.co_name -## # print "co_argcount", cpycode.co_argcount -## # print "co_nlocals", cpycode.co_nlocals -## # print "co_varnames", cpycode.co_varnames -## # print "args", args -## # print "kwargs", kwargs -## # print "defs", defs - -## CO_VARARGS, CO_VARKEYWORDS = 0x4, 0x8 - -## # co_argcount number of expected positional arguments -## # (elipsis args like *args and **kwargs do not count) -## co_argcount = cpycode.co_argcount - -## # construct list of positional args -## positional_args = list(args[:co_argcount]) - -## len_args = len(args) -## len_defs = len(defs) - -## if len_args < co_argcount: -## # not enough args, fill in kwargs or defaults if exists -## i = len_args -## while i < co_argcount: -## name = cpycode.co_varnames[i] -## if name in kwargs: -## positional_args.append(kwargs[name]) -## del kwargs[name] -## else: -## if i + len_defs < co_argcount: -## raise TypeError, "Not enough arguments" -## positional_args.append(defs[i-co_argcount]) -## i+=1 -## if cpycode.co_flags & CO_VARARGS: -## positional_args.append(tuple(args[co_argcount:])) -## elif len_args > co_argcount: -## raise TypeError, "Too many arguments" - -## # we only do the next loop for determining multiple kw-values -## i = 0 -## while i < len_args and i < co_argcount: -## name = cpycode.co_varnames[i] -## if name in kwargs: -## raise TypeError, "got multiple values for argument %r" % name -## i+=1 - -## if cpycode.co_flags & CO_VARKEYWORDS: -## positional_args.append(kwargs) -## elif kwargs: -## raise TypeError, "got unexpected keyword argument(s) %s" % repr(kwargs.keys()[0]) - -## return positional_args -## parse_args_complex = app2interp(app_parse_args_complex) - -## def __call__(self, *args_w, **kwargs_w): -## """ execute function and take arguments with -## native interp-level parameter passing convention """ -## w_args = self.space.newtuple(args_w) -## w = self.space.wrap -## w_kwargs = self.space.newdict([]) -## for name, w_value in kwargs_w.items(): -## self.space.setitem(w_kwargs, w(name), w_value) -## return self.eval_frame(w_args, w_kwargs) - -##class InterpretedFunction(InterpretedFunctionFromCode): -## """ a function which executes at app-level (by interpreting bytecode -## and dispatching operations on an objectspace). -## """ - -## def __init__(self, space, cpyfunc, w_globals=None, closure_w=()): -## """ initialization similar to base class but it also wraps -## some function-specific stuff (like defaults). -## """ -## assert not hasattr(cpyfunc, 'im_self') -## InterpretedFunctionFromCode.__init__(self, space, -## cpyfunc.func_code, -## space.wrap(cpyfunc.func_defaults or ()), -## w_globals, closure_w) - -##class InterpretedMethod(InterpretedFunction): -## """ an InterpretedFunction with 'self' spice. - -## XXX hpk: i think we want to eliminate all uses for this class -## as bound/unbound methods should be done in objspace?! - -## """ - -## def __init__(self, *args): -## InterpretedFunction.__init__(self, *args) - -## def parse_args(self, frame, w_args, w_kwargs): -## """ fills in "self" arg and dispatch to InterpreterFunction. -## """ -## space = self.space -## args_w = space.unpacktuple(w_args) -## args_w = [space.wrap(self)] + args_w -## w_args = space.newtuple(args_w) -## return InterpretedFunction.parse_args(self, frame, w_args, w_kwargs) - -##class AppVisibleModule: -## """ app-level visible Module defined at interpreter-level. - -## Inherit from this class if you want to have a module that accesses -## the PyPy interpreter (e.g. builtins like 'locals()' require accessing the -## frame). You can mix in application-level code by prefixing your method -## with 'app_'. Both non-underscore methods and app-level methods will -## be available on app-level with their respective name. - -## Note that app-level functions don't get a 'self' argument because it doesn't -## make sense and we really only need the function (there is no notion of beeing -## 'bound' or 'unbound' for them). - -## """ -## def __init__(self, space): -## self.space = space - -## space = self.space -## modname = self.__class__.__name__ -## self.w___name__ = space.wrap(modname) -## self._wrapped = _wrapped = space.newmodule(self.w___name__) - -## # go through all items in the module class instance -## for name in dir(self): -## # skip spurious info and internal methods -## if name == '__module__' or name.startswith('_') and not name.endswith('_'): -## #print modname, "skipping", name -## continue -## obj = getattr(self, name) -## # see if we just need to expose an already wrapped value -## if name.startswith('w_'): -## space.setattr(_wrapped, space.wrap(name[2:]), obj) - -## # see if have something defined at app-level -## elif name.startswith('app_'): -## obj = self.__class__.__dict__.get(name) -## name = name[4:] -## w_res = wrap_applevel(space, name, obj) -## # nope then we must expose interpreter-level to app-level -## else: -## w_res = wrap_interplevel(space, name, obj) -## setattr(self, 'w_'+name, w_res) -## w_name = space.wrap(name) -## space.setattr(_wrapped, w_name, w_res) - -##def wrap_applevel(space, name, obj): -## """ wrap an app-level style object which was compiled at interp-level. """ -## if hasattr(obj, 'func_code'): -## return space.wrap(InterpretedFunction(space, obj)) -## elif inspect.isclass(obj): -## # XXX currently (rev 1020) unused, but it may be useful -## # to define builtin app-level classes at interp-level. -## return wrap_applevel_class(space, name, obj) -## else: -## raise ValueError, "cannot wrap %s, %s" % (name, obj) - -##def wrap_applevel_class(space, name, obj): -## """ construct an app-level class by reproducing the -## source definition and running it through the interpreter. -## It's a bit ugly but i don't know a better way (holger). -## """ -## assert 1!=1, "Oh you want to use this function?" -## l = ['class %s:' % name] -## indent = ' ' -## for key, value in vars(obj).items(): -## if hasattr(value, 'func_code'): -## s = inspect.getsource(value) -## l.append(s) -## indent = " " * (len(s) - len(s.lstrip())) - -## if getattr(obj, '__doc__', None): -## l.insert(1, indent + obj.__doc__) - -## for key, value in vars(obj).items(): -## if not key in ('__module__', '__doc__'): -## if isinstance(value, (str, int, float, tuple, list)): -## l.append('%s%s = %r' % (indent, key, value)) - -## s = "\n".join(l) -## code = compile(s, s, 'exec') -## scopedcode = ScopedCode(space, code, None) -## scopedcode.eval_frame() -## w_name = space.wrap(name) -## w_res = space.getitem(scopedcode.w_globals, w_name) -## return w_res - - -##def wrap_interplevel(space, name, obj): -## """ make an interp-level object accessible on app-level. """ -## return space.wrap(obj) - -#### Cells (used for nested scopes only) ## - -##_NULL = object() # Marker object - -##class Cell: -## def __init__(self, w_value=_NULL): -## self.w_value = w_value - -## def clone(self): -## return self.__class__(self.w_value) - -## def get(self): -## if self.w_value is _NULL: -## raise ValueError, "get() from an empty cell" -## return self.w_value - -## def set(self, w_value): -## self.w_value = w_value - -## def delete(self): -## if self.w_value is _NULL: -## raise ValueError, "make_empty() on an empty cell" -## self.w_value = _NULL - -## def __repr__(self): -## """ representation for debugging purposes """ -## if self.w_value is _NULL: -## return "%s()" % self.__class__.__name__ -## else: -## return "%s(%s)" % (self.__class__.__name__, self.w_value) + #def __getitem__(self, key): + # return self.content[key] + # + #def __iter__(self): + # return iter(self.content) + def __wrap__(self, space): + return self.makedict(space) + + def app2interp(self, app, app_name=None, **argflags): + """Build a Gateway that calls 'app' at app-level and insert it + into the DictProxy.""" + # app must be a function whose name starts with 'app_' + if not isinstance(app, types.FunctionType): + raise TypeError, "function expected, got %r instead" % app + if app_name is None: + if not app.func_name.startswith('app_'): + raise ValueError, ("function name must start with 'app_'; " + "%r does not" % app.func_name) + app_name = app.func_name[4:] + argflags1 = self.defaultargflags.copy() + argflags1.update(argflags) + code = pycode.PyCode(None) + code._from_code(app.func_code) + staticdefs = list(app.func_defaults or ()) + gateway = Gateway(code, self, staticdefs, **argflags1) + self.content[app_name] = gateway + return gateway + + def interp2app(self, f, **argflags): + """Insert an interp-level function into the DictProxy to make + it callable from app-level.""" + # f must be a function whose name does NOT starts with 'app_' + if not isinstance(f, types.FunctionType): + raise TypeError, "function expected, got %r instead" % f + assert not f.func_name.startswith('app_'), ( + "function name %r suspiciously starts with 'app_'" % f.func_name) + argflags1 = self.defaultargflags.copy() + argflags1.update(argflags) + builtincode = BuiltinCode(f, **argflags1) + staticdefs = list(f.func_defaults or ()) + gateway = Gateway(builtincode, self, staticdefs, **argflags1) + self.content[f.func_name] = gateway + + def exportname(self, name, obj, optional=0): + """Publish an object of the given name by inserting it into the + DictProxy. See implementation for the known types of 'obj'.""" + if name.startswith('app_'): + publicname = name[4:] + optional = 0 + else: + publicname = name + if isinstance(obj, types.FunctionType): + assert name == obj.func_name + if name == publicname: + # an interpreter-level function + self.interp2app(obj) + else: + # an app-level function + self.app2interp(obj) + elif not optional: + # assume a simple, easily wrappable object + self.content[publicname] = obj + # else skip the object if we cannot recognize it + + def exportall(self, d): + """Publish every object from a dict.""" + for name, obj in d.items(): + # ignore names in '_xyz' + if not name.startswith('_') or name.endswith('_'): + self.exportname(name, obj, optional=1) + + def importall(self, newd): + """Import all app_-level functions as Gateways into a dict. + Also import literals whose name starts with 'app_'.""" + for name, obj in newd.items(): + if name.startswith('app_') and name[4:] not in newd: + if isinstance(obj, types.FunctionType): + # an app-level function + assert name == obj.func_name + newd[name[4:]] = self.app2interp(obj) + else: + # assume a simple, easily wrappable object + newd[name[4:]] = obj + + def makedict(self, space, bind_instance=None): + """Turn the proxy into a normal dict. + Gateways to interpreter-level functions that were defined + with 'implicitself' are bound to 'bind_instance', so that + calling them from app-space will add this extra hidden argument.""" + w_dict = space.newdict([]) + for key, value in self.content.items(): + if isinstance(value, Gateway): + value = value.make_function(space, bind_instance, w_dict) + space.setitem(w_dict, space.wrap(key), space.wrap(value)) + return w_dict Modified: pypy/branch/builtinrefactor/pypy/interpreter/interactive.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/interactive.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/interactive.py Mon Sep 8 20:53:13 2003 @@ -34,12 +34,17 @@ self.space.__class__.__name__) code.InteractiveConsole.interact(self, banner) + def raw_input(self, prompt=""): + # add a character to the PyPy prompt so that you know where you + # are when you debug it with "python -i py.py" + return code.InteractiveConsole.raw_input(self, prompt[0] + prompt) + def runcode(self, code): - from pypy.interpreter.gateway import ScopedCode - scopedcode = ScopedCode(self.space, code, self.w_globals) - frame = scopedcode.create_frame() + # 'code' is a CPython code object + from pypy.interpreter.pycode import PyCode + pycode = PyCode()._from_code(code) try: - self.ec.eval_frame(frame) + pycode.exec_code(self.space, self.w_globals, self.w_globals) except baseobjspace.OperationError, operationerr: # XXX insert exception info into the application-level sys.last_xxx operationerr.print_detailed_traceback(self.space) Modified: pypy/branch/builtinrefactor/pypy/interpreter/main.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/main.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/main.py Mon Sep 8 20:53:13 2003 @@ -1,5 +1,5 @@ import autopath -from pypy.tool import test, option +from pypy.tool import option from pypy.interpreter import executioncontext, baseobjspace, gateway import sys, os @@ -23,18 +23,17 @@ w_mainmodule = space.newmodule(space.wrap("__main__")) w_globals = space.getattr(w_mainmodule, space.wrap("__dict__")) - space.setitem(w_globals, space.wrap("__builtins__"), space.w_builtins) except baseobjspace.OperationError, operationerr: operationerr.record_interpreter_traceback() raise baseobjspace.PyPyError(space, operationerr) else: - scopedcode = gateway.ScopedCode(space, space.unwrap(w_code), w_globals) - frame = scopedcode.create_frame() + pycode = space.unwrap(w_code) + retval = pycode.exec_code(space, w_globals, w_globals) if eval: - return ec.eval_frame(frame) + return retval else: - ec.eval_frame(frame) + return def run_string(source, filename='', space=None): _run_eval_string(source, filename, space, False) Copied: pypy/branch/builtinrefactor/pypy/interpreter/miscutils.py (from rev 1267, pypy/branch/builtinrefactor/pypy/interpreter/executioncontext.py) ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/miscutils.py Mon Sep 8 20:53:13 2003 @@ -1,54 +1,8 @@ -from pypy.interpreter import threadlocals +""" +Miscellaneous utilities. +""" -class ExecutionContext: - """An ExecutionContext holds the state of an execution thread - in the Python interpreter.""" - - def __init__(self, space): - # Note that self.framestack only contains PyFrames - self.space = space - self.framestack = Stack() - - def enter(self, frame): - locals = threadlocals.getlocals() - self.framestack.push(frame) - previous_ec = locals.executioncontext - locals.executioncontext = self - return previous_ec - - def leave(self, previous_ec): - locals.executioncontext = previous_ec - self.framestack.pop() - - def get_w_builtins(self): - if self.framestack.empty(): - return self.space.w_builtins - else: - return self.framestack.top().w_builtins - - def make_standard_w_globals(self): - "Create a new empty 'globals' dictionary." - w_key = self.space.wrap("__builtins__") - w_value = self.get_w_builtins() - w_globals = self.space.newdict([(w_key, w_value)]) - return w_globals - - def bytecode_trace(self, frame): - "Trace function called before each bytecode." - - def exception_trace(self, operationerr): - "Trace function called upon OperationError." - operationerr.record_interpreter_traceback() - #operationerr.print_detailed_traceback(self.space) - - def sys_exc_info(self): - """Implements sys.exc_info(). - Return an OperationError instance or None.""" - for i in range(self.framestack.depth()): - frame = self.framestack.top(i) - if frame.last_exception is not None: - return frame.last_exception - return None +import types class Stack: @@ -83,3 +37,27 @@ def empty(self): return not self.items + + +class InitializedClass(type): + """A meta-class that allows a class to initialize itself (or its + subclasses) by calling __initclass__() as a class method.""" + def __init__(self, name, bases, dict): + super(InitializedClass, self).__init__(name, bases, dict) + if hasattr(self, '__initclass__'): + raw = dict.get('__initclass__') + if isinstance(raw, types.FunctionType): + self.__initclass__ = classmethod(raw) + self.__initclass__() + + +class ThreadLocals: + """Thread-local storage.""" + + def __init__(self): + self.executioncontext = None + +# XXX no thread support yet, so this is easy :-) +_locals = ThreadLocals() +def getthreadlocals(): + return _locals Modified: pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py Mon Sep 8 20:53:13 2003 @@ -165,6 +165,3 @@ w_func = f.space.newfunction(f.space.unwrap(w_codeobj), f.w_globals, w_defaultarguments, w_freevars) f.valuestack.push(w_func) - - -PyNestedScopeFrame.setup_dispatch_table() Deleted: /pypy/branch/builtinrefactor/pypy/interpreter/opcode_app.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/interpreter/opcode_app.py Mon Sep 8 20:53:13 2003 +++ (empty file) @@ -1,185 +0,0 @@ -def prepare_raise0(): - import sys - return sys.exc_info() - -def prepare_raise(etype, value, traceback): -# import types - # XXX we get an infinite loop if this import fails: - # import types -> IMPORT_NAME -> import_name -> raise ImportError - # -> RAISE_VARARGS -> prepare_raise -> import types ... -# if not isinstance(traceback, (types.NoneType, types.TracebackType)): -# raise TypeError, "raise: arg 3 must be traceback or None" - while isinstance(etype, tuple): - etype = etype[0] - if type(etype) is str: - # warn - pass - elif isinstance(etype, Exception): - if value is not None: - raise TypeError, "instance exception may not have a separate value" - value = etype - etype = value.__class__ - elif isinstance(etype, type) and issubclass(etype, Exception): - if value is None: - value = () - elif not isinstance(value, tuple): - value = (value,) - value = etype(*value) - else: - raise TypeError, ("exceptions must be instances or subclasses of " - "Exception or strings (deprecated), not %s" % - (type(etype).__name__,)) - return etype, value, traceback - -def print_expr(x): - import sys - try: - displayhook = sys.displayhook - except AttributeError: - raise RuntimeError, "lost sys.displayhook" - displayhook(x) - - -def file_softspace(file, newflag): - try: - softspace = file.softspace - except AttributeError: - softspace = 0 - try: - file.softspace = newflag - except AttributeError: - pass - return softspace - -def print_item_to(x, stream): - if file_softspace(stream, False): - stream.write(" ") - stream.write(str(x)) - # add a softspace unless we just printed a string which ends in a '\t' - # or '\n' -- or more generally any whitespace character but ' ' -# if isinstance(x, str) and len(x) and x[-1].isspace() and x[-1]!=' ': -# return - # XXX add unicode handling - file_softspace(stream, True) - -def print_item(x): - import sys - try: - stream = sys.stdout - except AttributeError: - raise RuntimeError, "lost sys.stdout" - print_item_to(x, stream) - -def print_newline_to(stream): - stream.write("\n") - file_softspace(stream, False) - -def print_newline(): - import sys - try: - stream = sys.stdout - except AttributeError: - raise RuntimeError, "lost sys.stdout" - print_newline_to(stream) - -def import_name(builtins, modulename, globals, locals, fromlist): - try: - import_ = builtins["__import__"] - except KeyError: - raise ImportError, "__import__ not found" - return import_(modulename, globals, locals, fromlist) - -def import_all_from(module, locals): - try: - all = module.__all__ - except AttributeError: - try: - dict = module.__dict__ - except AttributeError: - raise ImportError, ("from-import-* object has no __dict__ " - "and no __all__") - all = dict.keys() - skip_leading_underscores = True - else: - skip_leading_underscores = False - for name in all: - if skip_leading_underscores and name[0]=='_': - continue - locals[name] = getattr(module, name) - -def import_from(module, name): - try: - return getattr(module, name) - except AttributeError: - raise ImportError, "cannot import name '%s'" % name - -def load_name(name, locals, globals, builtins): - try: - return locals[name] - except KeyError: - try: - return globals[name] - except KeyError: - try: - return builtins[name] - except KeyError: - raise NameError, "name '"+name+"' is not defined" - -def concatenate_arguments(args, extra_args): - return args + tuple(extra_args) - -def concatenate_keywords(kw, extra_kw): - if not isinstance(extra_kw, dict): - raise TypeError, "argument after ** must be a dictionary" - result = kw.copy() - for key, value in extra_kw.items(): - if key in result: - # XXX fix error message - raise TypeError, ("got multiple values " - "for keyword argument '%s'" % key) - result[key] = value - return result - -def exec_statement(prog, globals, locals, - builtins, caller_globals, caller_locals): - """Manipulate parameters to exec statement to (codeobject, dict, dict). - """ - import types - if (globals is None and locals is None and - isinstance(prog, builtins['tuple']) and - (len(prog) == 2 or len(prog) == 3)): - globals = prog[1] - if len(prog) == 3: - locals = prog[2] - prog = prog[0] - if globals is None: - globals = caller_globals - if locals is None: - locals = caller_locals - if locals is None: - locals = globals - if not isinstance(globals, dict): - raise TypeError("exec: arg 2 must be a dictionary or None") - elif not globals.has_key('__builtins__'): - globals['__builtins__'] = builtins - if not isinstance(locals, dict): - raise TypeError("exec: arg 3 must be a dictionary or None") -## # XXX - HACK to check for code object -## co = compile('1','','eval') -## print prog - if isinstance(prog, types.CodeType): - return (prog, globals, locals) - if not isinstance(prog, types.StringTypes): -## if not (isinstance(prog, types.StringTypes) or -## isinstance(prog, types.FileType)): - raise TypeError("exec: arg 1 must be a string, file, or code object") -## if isinstance(prog, types.FileType): -## flags = 0 -## ## XXX add in parent flag merging -## co = compile(prog.read(),prog.name,'exec',flags,1) -## return (co,globals,locals) - else: # prog is a string - flags = 0 - ## XXX add in parent flag merging - co = compile(prog,'','exec',flags,1) - return (co,globals,locals) Modified: pypy/branch/builtinrefactor/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/pycode.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/pycode.py Mon Sep 8 20:53:13 2003 @@ -19,7 +19,7 @@ class PyCode(eval.Code): "CPython-style code objects." - def __init__(self, co_name): + def __init__(self, co_name=''): eval.Code.__init__(self, co_name) self.co_argcount = 0 # #arguments, except *vararg and **kwarg self.co_nlocals = 0 # #local variables @@ -52,12 +52,10 @@ newconsts = () for const in code.co_consts: if isinstance(const, types.CodeType): - newc = PyCode(const.co_name) - newc._from_code(const) - newconsts = newconsts + (newc,) - else: - newconsts = newconsts + (const,) + const = PyCode()._from_code(const) + newconsts = newconsts + (const,) self.co_consts = newconsts + return self def create_frame(self, space, w_globals, closure=None): "Create an empty PyFrame suitable for this code object." @@ -89,3 +87,8 @@ def is_generator(self): return self.co_flags & CO_GENERATOR + + def dictscope_needed(self): + # regular functions always have CO_OPTIMIZED and CO_NEWLOCALS. + # class bodies only have CO_NEWLOCALS. + return not (self.co_flags & CO_OPTIMIZED) Modified: pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py Mon Sep 8 20:53:13 2003 @@ -1,10 +1,10 @@ """ PyFrame class implementation with the interpreter main loop. """ -from pypy.interpreter.executioncontext import Stack -from pypy.interpreter.error import OperationError from pypy.interpreter import eval, baseobjspace -from pypy.interpreter.gateway import noglobals +from pypy.interpreter.miscutils import Stack +from pypy.interpreter.error import OperationError +from pypy.interpreter.gateway import DictProxy class PyFrame(eval.Frame): @@ -23,12 +23,17 @@ """ def __init__(self, space, code, w_globals, closure): - eval.Frame.__init__(self, space, code, w_globals) + eval.Frame.__init__(self, space, code, w_globals, code.co_nlocals) self.valuestack = Stack() self.blockstack = Stack() self.last_exception = None self.next_instr = 0 self.w_builtins = self.space.w_builtins + # regular functions always have CO_OPTIMIZED and CO_NEWLOCALS. + # class bodies only have CO_NEWLOCALS. + if code.dictscope_needed(): + self.w_locals = space.newdict([]) # set to None by Frame.__init__ + ## def XXXclone(self): ## f = self.__class__() @@ -145,13 +150,16 @@ # push the exception to the value stack for inspection by the # exception handler (the code after the except:) operationerr = unroller.args[0] - operationerr.normalize(frame.space) + w_normalized = normalize_exception(frame.space, + operationerr.w_type, + operationerr.w_value) + w_type, w_value = frame.space.unpacktuple(w_normalized, 2) # the stack setup is slightly different than in CPython: # instead of the traceback, we store the unroller object, # wrapped. frame.valuestack.push(frame.space.wrap(unroller)) - frame.valuestack.push(operationerr.w_value) - frame.valuestack.push(operationerr.w_type) + frame.valuestack.push(w_value) + frame.valuestack.push(w_type) frame.next_instr = self.handlerposition # jump to the handler raise StopUnrolling @@ -174,7 +182,7 @@ else: raise Exception, "?!" # XXX return etype, evalue -normalize_exception = noglobals.app2interp(app_normalize_exception) +normalize_exception = DictProxy().app2interp(app_normalize_exception) class FinallyBlock(FrameBlock): Modified: pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py Mon Sep 8 20:53:13 2003 @@ -5,9 +5,10 @@ """ from pypy.interpreter.baseobjspace import OperationError, NoValue -import dis -from pypy.interpreter import baseobjspace -from pypy.interpreter.pyframe import PyFrame +from pypy.interpreter.eval import UNDEFINED +from pypy.interpreter import baseobjspace, pyframe +from pypy.interpreter.miscutils import InitializedClass +from pypy.interpreter.gateway import DictProxy class unaryoperation: @@ -30,14 +31,11 @@ f.valuestack.push(w_result) -class PyInterpFrame(PyFrame): - """A PyFrame that knows about interpretation of standard Python opcodes, - with the exception of nested scopes.""" - - def __init__(self, space, code, w_globals, closure): - PyFrame.__init__(self, space, code, w_globals, closure, - code.co_nlocals) # size of fastlocals_w array - +class PyInterpFrame(pyframe.PyFrame): + """A PyFrame that knows about interpretation of standard Python opcodes + minus the ones related to nested scopes.""" + appdict = DictProxy(implicitspace=True) + ### opcode dispatch ### # 'dispatch_table' is a class attribute: a list of functions. @@ -270,24 +268,23 @@ def PRINT_EXPR(f): w_expr = f.valuestack.pop() - #print f.space.unwrap(w_expr) - f.space.gethelper(appfile).call("print_expr", [w_expr]) + print_expr(f.space, w_expr) def PRINT_ITEM_TO(f): w_stream = f.valuestack.pop() w_item = f.valuestack.pop() - f.space.gethelper(appfile).call("print_item_to", [w_item, w_stream]) + print_item_to(f.space, w_item, w_stream) def PRINT_ITEM(f): w_item = f.valuestack.pop() - f.space.gethelper(appfile).call("print_item", [w_item]) + print_item_to(f.space, w_item, sys_stdout(f.space)) def PRINT_NEWLINE_TO(f): w_stream = f.valuestack.pop() - f.space.gethelper(appfile).call("print_newline_to", [w_stream]) + print_newline_to(f.space, w_stream) def PRINT_NEWLINE(f): - f.space.gethelper(appfile).call("print_newline", []) + print_newline_to(f.space, sys_stdout(f.space)) def BREAK_LOOP(f): raise pyframe.SBreakLoop @@ -299,26 +296,12 @@ # we use the .app.py file to prepare the exception/value/traceback # but not to actually raise it, because we cannot use the 'raise' # statement to implement RAISE_VARARGS - if nbargs == 0: - w_resulttuple = f.space.gethelper(appfile).call("prepare_raise0", []) - elif nbargs == 1: - w_type = f.valuestack.pop() - w_resulttuple = f.space.gethelper(appfile).call( - "prepare_raise", [w_type, f.space.w_None, f.space.w_None]) - elif nbargs == 2: - w_value = f.valuestack.pop() - w_type = f.valuestack.pop() - w_resulttuple = f.space.gethelper(appfile).call( - "prepare_raise", [w_type, w_value, f.space.w_None]) - elif nbargs == 3: - w_traceback = f.valuestack.pop() - w_value = f.valuestack.pop() - w_type = f.valuestack.pop() - w_resulttuple = f.space.gethelper(appfile).call( - "prepare_raise", [w_type, w_value, w_traceback]) - else: - raise pyframe.BytecodeCorruption, "bad RAISE_VARARGS oparg" - w_type, w_value, w_traceback = f.space.unpacktuple(w_resulttuple) + w_type = w_value = w_traceback = f.space.w_None + if nbargs >= 3: w_traceback = f.valuestack.pop() + if nbargs >= 2: w_value = f.valuestack.pop() + if nbargs >= 1: w_type = f.valuestack.pop() + w_resulttuple = prepare_raise(f.space, w_type, w_value, w_traceback) + w_type, w_value, w_traceback = f.space.unpacktuple(w_resulttuple, 3) # XXX the three-arguments 'raise' is not supported yet raise OperationError(w_type, w_value) @@ -338,20 +321,58 @@ w_locals = f.valuestack.pop() w_globals = f.valuestack.pop() w_prog = f.valuestack.pop() - w_tuple = f.space.gethelper(appfile).call("exec_statement", - [w_prog, w_globals, w_locals, - f.w_builtins, f.w_globals, f.w_locals]) - w_prog = f.space.getitem(w_tuple,f.space.wrap(0)) - w_globals = f.space.getitem(w_tuple,f.space.wrap(1)) - w_locals = f.space.getitem(w_tuple,f.space.wrap(2)) - - #plain = ... ... w_locals = f.getlocaldict() XXX XXX - - scopedcode = ScopedCode(f.space, f.space.unwrap(w_prog), w_globals) - scopedcode.eval_frame(w_locals) - #f.space.unwrap(w_prog).eval_code(f.space, w_globals, w_locals) + w_resulttuple = f.prepare_exec(w_prog, w_globals, w_locals) + w_prog, w_globals, w_locals = f.space.unpacktuple(w_resulttuple) + + plain = f.space.is_true(f.space.is_(w_locals, f.w_locals)) + if plain: + w_locals = f.getdictscope() + pycode = f.space.unwrap(w_prog) + pycode.exec_code(f.space, w_globals, w_locals) if plain: - f.setlocaldict(w_locals) + f.setdictscope(w_locals) + + def app_prepare_exec(f, prog, globals, locals): + """Manipulate parameters to exec statement to (codeobject, dict, dict). + """ + # XXX INCOMPLETE + if (globals is None and locals is None and + isinstance(prog, tuple) and + (len(prog) == 2 or len(prog) == 3)): + globals = prog[1] + if len(prog) == 3: + locals = prog[2] + prog = prog[0] + if globals is None: + globals = f.w_globals # XXX should be f.f_globals + if locals is None: + locals = f.w_locals # XXX should be f.f_locals + if locals is None: + locals = globals + if not isinstance(globals, dict): + raise TypeError("exec: arg 2 must be a dictionary or None") + elif not globals.has_key('__builtins__'): + globals['__builtins__'] = f.w_builtins # XXX should be f.f_builtins + if not isinstance(locals, dict): + raise TypeError("exec: arg 3 must be a dictionary or None") + # XXX - HACK to check for code object + co = compile('1','','eval') + if isinstance(prog, type(co)): + return (prog, globals, locals) + if not isinstance(prog, str): + ## if not (isinstance(prog, types.StringTypes) or + ## isinstance(prog, types.FileType)): + raise TypeError("exec: arg 1 must be a string, file, or code object") + ## if isinstance(prog, types.FileType): + ## flags = 0 + ## ## XXX add in parent flag merging + ## co = compile(prog.read(),prog.name,'exec',flags,1) + ## return (co,globals,locals) + else: # prog is a string + flags = 0 + ## XXX add in parent flag merging + co = compile(prog,'','exec',flags,1) + return (co, globals, locals) def POP_BLOCK(f): block = f.blockstack.pop() @@ -373,27 +394,12 @@ w_methodsdict = f.valuestack.pop() w_bases = f.valuestack.pop() w_name = f.valuestack.pop() - # XXX it would be best to have all opcodes on a class that has a 'space' attribute - # then the following initialization could be done at init-time. - build_class = InterpretedFunction(f.space.gethelperspace(), app_build_class) - w_newclass = build_class(w_name, w_bases, w_methodsdict, f.w_globals) + w_metaclass = find_metaclass(f.space, w_bases, + w_methodsdict, f.w_globals) + w_newclass = f.space.call_function(w_metaclass, w_name, + w_bases, w_methodsdict) f.valuestack.push(w_newclass) - def app_build_class(name, bases, namespace, globals): - if '__metaclass__' in namespace: - metaclass = namespace['__metaclass__'] - elif len(bases) > 0: - base = bases[0] - if hasattr(base, '__class__'): - metaclass = base.__class__ - else: - metaclass = type(base) - elif '__metaclass__' in globals: - metaclass = globals['__metaclass__'] - else: - metaclass = type - return metaclass(name, bases, namespace) - def STORE_NAME(f, varindex): varname = f.getname(varindex) w_varname = f.space.wrap(varname) @@ -474,9 +480,8 @@ # XXX the implementation can be pushed back into app-space as an # when exception handling begins to behave itself. For now, it # was getting on my nerves -- mwh - # w_value = f.space.gethelper(appfile).call( - # "load_name", [w_varname, f.w_locals, f.w_globals, f.w_builtins]) - # f.valuestack.push(w_value) + # w_value = f.load_name(w_varname) + # f.valuestack.push(w_value) def LOAD_GLOBAL(f, nameindex): assert f.w_globals is not None @@ -576,23 +581,31 @@ f.valuestack.push(w_result) def IMPORT_NAME(f, nameindex): + space = f.space modulename = f.getname(nameindex) - w_modulename = f.space.wrap(modulename) w_fromlist = f.valuestack.pop() - w_obj = f.space.gethelper(appfile).call( - "import_name", [f.w_builtins, - w_modulename, f.w_globals, f.w_locals, w_fromlist]) + try: + w_import = space.getitem(f.w_builtins, space.wrap("__import__")) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + raise OperationError(space.w_ImportError, + space.wrap("__import__ not found")) + w_obj = space.call_function(w_import, space.wrap(modulename), + f.w_globals, f.w_locals, w_fromlist) f.valuestack.push(w_obj) def IMPORT_STAR(f): w_module = f.valuestack.pop() - f.space.gethelper(appfile).call("import_all_from", [w_module, f.w_locals]) + w_locals = f.getfastscope() + import_all_from(f.space, w_module, w_locals) + f.setfastscope(w_locals) def IMPORT_FROM(f, nameindex): name = f.getname(nameindex) w_name = f.space.wrap(name) w_module = f.valuestack.top() - w_obj = f.space.gethelper(appfile).call("import_from", [w_module, w_name]) + w_obj = import_from(f.space, w_module, w_name) f.valuestack.push(w_obj) def JUMP_FORWARD(f, stepby): @@ -660,14 +673,27 @@ w_arguments = f.space.newtuple(arguments) w_keywords = f.space.newdict(keywords) if with_varargs: - w_arguments = f.space.gethelper(appfile).call("concatenate_arguments", - [w_arguments, w_varargs]) + w_arguments = f.update_star_args(w_arguments, w_varargs) if with_varkw: - w_keywords = f.space.gethelper(appfile).call("concatenate_keywords", - [w_keywords, w_varkw]) + w_keywords = f.update_keyword_args(w_keywords, w_varkw) w_result = f.space.call(w_function, w_arguments, w_keywords) f.valuestack.push(w_result) + def app_update_star_args(f, args, extra_args): + return args + tuple(extra_args) + + def app_update_keyword_args(f, kw, extra_kw): + if not isinstance(extra_kw, dict): + raise TypeError, "argument after ** must be a dictionary" + result = kw.copy() + for key, value in extra_kw.items(): + if key in result: + # XXX should mention the function name in error message + raise TypeError, ("got multiple values " + "for keyword argument '%s'" % key) + result[key] = value + return result + def CALL_FUNCTION(f, oparg): f.call_function_extra(oparg, False, False) @@ -715,24 +741,154 @@ def MISSING_OPCODE(f, oparg=None): raise pyframe.BytecodeCorruption, "unknown opcode" - ################################################################ + ### dispatch_table ### # 'dispatch_table' is a class attribute: a list of functions # it is created by 'cls.setup_dispatch_table()'. - def setup_dispatch_table(cls): + __metaclass__ = InitializedClass + def __initclass__(cls): # create the 'cls.dispatch_table' attribute + import dis dispatch_table = [] missing_opcode = cls.MISSING_OPCODE for i in range(256): opname = dis.opname[i].replace('+', '_') fn = getattr(cls, opname, missing_opcode) + fn = getattr(fn, 'im_func',fn) + fn.has_arg = i >= dis.HAVE_ARGUMENT #if fn is missing_opcode and not opname.startswith('<') and i>0: # import warnings # warnings.warn("* Warning, missing opcode %s" % opname) dispatch_table.append(fn) cls.dispatch_table = dispatch_table - setup_dispatch_table = classmethod(setup_dispatch_table) -PyInterpFrame.setup_dispatch_table() + appdict.importall(locals()) + + +### helpers written at the application-level ### +# Some of these functions are expected to be generally useful if other +# parts of the code needs to do the same thing as a non-trivial opcode, +# like finding out which metaclass a new class should have. +# This is why they are not methods of PyInterpFrame. +# There are also a couple of helpers that are methods, defined in the +# class above. + +def app_print_expr(x): + import sys + try: + displayhook = sys.displayhook + except AttributeError: + raise RuntimeError("lost sys.displayhook") + displayhook(x) + +def app_file_softspace(file, newflag): + try: + softspace = file.softspace + except AttributeError: + softspace = 0 + try: + file.softspace = newflag + except AttributeError: + pass + return softspace + +def app_sys_stdout(): + import sys + try: + return sys.stdout + except AttributeError: + raise RuntimeError("lost sys.stdout") + +def app_print_item_to(x, stream): + if file_softspace(stream, False): + stream.write(" ") + stream.write(str(x)) + # add a softspace unless we just printed a string which ends in a '\t' + # or '\n' -- or more generally any whitespace character but ' ' + # if isinstance(x, str) and len(x) and x[-1].isspace() and x[-1]!=' ': + # return + # XXX add unicode handling + file_softspace(stream, True) + +def app_print_newline_to(stream): + stream.write("\n") + file_softspace(stream, False) + +def app_prepare_raise(etype, value, traceback): + # careful if 'import types' is added here! + # we get an infinite loop if this import fails: + # import types -> IMPORT_NAME -> import_name -> raise ImportError + # -> RAISE_VARARGS -> prepare_raise -> import types ... + if etype is None: + # reraise + # XXX this means that "raise" is equivalent to "raise None" + # which is not the case in CPython, but well + import sys + etype, value, traceback = sys.exc_info() + #if not isinstance(traceback, (types.NoneType, types.TracebackType)): + # raise TypeError, "raise: arg 3 must be traceback or None" + while isinstance(etype, tuple): + etype = etype[0] + if type(etype) is str: + # XXX warn + pass + elif isinstance(etype, Exception): + if value is not None: + raise TypeError("instance exception may not have a separate value") + value = etype + etype = value.__class__ + elif isinstance(etype, type) and issubclass(etype, Exception): + if value is None: + value = () + elif not isinstance(value, tuple): + value = (value,) + value = etype(*value) + else: + raise TypeError("exceptions must be instances or subclasses of " + "Exception or strings (deprecated), not %s" % + (type(etype).__name__,)) + return etype, value, traceback + +def app_find_metaclass(bases, namespace, globals): + if '__metaclass__' in namespace: + return namespace['__metaclass__'] + elif len(bases) > 0: + base = bases[0] + if hasattr(base, '__class__'): + return base.__class__ + else: + return type(base) + elif '__metaclass__' in globals: + return globals['__metaclass__'] + else: + return type + +def app_import_all_from(module, into_locals): + try: + all = module.__all__ + except AttributeError: + try: + dict = module.__dict__ + except AttributeError: + raise ImportError("from-import-* object has no __dict__ " + "and no __all__") + all = dict.keys() + skip_leading_underscores = True + else: + skip_leading_underscores = False + for name in all: + if skip_leading_underscores and name[0]=='_': + continue + into_locals[name] = getattr(module, name) + +def app_import_from(module, name): + try: + return getattr(module, name) + except AttributeError: + raise ImportError("cannot import name '%s'" % name) + + +appdict = DictProxy() +appdict.importall(globals()) # app_xxx() -> xxx() Deleted: /pypy/branch/builtinrefactor/pypy/interpreter/threadlocals.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/interpreter/threadlocals.py Mon Sep 8 20:53:13 2003 +++ (empty file) @@ -1,13 +0,0 @@ -# Thread-local storage. - -# XXX no thread support yet, so this is easy :-) - -class ThreadLocals: - pass - -locals = ThreadLocals() -locals.executioncontext = None - - -def getlocals(): - return locals Modified: pypy/branch/builtinrefactor/pypy/interpreter/unittest_w.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/unittest_w.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/unittest_w.py Mon Sep 8 20:53:13 2003 @@ -2,28 +2,28 @@ import sys, os import unittest -#from pypy.interpreter.gateway import InterpretedMethod, InterpretedFunction -# FIX ME FIX ME FIX ME +from pypy.interpreter.gateway import DictProxy def make_testcase_class(space, tc_w): # XXX this is all a bit insane (but it works) - - #from pypy.interpreter.extmodule import make_builtin_func - w = space.wrap - d = space.newdict([]) - space.setitem(d, w('failureException'), space.w_AssertionError) + # space-independent part: collect the test methods into + # a DictProxy. + d = DictProxy(implicitspace=True) for name in dir(AppTestCase): if ( name.startswith('assert') or name.startswith('fail') and name != 'failureException'): - func = InterpretedMethod(space, getattr(tc_w, name).im_func) - w_func = space.wrap(func) - #w_func = wrap_func(space, getattr(tc_w, name).im_func) - space.setitem(d, w(name), w_func) + d.app2interp(getattr(tc_w, name).im_func, name) + + # space-dependent part: make an object-space-level dictionary + # and use it to build the class. + w = space.wrap + w_dict = d.makedict(space) + space.setitem(w_dict, w('failureException'), space.w_AssertionError) w_tc = space.call_function(space.w_type, w('TestCase'), space.newtuple([]), - d) + w_dict) return space.call_function(w_tc) @@ -34,9 +34,7 @@ self.testMethod = testMethod def __call__(self): - from pypy.interpreter import executioncontext space = self.testCase.space - w = space.wrap w_tc_attr = 'tc-attr-hacky-thing' if hasattr(space, w_tc_attr): @@ -45,7 +43,9 @@ w_tc = make_testcase_class(space, self.testCase) setattr(space, w_tc_attr, w_tc) - w_f = space.wrap(InterpretedFunction(space, self.testMethod.im_func)) + f = self.testMethod.im_func + gateway = DictProxy(implicitspace=True).app2interp(f, f.func_name) + w_f = space.wrap(gateway) # w_f = wrap_func(space, self.testMethod.im_func) space.call_function(w_f, w_tc) Modified: pypy/branch/builtinrefactor/pypy/module/builtin.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/module/builtin.py (original) +++ pypy/branch/builtinrefactor/pypy/module/builtin.py Mon Sep 8 20:53:13 2003 @@ -1,5 +1,6 @@ from pypy.interpreter import executioncontext -from pypy.interpreter.gateway import AppVisibleModule, ScopedCode +from pypy.interpreter.extmodule import ExtModule +from pypy.interpreter.error import OperationError ####################### #### __builtin__ #### @@ -7,7 +8,11 @@ import __builtin__ as cpy_builtin -class __builtin__(AppVisibleModule): +class __builtin__(ExtModule): + """ Template for PyPy's '__builtin__' module. + """ + + app___name__ = '__builtin__' def _actframe(self, index=-1): return self.space.getexecutioncontext().framestack.items[index] @@ -24,7 +29,7 @@ try: w_mod = space.getitem(space.sys.w_modules, w_modulename) return w_mod - except executioncontext.OperationError,e: + except OperationError,e: if not e.match(space, space.w_KeyError): raise w_mod = space.get_builtin_module(w_modulename) @@ -53,7 +58,7 @@ return w_mod w_exc = space.call_function(space.w_ImportError, w_modulename) - raise executioncontext.OperationError( + raise OperationError( space.w_ImportError, w_exc) def compile(self, w_str, w_filename, w_startstr, @@ -77,7 +82,8 @@ #print (str, filename, startstr, supplied_flags, dont_inherit) c = cpy_builtin.compile(str, filename, startstr, supplied_flags, dont_inherit) - return space.wrap(c) + from pypy.interpreter.pycode import PyCode + return space.wrap(PyCode()._from_code(c)) def execfile(self, w_filename, w_globals=None, w_locals=None): space = self.space @@ -361,7 +367,7 @@ else: return 1 - def app__vars(*obj): + def app_vars(*obj): """return a dictionary of all the attributes currently bound in obj. If called with no argument, return the variables bound in local scope.""" Modified: pypy/branch/builtinrefactor/pypy/module/sysmodule.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/module/sysmodule.py (original) +++ pypy/branch/builtinrefactor/pypy/module/sysmodule.py Mon Sep 8 20:53:13 2003 @@ -1,28 +1,32 @@ -from pypy.interpreter.baseobjspace import OperationError -from pypy.interpreter.gateway import AppVisibleModule +from pypy.interpreter.error import OperationError +from pypy.interpreter.extmodule import ExtModule import os, pypy import sys as cpy_sys -class sys(AppVisibleModule): - """ PyPy's 'sys' module. +class Sys(ExtModule): + """ Template for PyPy's 'sys' module. Currently we only provide 'path', 'modules', 'stdout' and 'displayhook' """ + app___name__ = 'sys' + def __init__(self, space): opd = os.path.dirname pypydir = opd(opd(os.path.abspath(pypy.__file__))) appdir = os.path.join(pypydir, 'pypy', 'appspace') self.path = [appdir] + [p for p in cpy_sys.path if p!= pypydir] self.w_modules = space.newdict([]) - AppVisibleModule.__init__(self, space) - - stdout = cpy_sys.stdout + ExtModule.__init__(self, space) + + stdout = cpy_sys.stdout + stderr = cpy_sys.stderr - def _setmodule(self, module): + def _setmodule(self, w_module): """ put a module into the modules list """ - self.space.setitem(self.w_modules, module.w___name__, module._wrapped) + w_name = self.space.getattr(w_module, self.space.wrap('__name__')) + self.space.setitem(self.w_modules, w_name, w_module) def displayhook(self, w_x): space = self.space Modified: pypy/branch/builtinrefactor/pypy/objspace/trivial.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/trivial.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/trivial.py Mon Sep 8 20:53:13 2003 @@ -4,62 +4,62 @@ # correctly wrap the exceptions. # -from pypy.interpreter import pyframe +from pypy.interpreter import pyframe, gateway from pypy.interpreter.baseobjspace import * -import operator, types, new, sys +import operator, types, new, sys, __builtin__ -class nugen(object): - def __init__(self, frame): - self.space = frame.space - self.frame = frame - self.running = 0 - - def next(self): - if self.running: - raise OperationError(self.space.w_ValueError, - "generator already executing") - ec = self.space.getexecutioncontext() +##class nugen(object): +## def __init__(self, frame): +## self.space = frame.space +## self.frame = frame +## self.running = 0 + +## def next(self): +## if self.running: +## raise OperationError(self.space.w_ValueError, +## "generator already executing") +## ec = self.space.getexecutioncontext() + +## self.running = 1 +## try: +## try: +## ret = ec.eval_frame(self.frame) +## except NoValue: +## raise StopIteration +## finally: +## self.running = 0 + +## return ret + +## def __iter__(self): +## return self + +##from pypy.interpreter.gateway import InterpretedFunction, InterpretedFunctionFromCode + +##class numeth(InterpretedFunction): +## def __init__(self, ifunc, instance, cls): +## self.ifunc = ifunc +## self.instance = instance +## self.cls = cls + +## def __call__(self, *args, **kws): +## if self.instance is None and self.cls is not type(None): +## pass +## else: +## args = (self.instance,) + args +## return self.ifunc(*args, **kws) + +##class nufun(InterpretedFunctionFromCode): + +## def __call__(self, *args, **kwargs): +## if self.cpycode.co_flags & 0x0020: +## frame = self.create_frame(args, kwargs) +## return nugen(frame) +## else: +## return self.eval_frame(args, kwargs) - self.running = 1 - try: - try: - ret = ec.eval_frame(self.frame) - except NoValue: - raise StopIteration - finally: - self.running = 0 - - return ret - - def __iter__(self): - return self - -from pypy.interpreter.gateway import InterpretedFunction, InterpretedFunctionFromCode - -class numeth(InterpretedFunction): - def __init__(self, ifunc, instance, cls): - self.ifunc = ifunc - self.instance = instance - self.cls = cls - - def __call__(self, *args, **kws): - if self.instance is None and self.cls is not type(None): - pass - else: - args = (self.instance,) + args - return self.ifunc(*args, **kws) - -class nufun(InterpretedFunctionFromCode): - - def __call__(self, *args, **kwargs): - if self.cpycode.co_flags & 0x0020: - frame = self.create_frame(args, kwargs) - return nugen(frame) - else: - return self.eval_frame(args, kwargs) - - def __get__(self, ob, cls=None): - return numeth(self, ob, cls) +## def __get__(self, ob, cls=None): +## return numeth(self, ob, cls) class TrivialObjSpace(ObjSpace): @@ -120,7 +120,6 @@ self.w_False = False self.w_NotImplemented = NotImplemented self.w_Ellipsis = Ellipsis - import __builtin__, types newstuff = {"False": self.w_False, "True" : self.w_True, "NotImplemented" : self.w_NotImplemented, @@ -144,7 +143,10 @@ # general stuff def wrap(self, x): - return x + if hasattr(x, '__wrap__'): + return x.__wrap__(self) + else: + return x def unwrap(self, w): return w @@ -171,88 +173,79 @@ nv = evalue raise OperationError(nt, nv) + def _auto(name, sourcefn, classlocals): + s = """ +def %(name)s(self, *args): + try: + value = %(sourcefn)s(*args) + except: + self.reraise() + return self.wrap(value) +""" % locals() + exec s in globals(), classlocals + # from the built-ins - id = id - type = type - issubtype = issubclass - newtuple = tuple - newlist = list - newdict = dict - newslice = slice # maybe moved away to application-space at some time - newmodule = new.module - iter = iter - repr = repr - str = str - len = len - pow = pow - divmod = divmod - hash = hash - setattr = setattr - delattr = delattr + _auto('issubtype', 'issubclass', locals()) + _auto('newmodule', 'new.module', locals()) + _auto('newtuple', 'tuple', locals()) + _auto('newlist', 'list', locals()) + _auto('newdict', 'dict', locals()) + _auto('newslice', 'slice', locals()) is_true = operator.truth # 'is_true' is not called 'truth' because it returns a *non-wrapped* boolean - def getattr(self, w_obj, w_name): - obj = self.unwrap(w_obj) - name = self.unwrap(w_name) - try: - return getattr(obj, name) - except: - self.reraise() + for _name in ('id', 'type', 'iter', 'repr', 'str', 'len', + 'pow', 'divmod', 'hash', 'setattr', 'delattr', 'hex', + 'oct', 'ord', 'getattr'): + _auto(_name, _name, locals()) for _name in ('pos', 'neg', 'not_', 'abs', 'invert', 'mul', 'truediv', 'floordiv', 'div', 'mod', 'add', 'sub', 'lshift', 'rshift', 'and_', 'xor', 'or_', 'lt', 'le', 'eq', 'ne', 'gt', 'ge', 'contains'): - exec """ -def %(_name)s(self, *args): - try: - return operator.%(_name)s(*args) - except: - self.reraise() -""" % locals() + _auto(_name, 'operator.' + _name, locals()) # in-place operators def inplace_pow(self, w1, w2): w1 **= w2 - return w1 + return self.wrap(w1) def inplace_mul(self, w1, w2): w1 *= w2 - return w1 + return self.wrap(w1) def inplace_truediv(self, w1, w2): w1 /= w2 # XXX depends on compiler flags - return w1 + return self.wrap(w1) def inplace_floordiv(self, w1, w2): w1 //= w2 - return w1 + return self.wrap(w1) def inplace_div(self, w1, w2): w1 /= w2 # XXX depends on compiler flags - return w1 + return self.wrap(w1) def inplace_mod(self, w1, w2): w1 %= w2 - return w1 + return self.wrap(w1) def inplace_add(self, w1, w2): w1 += w2 - return w1 + return self.wrap(w1) def inplace_sub(self, w1, w2): w1 -= w2 - return w1 + return self.wrap(w1) def inplace_lshift(self, w1, w2): w1 <<= w2 - return w1 + return self.wrap(w1) def inplace_rshift(self, w1, w2): w1 >>= w2 - return w1 + return self.wrap(w1) def inplace_and(self, w1, w2): w1 &= w2 - return w1 + return self.wrap(w1) def inplace_or(self, w1, w2): w1 |= w2 - return w1 + return self.wrap(w1) def inplace_xor(self, w1, w2): w1 ^= w2 - return w1 + return self.wrap(w1) # slicing @@ -274,9 +267,9 @@ sindex = self.old_slice(index) try: if sindex is None: - return obj[index] + return self.wrap(obj[index]) else: - return operator.getslice(obj, sindex[0], sindex[1]) + return self.wrap(operator.getslice(obj, sindex[0], sindex[1])) except: self.reraise() @@ -289,7 +282,7 @@ if sindex is None: obj[index] = value else: - return operator.setslice(obj, sindex[0], sindex[1], value) + operator.setslice(obj, sindex[0], sindex[1], value) except: self.reraise() @@ -308,7 +301,7 @@ # misc def next(self, w): try: - return w.next() + return self.wrap(w.next()) except StopIteration: raise NoValue @@ -319,7 +312,12 @@ #assert hasattr(code, 'co_name') #assert hasattr(code, 'build_arguments') #assert hasattr(code, 'eval_code') - return nufun(self, code, defs, globals, closure) + + # XXX the newxxx() interface should disappear at some point + # XXX because the interpreter can perfectly do the following + # XXX itself without bothering the object space + from pypy.interpreter.function import Function + return self.wrap(Function(self, code, globals, defs, closure)) def newstring(self, asciilist): try: @@ -336,13 +334,16 @@ self.reraise() if hasattr(r, '__init__'): self.call(r.__init__, args, kwds) - return r - if (isinstance(callable, types.MethodType) - and callable.im_self is not None): - args = (callable.im_self,) + args - callable = callable.im_func + return self.wrap(r) + #if (isinstance(callable, types.MethodType) + # and callable.im_self is not None): + # args = (callable.im_self,) + args + # callable = callable.im_func + assert not isinstance(callable, gateway.Gateway), ( + "trivial object space is detecting an object that has not " + "been wrapped") try: - return callable(*args, **(kwds or {})) + return self.wrap(callable(*args, **(kwds or {}))) except OperationError: raise except: @@ -351,32 +352,14 @@ #print "kwds", kwds self.reraise() - def hex(self, ob): - try: - return hex(ob) - except: - self.reraise() - - def oct(self, ob): - try: - return oct(ob) - except: - self.reraise() - - def ord(self, ob): - try: - return ord(ob) - except: - self.reraise() - def get(self, descr, ob, cls): try: - return descr.__get__(ob, cls) + return self.wrap(descr.__get__(ob, cls)) except: self.reraise() def new(self, type, args, kw): - return type(args, kw) + return self.wrap(type(args, kw)) def init(self, type, args, kw): pass From arigo at codespeak.net Tue Sep 9 21:38:31 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 9 Sep 2003 21:38:31 +0200 (MEST) Subject: [pypy-svn] rev 1278 - in pypy/branch/builtinrefactor/pypy: interpreterinterpreter/test objspace Message-ID: <20030909193831.5C3755A144@thoth.codespeak.net> Author: arigo Date: Tue Sep 9 21:38:29 2003 New Revision: 1278 Added: pypy/branch/builtinrefactor/pypy/interpreter/generator.py Modified: pypy/branch/builtinrefactor/pypy/interpreter/function.py pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py pypy/branch/builtinrefactor/pypy/interpreter/pycode.py pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py pypy/branch/builtinrefactor/pypy/interpreter/test/test_interpreter.py pypy/branch/builtinrefactor/pypy/interpreter/test/test_objspace.py pypy/branch/builtinrefactor/pypy/objspace/trivial.py Log: interpreter tests working again Modified: pypy/branch/builtinrefactor/pypy/interpreter/function.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/function.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/function.py Tue Sep 9 21:38:29 2003 @@ -25,7 +25,8 @@ def call(self, w_args, w_kwds=None): scope_w = self.parse_args(w_args, w_kwds) - frame = self.func_code.create_frame(self.space, self.w_globals) + frame = self.func_code.create_frame(self.space, self.w_globals, + self.closure) frame.setfastscope(scope_w) return frame.run() Added: pypy/branch/builtinrefactor/pypy/interpreter/generator.py ============================================================================== --- (empty file) +++ pypy/branch/builtinrefactor/pypy/interpreter/generator.py Tue Sep 9 21:38:29 2003 @@ -0,0 +1,88 @@ +from pypy.interpreter.error import OperationError +from pypy.interpreter.baseobjspace import NoValue +from pypy.interpreter.eval import Frame +from pypy.interpreter.pyframe import ControlFlowException, ExitFrame + +# +# Generator support. Note that GeneratorFrame is not a subclass of PyFrame. +# PyCode objects use a custom subclass of both PyFrame and GeneratorFrame +# when they need to interpret Python bytecode that is a generator. +# Otherwise, GeneratorFrame could also be used to define, say, +# built-in generators (which are usually done in CPython as functions +# that return iterators). +# + +class GeneratorFrame(Frame): + "A frame attached to a generator." + + def run(self): + "Build a generator-iterator." + self.exhausted = False + return GeneratorIterator(self) + + ### extra opcodes ### + + # XXX mmmh, GeneratorFrame is supposed to be independent from + # Python bytecode... Well, it is. These are not used when + # GeneratorFrame is used with other kinds of Code subclasses. + + def RETURN_VALUE(f): # overridden + raise SGeneratorReturn() + + def YIELD_VALUE(f): + w_yieldedvalue = f.valuestack.pop() + raise SYieldValue(w_yieldedvalue) + YIELD_STMT = YIELD_VALUE # misnamed in old versions of dis.opname + + +class GeneratorIterator(object): + "An iterator created by a generator." + + def __init__(self, frame): + self.frame = frame + self.running = False + + def nextvalue(self): + # raise NoValue when exhausted + if self.running: + space = self.frame.space + raise OperationError(space.w_ValueError, + space.wrap('generator already executing')) + if self.frame.exhausted: + raise NoValue + self.running = True + try: + return Frame.run(self.frame) + finally: + self.running = False + + + # XXX trick for trivialobjspace + # XXX make these __iter__() and next() app-visible + + def __iter__(self): + return self + + def next(self): + # XXX trivialobjspace only !! + try: + return self.nextvalue() + except NoValue: + raise StopIteration + +# +# the specific ControlFlowExceptions used by generators +# + +class SYieldValue(ControlFlowException): + """Signals a 'yield' statement. + Argument is the wrapped object to return.""" + def action(self, frame, last_instr, executioncontext): + w_yieldvalue = self.args[0] + raise ExitFrame(w_yieldvalue) + +class SGeneratorReturn(ControlFlowException): + """Signals a 'return' statement inside a generator.""" + def emptystack(self, frame): + frame.exhausted = True + raise NoValue Modified: pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py Tue Sep 9 21:38:29 2003 @@ -1,3 +1,4 @@ +from pypy.interpreter.error import OperationError from pypy.interpreter.eval import UNDEFINED from pypy.interpreter.pyopcode import PyInterpFrame @@ -56,6 +57,7 @@ raise OperationError(space.w_TypeError, "directly executed code object " "may not contain free variables") + closure = [] else: if len(closure) != nfreevars: raise ValueError("code object received a closure with " @@ -88,7 +90,7 @@ cell.set(w_value) def setfastscope(self, scope_w): - PyInterpFrame.setfastscope(scope_w) + PyInterpFrame.setfastscope(self, scope_w) if self.code.co_cellvars: # the first few cell vars could shadow already-set arguments, # in the same order as they appear in co_varnames @@ -158,10 +160,9 @@ nfreevars = len(codeobj.co_freevars) freevars = [f.valuestack.pop() for i in range(nfreevars)] freevars.reverse() - w_freevars = f.space.newtuple(freevars) defaultarguments = [f.valuestack.pop() for i in range(numdefaults)] defaultarguments.reverse() w_defaultarguments = f.space.newtuple(defaultarguments) w_func = f.space.newfunction(f.space.unwrap(w_codeobj), - f.w_globals, w_defaultarguments, w_freevars) + f.w_globals, w_defaultarguments, freevars) f.valuestack.push(w_func) Modified: pypy/branch/builtinrefactor/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/pycode.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/pycode.py Tue Sep 9 21:38:29 2003 @@ -60,11 +60,14 @@ def create_frame(self, space, w_globals, closure=None): "Create an empty PyFrame suitable for this code object." # select the appropriate kind of frame + from pypy.interpreter.pyopcode import PyInterpFrame as Frame if self.co_cellvars or self.co_freevars: from pypy.interpreter.nestedscope import PyNestedScopeFrame as F - else: - from pypy.interpreter.pyopcode import PyInterpFrame as F - return F(space, self, w_globals, closure) + Frame = enhanceclass(Frame, F) + if self.co_flags & CO_GENERATOR: + from pypy.interpreter.generator import GeneratorFrame as F + Frame = enhanceclass(Frame, F) + return Frame(space, self, w_globals, closure) def signature(self): "([list-of-arg-names], vararg-name-or-None, kwarg-name-or-None)." @@ -85,10 +88,23 @@ def getvarnames(self): return self.co_varnames - def is_generator(self): - return self.co_flags & CO_GENERATOR - def dictscope_needed(self): # regular functions always have CO_OPTIMIZED and CO_NEWLOCALS. # class bodies only have CO_NEWLOCALS. return not (self.co_flags & CO_OPTIMIZED) + + +def enhanceclass(baseclass, newclass, cache={}): + # this is a bit too dynamic for RPython, but it looks nice + # and I assume that we can easily change it into a static + # pre-computed table + if issubclass(newclass, baseclass): + return newclass + else: + try: + return cache[baseclass, newclass] + except KeyError: + class Mixed(newclass, baseclass): + pass + cache[baseclass, newclass] = Mixed + return Mixed Modified: pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py Tue Sep 9 21:38:29 2003 @@ -265,15 +265,6 @@ """Signals a 'return' statement. Argument is the wrapped object to return.""" def emptystack(self, frame): - if frame.code.is_generator(): - raise baseobjspace.NoValue - w_returnvalue = self.args[0] - raise ExitFrame(w_returnvalue) - -class SYieldValue(ControlFlowException): - """Signals a 'yield' statement. - Argument is the wrapped object to return.""" - def action(self, frame, last_instr, executioncontext): w_returnvalue = self.args[0] raise ExitFrame(w_returnvalue) Modified: pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py Tue Sep 9 21:38:29 2003 @@ -312,11 +312,6 @@ w_returnvalue = f.valuestack.pop() raise pyframe.SReturnValue(w_returnvalue) - def YIELD_VALUE(f): - w_yieldedvalue = f.valuestack.pop() - raise pyframe.SYieldValue(w_yieldedvalue) - YIELD_STMT = YIELD_VALUE # misnamed in dis.opname - def EXEC_STMT(f): w_locals = f.valuestack.pop() w_globals = f.valuestack.pop() Modified: pypy/branch/builtinrefactor/pypy/interpreter/test/test_interpreter.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/test/test_interpreter.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/test/test_interpreter.py Tue Sep 9 21:38:29 2003 @@ -19,9 +19,9 @@ w_tempmodule = space.newmodule(w("__temp__")) w_glob = space.getattr(w_tempmodule, w("__dict__")) space.setitem(w_glob, w("__builtins__"), space.w_builtins) - - scopedcode = gateway.ScopedCode(space, space.unwrap(w_code), w_glob) - scopedcode.eval_frame() + + code = space.unwrap(w_code) + code.exec_code(space, w_glob, w_glob) wrappedargs = w(args) wrappedfunc = space.getitem(w_glob, w(functionname)) Modified: pypy/branch/builtinrefactor/pypy/interpreter/test/test_objspace.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/test/test_objspace.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/test/test_objspace.py Tue Sep 9 21:38:29 2003 @@ -68,7 +68,8 @@ def f(x): return x - cpycode = f.func_code + from pypy.interpreter.pycode import PyCode + cpycode = PyCode()._from_code(f.func_code) w_globals = self.space.newdict([]) w_defs = self.space.newtuple([]) w_f = self.space.newfunction(cpycode, w_globals, w_defs) Modified: pypy/branch/builtinrefactor/pypy/objspace/trivial.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/trivial.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/trivial.py Tue Sep 9 21:38:29 2003 @@ -157,7 +157,9 @@ #ec = self.getexecutioncontext() # .framestack.items[-1] #ec.print_detailed_traceback(self) - etype, evalue = sys.exc_info()[:2] + etype, evalue, etb = sys.exc_info() + if etype is OperationError: + raise etype, evalue, etb # just re-raise it name = etype.__name__ if hasattr(self, 'w_' + name): nt = getattr(self, 'w_' + name) @@ -171,7 +173,7 @@ else: nt = etype nv = evalue - raise OperationError(nt, nv) + raise OperationError, OperationError(nt, nv), etb def _auto(name, sourcefn, classlocals): s = """ From arigo at codespeak.net Tue Sep 9 22:25:22 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 9 Sep 2003 22:25:22 +0200 (MEST) Subject: [pypy-svn] rev 1279 - in pypy/branch/builtinrefactor/pypy: interpreterinterpreter/test module Message-ID: <20030909202522.B95865A144@thoth.codespeak.net> Author: arigo Date: Tue Sep 9 22:25:22 2003 New Revision: 1279 Modified: pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py pypy/branch/builtinrefactor/pypy/interpreter/gateway.py pypy/branch/builtinrefactor/pypy/interpreter/test/test_executioncontext.py pypy/branch/builtinrefactor/pypy/module/builtin.py Log: test failures reduced Modified: pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py Tue Sep 9 22:25:22 2003 @@ -25,6 +25,8 @@ implicitself=True) # automatically publish all functions from this class cls._appdict.exportall(cls.__dict__) + # automatically import all app_ functions + cls._appdict.importall(cls.__dict__, cls) def __getattr__(self, attr): # XXX temporary, for use by objspace.trivial and Modified: pypy/branch/builtinrefactor/pypy/interpreter/gateway.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/gateway.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/gateway.py Tue Sep 9 22:25:22 2003 @@ -119,7 +119,8 @@ def make_function(self, space, bind_instance=None, w_globals=None): if w_globals is None: - w_globals = space.wrap(self.staticglobals) + #w_globals = space.wrap(self.staticglobals) + w_globals = self.staticglobals.makedict(space, bind_instance) defs_w = [space.wrap(def_value) for def_value in self.staticdefs] code = self.code if self.argflags.get('implicitself') and isinstance(code, BuiltinCode): @@ -163,11 +164,16 @@ args = args[1:] if not isinstance(space, ObjSpace): raise TypeError, "'space' expected as first argument" + wrap = space.wrap if self.gateway.argflags.get('implicitself'): pass # app-space gets no 'self' argument else: - args = (space.wrap(self.obj),) + args # insert 'w_self' - return self.gateway(space, *args, **kwds) + args = (wrap(self.obj),) + args # insert 'w_self' + w_args = [wrap(arg) for arg in args] + w_kwds = space.newdict([(wrap(key), wrap(value)) + for key, value in kwds.items()]) + fn = self.gateway.make_function(space, self.obj) + return fn.call(w_args, w_kwds) class DictProxy(Wrappable): @@ -265,18 +271,21 @@ if not name.startswith('_') or name.endswith('_'): self.exportname(name, obj, optional=1) - def importall(self, newd): + def importall(self, d, cls=None): """Import all app_-level functions as Gateways into a dict. Also import literals whose name starts with 'app_'.""" - for name, obj in newd.items(): - if name.startswith('app_') and name[4:] not in newd: + for name, obj in d.items(): + if name.startswith('app_') and name[4:] not in d: if isinstance(obj, types.FunctionType): # an app-level function assert name == obj.func_name - newd[name[4:]] = self.app2interp(obj) + obj = self.app2interp(obj) + else: + pass # assume a simple, easily wrappable object + if cls is None: + d[name[4:]] = obj else: - # assume a simple, easily wrappable object - newd[name[4:]] = obj + setattr(cls, name[4:], obj) def makedict(self, space, bind_instance=None): """Turn the proxy into a normal dict. Modified: pypy/branch/builtinrefactor/pypy/interpreter/test/test_executioncontext.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/test/test_executioncontext.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/test/test_executioncontext.py Tue Sep 9 22:25:22 2003 @@ -1,24 +1,31 @@ import autopath from pypy.tool import test -from pypy.interpreter.gateway import ScopedCode from pypy.interpreter import baseobjspace, executioncontext + + class TestExecutionContext(test.TestCase): - def test_trivial1(self): - # build frame - space = test.objspace() - ec = executioncontext.ExecutionContext(space) - compile = space.builtin.compile - bytecode = space.unwrap(compile(space.wrap('def f(x): return x+1'), - space.wrap(''), - space.wrap('exec'))).co_consts[0] - w_globals = ec.make_standard_w_globals() - w_locals = space.newdict([(space.wrap('x'), space.wrap(5))]) - scopedcode = ScopedCode(space, bytecode, w_globals) - w_output = scopedcode.eval_frame(w_locals) - self.assertEquals(space.unwrap(w_output), 6) + # + # XXX not sure what specifically this class used to test + # XXX turn this into a proper unit test + # + pass + +## def test_trivial1(self): +## # build frame +## space = test.objspace() +## ec = executioncontext.ExecutionContext(space) +## compile = space.builtin.compile +## bytecode = space.unwrap(compile(space.wrap('def f(x): return x+1'), +## space.wrap(''), +## space.wrap('exec'))).co_consts[0] +## w_globals = ec.make_standard_w_globals() +## w_locals = space.newdict([(space.wrap('x'), space.wrap(5))]) +## scopedcode = ScopedCode(space, bytecode, w_globals) +## w_output = scopedcode.eval_frame(w_locals) +## self.assertEquals(space.unwrap(w_output), 6) if __name__ == '__main__': Modified: pypy/branch/builtinrefactor/pypy/module/builtin.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/module/builtin.py (original) +++ pypy/branch/builtinrefactor/pypy/module/builtin.py Tue Sep 9 22:25:22 2003 @@ -1,3 +1,4 @@ +from __future__ import generators from pypy.interpreter import executioncontext from pypy.interpreter.extmodule import ExtModule from pypy.interpreter.error import OperationError @@ -37,29 +38,20 @@ space.setitem(space.sys.w_modules, w_modulename, w_mod) return w_mod - import os, __future__ - for path in space.unwrap(space.sys.w_path): + import os + for path in space.sys.path: f = os.path.join(path, space.unwrap(w_modulename) + '.py') if os.path.exists(f): w_mod = space.newmodule(w_modulename) space.setitem(space.sys.w_modules, w_modulename, w_mod) space.setattr(w_mod, w('__file__'), w(f)) - w_source = w(open(f, 'r').read()) - # wrt the __future__.generators.compiler_flag, "um" -- mwh - w_code = self.compile(w_source, w(f), w('exec'), - w(__future__.generators.compiler_flag)) w_dict = space.getattr(w_mod, w('__dict__')) - - code = space.unwrap(w_code) - from pypy.interpreter.gateway import ScopedCode - scopedcode = ScopedCode(space, code, w_dict) - scopedcode.eval_frame() - + import sys; print >> sys.stderr, self.__class__.__dict__['execfile'] + self.execfile(w(f), w_dict, w_dict) return w_mod w_exc = space.call_function(space.w_ImportError, w_modulename) - raise OperationError( - space.w_ImportError, w_exc) + raise OperationError(space.w_ImportError, w_exc) def compile(self, w_str, w_filename, w_startstr, w_supplied_flags=None, w_dont_inherit=None): @@ -85,23 +77,17 @@ from pypy.interpreter.pycode import PyCode return space.wrap(PyCode()._from_code(c)) - def execfile(self, w_filename, w_globals=None, w_locals=None): - space = self.space - #XXX why do i have to check against space.w_None instead of None? - # above the compile commands *does* check against None - if w_globals is space.w_None: - w_globals = self._actframe().w_globals - if w_locals is space.w_None: - w_locals = w_globals - - filename = space.unwrap(w_filename) - s = open(filename).read() - c = cpy_builtin.compile(s, filename, 'exec', 4096) # XXX generators - - - scopedcode = ScopedCode(space, c, w_globals) - scopedcode.eval_frame(w_locals) - return space.w_None + def app_execfile(filename, glob=None, loc=None): + if glob is None: + glob = globals() + if loc is None: + loc = locals() + elif loc is None: + loc = glob + f = file(filename) + source = f.read() + f.close() + exec source in glob, loc ####essentially implemented by the objectspace def abs(self, w_val): From jum at codespeak.net Thu Sep 11 00:10:28 2003 From: jum at codespeak.net (jum at codespeak.net) Date: Thu, 11 Sep 2003 00:10:28 +0200 (MEST) Subject: [pypy-svn] rev 1285 - pypy/trunk/doc/devel Message-ID: <20030910221028.890CA5ADE0@thoth.codespeak.net> Author: jum Date: Thu Sep 11 00:10:27 2003 New Revision: 1285 Modified: pypy/trunk/doc/devel/howtosvn.txt Log: Updated to the newest svn version. Modified: pypy/trunk/doc/devel/howtosvn.txt ============================================================================== --- pypy/trunk/doc/devel/howtosvn.txt (original) +++ pypy/trunk/doc/devel/howtosvn.txt Thu Sep 11 00:10:27 2003 @@ -110,13 +110,13 @@ -------------------------------------------------------------------------------- -.. _commandline: http://codespeak.net/~jum/svn-0.27.0-setup.exe +.. _commandline: http://codespeak.net/~jum/svn-0.29.0-setup.exe .. _website: http://support.microsoft.com/default.aspx?scid=kb%3Ben-us%3B259403 -.. _GUI: http://codespeak.net/~jum/TortoiseSVN-0.16-UNICODE.msi +.. _GUI: http://codespeak.net/~jum/TortoiseSVN-0.18-UNICODE.msi .. _Win: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=4B6140F9-2D36-4977-8FA1-6F8A0F5DCA8F -.. _MacOS: http://codespeak.net/~jum/svn-0.27.0-darwin-ppc.tar.gz +.. _MacOS: http://codespeak.net/~jum/svn-0.29.0-darwin-ppc.tar.gz .. _iconv: http://codespeak.net/~jum/iconv-darwin-ppc.tar.gz -.. _tarball: http://codespeak.net/~jum/subversion-0.27.0.tar.gz +.. _tarball: http://codespeak.net/~jum/subversion-0.29.0.tar.gz .. _guide: http://svnbook.red-bean.com/book.html#svn-ch-1 .. _archives: http://codespeak.net/pipermail/pypy-svn/ From lac at codespeak.net Sun Sep 14 10:26:52 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Sun, 14 Sep 2003 10:26:52 +0200 (MEST) Subject: [pypy-svn] rev 1289 - pypy/trunk/doc Message-ID: <20030914082652.2FF2D5AAE5@thoth.codespeak.net> Author: lac Date: Sun Sep 14 10:26:51 2003 New Revision: 1289 Added: pypy/trunk/doc/B2.0.0_background Log: This is the background doc. Added: pypy/trunk/doc/B2.0.0_background ============================================================================== --- (empty file) +++ pypy/trunk/doc/B2.0.0_background Sun Sep 14 10:26:51 2003 @@ -0,0 +1,163 @@ +Background: + +Python is a portable, interpreted, object-oriented Very-High Level +Language (VHLL). Its development started in 1990 at CWI, Centrum voor +Wiskunde en Informatica, the National Research Institute for +Mathematics and Computer Science in the Netherlands. Its principal +author is Guido van Rossum, a Dutch citizen currently living in +the United States. It is an Free/Open-Source language. The most +recent version of the language is Python 2.3, released under +the Python Software Foundation License, which is approved by both +the Open Source Initiative, and the Free Software Foundation. + +FIXME: Include this as a reference. +http://www.opensource.org/licenses/PythonSoftFoundation.php + +Python has tens, perhaps hundreds of thousands of active users +worldwide, which makes it one of the top ten most popular programming +languages in the world. [FIXME Footnote -- see section XXX and Appendex XXX +for support for this assertion]. C, C++, Java, Perl, and Visual Basic +are acknowledged to have a larger user base. But the languages which +most excite the Computer Science Research community - Self, Lisp, +Haskell, Limbo, and so on, are nowhere on this list. Thus European +economic competitiveness suffers. The innovative research lives in +academia, trapped in languages that are rarely used. + +Of those more popular languages, two, Java and Visual Basic are +proprietary. Sun Microsystems owns Java, and Microsoft owns Visual +Basic. Any company which writes its software in Java or Visual Basic +is at the mercy of these large American companies. And this is a +real, and not theoretical threat. In 2002, Microsoft announced that +it would no longer be supporting Visual Basic 6.0. after the year +2005. All Visual Basic Developers have been told to convert their +code to run under Microsoft's new .NET framework. In 2001 Microsoft +immediately stopped supporting its Visual J++ language, meant to be a +direct competitor with Java, because after settling a lawsuit with Sun +Microsystems. Microsoft is making these decisions because it makes +business sense for Microsoft, regardless of its effect on the businesses +who develop software using Microsoft proprietary software. + +In the face of these threats to the very survival of one's business, +European SMEs are moving to Free/Open Source languages such as Python. +In the year 2002, a group of SME's who rely on the Python program +language came together to form the Python Business Forum. +(www.python-in-business.org), at EuroPython, the European Python +Community Conference (www.europython.org). + +FIXME! include the bylaws of the PBF, its Board of Directors and +the like as an Appendex. + +While each SME member of the Python Business Forum has sufficient +faith in the Python programming language to use it for the development +of its own projects, it was agreed that there are defects in the +current implementation of the language. The two most often cited was +that the Python was too large for embedded applications and +applications designed for handhelds, and that the interpreter itself +ran too slowly. + +The developers of systems intended to run on tiny machines would like +a language with a 'smaller footprint'. They would like to strip out +everything which they do not need from the language and run with the +bare-bones minimum. This is hard to do in any language, and Python +was not implemented with this goal in mind. Another goal that was not +paramont was execution speed. Python is a dynamically-typed, +late-binding, interpreted language. Optimisation must be done at +run-time, and is notoriously more difficult to optimise than +statically typed, early-binding compiled languages such as C or C++. + +Meanwhile, on the German Python mailing list, +http://starship.python.net/mailman/listinfo/python-de an unrelated +discussion was about to bear fruit. FOOTNOTE: +Here's the mail that started it all, +http://starship.python.net/pipermail/python-de/2003q1/002925.html +include full text. + +Academically trained Python +programmers were pondering writing an implementation of Python in +itself. This group included Armin Rigo, author of Psyco +http://psyco.sourceforge.net/introduction.html, a specialising +Just-In-Time compiler for Python, and Christian Tismer, author of +Stackless Python, http://www.stackless.com/ an implementation of +Python which does not use the C stack. These people knew and were +interested in doing exactly that as part of the design. + +It is useful to quote Armin Rigo, stating the goals of Psyco in full, +from its website: + + My goal in programming Psyco is to contribute to reduce the + following wide gap between academic computer science and industrial + programming tools. + + While the former develops a number of programming languages + with very cool semantics and features, the latter stick with low-level + languages principally for performance reasons, on the ground that the + higher the level of a language, the slower it is. Althought clearly + justified in practice, this belief is theoretically false, and even + completely inverted --- for large, evolving systems like a whole + operating system and its applications, high-level programming can + deliver much higher performances. + + The new class of languages called "dynamic scripting + languages", of which Python is an example, is semantically close to + long-studied languages like Lisp. The constrains behind their designs + are however different: some high-level languages can be relatively + well statically compiled, we can do some type inference, and so on, + whereas with Python it is much harder --- the design goal was + different. We now have powerful machines to stick with interpretation + for a number of applications. This, of course, contributes to the + common belief that high-level languages are terribly slow. + + Psyco is both an academic and an industrial project. It is an + academic experiment testing some new techniques in the field of + on-line specialization. It develops an industrially useful + performance benefit for Python. And first of all it is a modest + step towards: + + High-level languages are faster than low-level ones! + +FIXME Footnote to: http://psyco.sourceforge.net/introduction.html + +PyPy would be a more ambitious step, of course. + +Also instrumental was Holger Krekel, at TrillkeGut a (what the heck +do you call it?? in Hildesheim). Holger Krekel was experienced +in one of the latest in Agile Software methodologies, Sprint-driven +development, from his experience in the Zope HOLGER +WRITE ME A PARAGRAPH HERE PLEASE ABOUT WHY THIS IS COOL + + +This was precisely what the Python Business Forum members had been +looking for. Academics conversant in the latest in compiler theory +who were interested in producing the Python interpreter which they +needed. We began discussing the possibility of collaboration, both +on mailing lists, FIXME list our mailing list and in our first +face-to-face meeting at Hildesheim, Germany Feb 17-23 2003. + +We decided to make a prototype to test the proof of concept, and our +ability to work together. We would get together every six weeks in +our own time, and work on the prototype in intense work sessions which +are known as 'Sprints'. FIXME Etymology of Sprint? After the first +Sprint at TrillkeGut in Hildesheim, we held subsequent Sprints at AB +Strakt in Gothenburg, Sweden, and at the University of +Louvain-La-Neuve in Belgium. + +By the time the Louvain-La-Neuve Sprint, which was held June 20-24, +ended, the PyPy project had produced a working prototype. There +existed a working interpreter and standard implementation of 90% of +the python types, as well as such language features as nested scopes, +generators and metaclasses. We had begun work on a type inference +engine. Perhaps most importantly, we had the enthusiastic support of +the Python community. Guido van Rossum, author of Python, not only +endorsed the PyPy project, but attended our Louvain-La-Neuve Sprint. +No work had been done on actually Optimising the code, so it ran +FIXME 30,000 times(?) slower than existing Python. But for a +proof of concept, approximately three weeks work total, it was +a success. + +It was time to look for funding. + +On 17th of June, that is to say exactly one week before, the 2nd Call +of the Information Society Technologies [IST] Priority went out. +Included in it was IST-2002-2.3.2.3 - Open development platforms +for software and services. We believe that what we intend to do +is perfectly addressed by this Call. From lac at codespeak.net Sun Sep 14 12:11:34 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Sun, 14 Sep 2003 12:11:34 +0200 (MEST) Subject: [pypy-svn] rev 1290 - pypy/trunk/doc Message-ID: <20030914101134.79AB95AAE5@thoth.codespeak.net> Author: lac Date: Sun Sep 14 12:11:33 2003 New Revision: 1290 Added: pypy/trunk/doc/B2.0_relevance Log: New section for the bid Added: pypy/trunk/doc/B2.0_relevance ============================================================================== --- (empty file) +++ pypy/trunk/doc/B2.0_relevance Sun Sep 14 12:11:33 2003 @@ -0,0 +1,139 @@ +B.2 Relevance to the objectives of the IST Priority + +Describe the manner in which the proposed project's goals address the +scientific, technical, wider societal and policy objectives of the IST +Priority in the areas concerned. + +(Recommended length ? three pages) +------------------------------------------- + +PyPy is a precise match for IST-2002-2.3.2.3 - Open development platforms +for software and services. Its stated objective is as follows: + + To build open development and run-time environments for software + and services providing the next generation of methodologies, + interoperable middleware and tools to support developers - through all + phases of the software life-cycle, from requirements analysis until + deployment and maintenance - in the production of networked and + distributed software systems and services, embedded software and + value-added user services. This will enable the development of future + software engineering methods and tools. + +The PyPy project aims to build an flexible and fast implementation of +the Open Source programming language Python written in itself. Python +is a Very High Level Language, a modern tool for developing software. +While Python is already suitable for developing software of all sizes, +this new implementation of Python would be particularly suitable for +the development of networked, distributed software systems, and +embedded software. It will facilitate the development of future +software engineering methods and tools. + +Priority is to be given to projects in which + + 'strong industrial users join forces with software and service + suppliers in building common platforms with support of academic + research partners'. + +This Project is a collaboration between Academic Researchers, and +Software and Service providers to produce an outcome desired by +all industrial users. ASK_STOCKHOLM 'what's an Industrial User?' +Is AB Strakt one? Large companies, who are not software providers, +such as Bang and Bang & Olufsen, the Danish maker of high-end audio +equipment, http://www.bang-olufsen.com/ have expressed interest in +using PyPy once it is developed. + +Moreover, our project is relevant to every focus objective. + +Focus is on: + + a) High level methods and concepts (esp. at requirements and + architectural level) for system design, development and + integration, addressing non-functional aspects, complexity, + autonomy and composability. + +PyPy will advance the 'state-of-the-art' in specializing compilers, +with a new architecture for Object Libraries that provide for radical +improvements in composability, autonomy, and composability. + +ASK_STOCKHOLM DO I HAVE TO EXPLAIN THIS MORE? + + b) Open and modular development environments, enabling + flexibility and extensibility with new or sector-specific + tools (e.g. intelligent distributed decision support), + supporting different adaptable development processes and + methodologies and ensuring consistency and traceability + across the development lifecycle. + +Flexibility is one of the Primary goals of PyPy. Our new architecture +will be the basis of a host of more flexible and extensible ways to +create software products. PyPy will, of course, be released as +Free/Open software, under the MIT license. + +FOOTNOTE: Stick MIT License in here. + +ASK_STOCKHOLM -- Am I correct? They want the thing we made to be +Open and Modular? Or are they asking that our own way of working uses +Open and Modular techniques? We do this as well .... + + c) Light/agile methodologies and adaptive workflow providing + a dynamic and adaptive environment, suitable for + co-operative and distributed development. + +Each member of the group is committed to Agile Methodolgies. Some of +us have experience with Crystal and others with XP, [FIXME add URLS +and see if we have any more trained people on the team]two of the more +popular ones. Moreover, since we have been active for many years in +the Open Source community, we are already used to co-operating with +others in a distributed development process. + +ASK_STOCKHOLM -- is this what they want, or more like this? + +AB Strakt is a young Swedish software company whose main product, CAPS +is an adaptive workflow framework. A faster Python would seamlessly improve +this product, and those applications which are based upon it. + +[FIXME: if we want this bit, we want to refere to the AB Strakt Appendex, that + says that we are a cool company, and that we do workflow and procurement, + and that we are responsible -- blah blah blah. I'll boast about + Strakt later, or see if I can steal some boasts from marketing.] + + d) Open platforms, middleware and languages supporting + standards for interoperability, composability and + integration. (incl. e.g. P2P, GRID, autonomy, agents, + dynamic adaptability and evolvability, context awareness, + customer profiles). Open source middleware layers can + facilitate rapid and broad adoption. + +PyPy is an Open Language. ASK_STOCKHOLM. So they have just asked us to +make them a languaqe? Or they just want us to use a language to make +what we are making? + + In addition, related foundational research, to be implemented by + Specific Targeted Research Projects and Coordination Actions, should + focus on fundamental design concepts, systematisation of domain + specifications, concurrency, distribution and timing, formal and + quantitative analysis and testing tools, and future database and + information system concepts. + +ASK_STOCKHOLM PyPy is Applied Research. When they say 'foundational +research' do they want 'foundational applied research', in which case +we can all cheer and say, here we are with a STREP, new foundational +research in the field of specializing compilers, just what you asked +for? Or by 'foundational research' do they mean the sort of basic +research that has no direct applications, but goes under the heading +'it is nice to learn stuff'? + + Work should where appropriate, enhance and complement work + implemented under EUREKA/ITEA and in software initiatives at member + and associated state level. The IST programme will seek active + co-operation with ITEA in software intensive systems. + +ASK_STOCKHOLM I don't know how to find out which work implemented under +EUREKA/ITEA was implemented using Python. I do know of 2 prior IST +projects. FIXME: add URL for ASWAD. A faster Python will enhance +any project done in Python. Also, if this means that EUREKA/ITEA +has more pots of money that they would like to hand us, we'd love +to take it. + + + From lac at codespeak.net Sun Sep 14 12:39:39 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Sun, 14 Sep 2003 12:39:39 +0200 (MEST) Subject: [pypy-svn] rev 1291 - pypy/trunk/doc/funding Message-ID: <20030914103939.3E94E5AAE5@thoth.codespeak.net> Author: lac Date: Sun Sep 14 12:39:38 2003 New Revision: 1291 Added: pypy/trunk/doc/funding/ pypy/trunk/doc/funding/B2.0.0_background.txt pypy/trunk/doc/funding/B2.0.0_relevance.txt Log: Made a funding dir, and renames these things as per Holger's suggestion. Since I messed up and made an extra heirarchy level, and then when I deleted them I got rid of the files, I had to recopy. Shame on me. Added: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Sun Sep 14 12:39:38 2003 @@ -0,0 +1,163 @@ +Background: + +Python is a portable, interpreted, object-oriented Very-High Level +Language (VHLL). Its development started in 1990 at CWI, Centrum voor +Wiskunde en Informatica, the National Research Institute for +Mathematics and Computer Science in the Netherlands. Its principal +author is Guido van Rossum, a Dutch citizen currently living in +the United States. It is an Free/Open-Source language. The most +recent version of the language is Python 2.3, released under +the Python Software Foundation License, which is approved by both +the Open Source Initiative, and the Free Software Foundation. + +FIXME: Include this as a reference. +http://www.opensource.org/licenses/PythonSoftFoundation.php + +Python has tens, perhaps hundreds of thousands of active users +worldwide, which makes it one of the top ten most popular programming +languages in the world. [FIXME Footnote -- see section XXX and Appendex XXX +for support for this assertion]. C, C++, Java, Perl, and Visual Basic +are acknowledged to have a larger user base. But the languages which +most excite the Computer Science Research community - Self, Lisp, +Haskell, Limbo, and so on, are nowhere on this list. Thus European +economic competitiveness suffers. The innovative research lives in +academia, trapped in languages that are rarely used. + +Of those more popular languages, two, Java and Visual Basic are +proprietary. Sun Microsystems owns Java, and Microsoft owns Visual +Basic. Any company which writes its software in Java or Visual Basic +is at the mercy of these large American companies. And this is a +real, and not theoretical threat. In 2002, Microsoft announced that +it would no longer be supporting Visual Basic 6.0. after the year +2005. All Visual Basic Developers have been told to convert their +code to run under Microsoft's new .NET framework. In 2001 Microsoft +immediately stopped supporting its Visual J++ language, meant to be a +direct competitor with Java, because after settling a lawsuit with Sun +Microsystems. Microsoft is making these decisions because it makes +business sense for Microsoft, regardless of its effect on the businesses +who develop software using Microsoft proprietary software. + +In the face of these threats to the very survival of one's business, +European SMEs are moving to Free/Open Source languages such as Python. +In the year 2002, a group of SME's who rely on the Python program +language came together to form the Python Business Forum. +(www.python-in-business.org), at EuroPython, the European Python +Community Conference (www.europython.org). + +FIXME! include the bylaws of the PBF, its Board of Directors and +the like as an Appendex. + +While each SME member of the Python Business Forum has sufficient +faith in the Python programming language to use it for the development +of its own projects, it was agreed that there are defects in the +current implementation of the language. The two most often cited was +that the Python was too large for embedded applications and +applications designed for handhelds, and that the interpreter itself +ran too slowly. + +The developers of systems intended to run on tiny machines would like +a language with a 'smaller footprint'. They would like to strip out +everything which they do not need from the language and run with the +bare-bones minimum. This is hard to do in any language, and Python +was not implemented with this goal in mind. Another goal that was not +paramont was execution speed. Python is a dynamically-typed, +late-binding, interpreted language. Optimisation must be done at +run-time, and is notoriously more difficult to optimise than +statically typed, early-binding compiled languages such as C or C++. + +Meanwhile, on the German Python mailing list, +http://starship.python.net/mailman/listinfo/python-de an unrelated +discussion was about to bear fruit. FOOTNOTE: +Here's the mail that started it all, +http://starship.python.net/pipermail/python-de/2003q1/002925.html +include full text. + +Academically trained Python +programmers were pondering writing an implementation of Python in +itself. This group included Armin Rigo, author of Psyco +http://psyco.sourceforge.net/introduction.html, a specialising +Just-In-Time compiler for Python, and Christian Tismer, author of +Stackless Python, http://www.stackless.com/ an implementation of +Python which does not use the C stack. These people knew and were +interested in doing exactly that as part of the design. + +It is useful to quote Armin Rigo, stating the goals of Psyco in full, +from its website: + + My goal in programming Psyco is to contribute to reduce the + following wide gap between academic computer science and industrial + programming tools. + + While the former develops a number of programming languages + with very cool semantics and features, the latter stick with low-level + languages principally for performance reasons, on the ground that the + higher the level of a language, the slower it is. Althought clearly + justified in practice, this belief is theoretically false, and even + completely inverted --- for large, evolving systems like a whole + operating system and its applications, high-level programming can + deliver much higher performances. + + The new class of languages called "dynamic scripting + languages", of which Python is an example, is semantically close to + long-studied languages like Lisp. The constrains behind their designs + are however different: some high-level languages can be relatively + well statically compiled, we can do some type inference, and so on, + whereas with Python it is much harder --- the design goal was + different. We now have powerful machines to stick with interpretation + for a number of applications. This, of course, contributes to the + common belief that high-level languages are terribly slow. + + Psyco is both an academic and an industrial project. It is an + academic experiment testing some new techniques in the field of + on-line specialization. It develops an industrially useful + performance benefit for Python. And first of all it is a modest + step towards: + + High-level languages are faster than low-level ones! + +FIXME Footnote to: http://psyco.sourceforge.net/introduction.html + +PyPy would be a more ambitious step, of course. + +Also instrumental was Holger Krekel, at TrillkeGut a (what the heck +do you call it?? in Hildesheim). Holger Krekel was experienced +in one of the latest in Agile Software methodologies, Sprint-driven +development, from his experience in the Zope HOLGER +WRITE ME A PARAGRAPH HERE PLEASE ABOUT WHY THIS IS COOL + + +This was precisely what the Python Business Forum members had been +looking for. Academics conversant in the latest in compiler theory +who were interested in producing the Python interpreter which they +needed. We began discussing the possibility of collaboration, both +on mailing lists, FIXME list our mailing list and in our first +face-to-face meeting at Hildesheim, Germany Feb 17-23 2003. + +We decided to make a prototype to test the proof of concept, and our +ability to work together. We would get together every six weeks in +our own time, and work on the prototype in intense work sessions which +are known as 'Sprints'. FIXME Etymology of Sprint? After the first +Sprint at TrillkeGut in Hildesheim, we held subsequent Sprints at AB +Strakt in Gothenburg, Sweden, and at the University of +Louvain-La-Neuve in Belgium. + +By the time the Louvain-La-Neuve Sprint, which was held June 20-24, +ended, the PyPy project had produced a working prototype. There +existed a working interpreter and standard implementation of 90% of +the python types, as well as such language features as nested scopes, +generators and metaclasses. We had begun work on a type inference +engine. Perhaps most importantly, we had the enthusiastic support of +the Python community. Guido van Rossum, author of Python, not only +endorsed the PyPy project, but attended our Louvain-La-Neuve Sprint. +No work had been done on actually Optimising the code, so it ran +FIXME 30,000 times(?) slower than existing Python. But for a +proof of concept, approximately three weeks work total, it was +a success. + +It was time to look for funding. + +On 17th of June, that is to say exactly one week before, the 2nd Call +of the Information Society Technologies [IST] Priority went out. +Included in it was IST-2002-2.3.2.3 - Open development platforms +for software and services. We believe that what we intend to do +is perfectly addressed by this Call. Added: pypy/trunk/doc/funding/B2.0.0_relevance.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B2.0.0_relevance.txt Sun Sep 14 12:39:38 2003 @@ -0,0 +1,139 @@ +B.2 Relevance to the objectives of the IST Priority + +Describe the manner in which the proposed project's goals address the +scientific, technical, wider societal and policy objectives of the IST +Priority in the areas concerned. + +(Recommended length ? three pages) +------------------------------------------- + +PyPy is a precise match for IST-2002-2.3.2.3 - Open development platforms +for software and services. Its stated objective is as follows: + + To build open development and run-time environments for software + and services providing the next generation of methodologies, + interoperable middleware and tools to support developers - through all + phases of the software life-cycle, from requirements analysis until + deployment and maintenance - in the production of networked and + distributed software systems and services, embedded software and + value-added user services. This will enable the development of future + software engineering methods and tools. + +The PyPy project aims to build an flexible and fast implementation of +the Open Source programming language Python written in itself. Python +is a Very High Level Language, a modern tool for developing software. +While Python is already suitable for developing software of all sizes, +this new implementation of Python would be particularly suitable for +the development of networked, distributed software systems, and +embedded software. It will facilitate the development of future +software engineering methods and tools. + +Priority is to be given to projects in which + + 'strong industrial users join forces with software and service + suppliers in building common platforms with support of academic + research partners'. + +This Project is a collaboration between Academic Researchers, and +Software and Service providers to produce an outcome desired by +all industrial users. ASK_STOCKHOLM 'what's an Industrial User?' +Is AB Strakt one? Large companies, who are not software providers, +such as Bang and Bang & Olufsen, the Danish maker of high-end audio +equipment, http://www.bang-olufsen.com/ have expressed interest in +using PyPy once it is developed. + +Moreover, our project is relevant to every focus objective. + +Focus is on: + + a) High level methods and concepts (esp. at requirements and + architectural level) for system design, development and + integration, addressing non-functional aspects, complexity, + autonomy and composability. + +PyPy will advance the 'state-of-the-art' in specializing compilers, +with a new architecture for Object Libraries that provide for radical +improvements in composability, autonomy, and composability. + +ASK_STOCKHOLM DO I HAVE TO EXPLAIN THIS MORE? + + b) Open and modular development environments, enabling + flexibility and extensibility with new or sector-specific + tools (e.g. intelligent distributed decision support), + supporting different adaptable development processes and + methodologies and ensuring consistency and traceability + across the development lifecycle. + +Flexibility is one of the Primary goals of PyPy. Our new architecture +will be the basis of a host of more flexible and extensible ways to +create software products. PyPy will, of course, be released as +Free/Open software, under the MIT license. + +FOOTNOTE: Stick MIT License in here. + +ASK_STOCKHOLM -- Am I correct? They want the thing we made to be +Open and Modular? Or are they asking that our own way of working uses +Open and Modular techniques? We do this as well .... + + c) Light/agile methodologies and adaptive workflow providing + a dynamic and adaptive environment, suitable for + co-operative and distributed development. + +Each member of the group is committed to Agile Methodolgies. Some of +us have experience with Crystal and others with XP, [FIXME add URLS +and see if we have any more trained people on the team]two of the more +popular ones. Moreover, since we have been active for many years in +the Open Source community, we are already used to co-operating with +others in a distributed development process. + +ASK_STOCKHOLM -- is this what they want, or more like this? + +AB Strakt is a young Swedish software company whose main product, CAPS +is an adaptive workflow framework. A faster Python would seamlessly improve +this product, and those applications which are based upon it. + +[FIXME: if we want this bit, we want to refere to the AB Strakt Appendex, that + says that we are a cool company, and that we do workflow and procurement, + and that we are responsible -- blah blah blah. I'll boast about + Strakt later, or see if I can steal some boasts from marketing.] + + d) Open platforms, middleware and languages supporting + standards for interoperability, composability and + integration. (incl. e.g. P2P, GRID, autonomy, agents, + dynamic adaptability and evolvability, context awareness, + customer profiles). Open source middleware layers can + facilitate rapid and broad adoption. + +PyPy is an Open Language. ASK_STOCKHOLM. So they have just asked us to +make them a languaqe? Or they just want us to use a language to make +what we are making? + + In addition, related foundational research, to be implemented by + Specific Targeted Research Projects and Coordination Actions, should + focus on fundamental design concepts, systematisation of domain + specifications, concurrency, distribution and timing, formal and + quantitative analysis and testing tools, and future database and + information system concepts. + +ASK_STOCKHOLM PyPy is Applied Research. When they say 'foundational +research' do they want 'foundational applied research', in which case +we can all cheer and say, here we are with a STREP, new foundational +research in the field of specializing compilers, just what you asked +for? Or by 'foundational research' do they mean the sort of basic +research that has no direct applications, but goes under the heading +'it is nice to learn stuff'? + + Work should where appropriate, enhance and complement work + implemented under EUREKA/ITEA and in software initiatives at member + and associated state level. The IST programme will seek active + co-operation with ITEA in software intensive systems. + +ASK_STOCKHOLM I don't know how to find out which work implemented under +EUREKA/ITEA was implemented using Python. I do know of 2 prior IST +projects. FIXME: add URL for ASWAD. A faster Python will enhance +any project done in Python. Also, if this means that EUREKA/ITEA +has more pots of money that they would like to hand us, we'd love +to take it. + + + From lac at codespeak.net Sun Sep 14 15:51:15 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Sun, 14 Sep 2003 15:51:15 +0200 (MEST) Subject: [pypy-svn] rev 1292 - pypy/trunk/doc/funding Message-ID: <20030914135115.EC5BE5AAE5@thoth.codespeak.net> Author: lac Date: Sun Sep 14 15:51:15 2003 New Revision: 1292 Modified: pypy/trunk/doc/funding/B2.0.0_background.txt pypy/trunk/doc/funding/B2.0.0_relevance.txt Log: Encorporate Alex's changes Modified: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_background.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Sun Sep 14 15:51:15 2003 @@ -15,32 +15,34 @@ Python has tens, perhaps hundreds of thousands of active users worldwide, which makes it one of the top ten most popular programming -languages in the world. [FIXME Footnote -- see section XXX and Appendex XXX -for support for this assertion]. C, C++, Java, Perl, and Visual Basic -are acknowledged to have a larger user base. But the languages which -most excite the Computer Science Research community - Self, Lisp, -Haskell, Limbo, and so on, are nowhere on this list. Thus European -economic competitiveness suffers. The innovative research lives in -academia, trapped in languages that are rarely used. +languages in the world. [FIXME Footnote -- see section XXX and Appendex +XXX for support for this assertion]. Some other languages, specifically +C, C++, Java, Perl, and Visual Basic, have even larger user bases. But +the languages which most excite the Computer Science Research community +-- Self, Lisp, Haskell, Limbo, ML, and so on, are nowhere on this list +-- yet they're the targets of most European academic research and +innovation. Thus European economic competitiveness suffers. The +innovative research lives in academia, trapped in languages that are +rarely used for commercial development. Of those more popular languages, two, Java and Visual Basic are proprietary. Sun Microsystems owns Java, and Microsoft owns Visual -Basic. Any company which writes its software in Java or Visual Basic -is at the mercy of these large American companies. And this is a -real, and not theoretical threat. In 2002, Microsoft announced that -it would no longer be supporting Visual Basic 6.0. after the year -2005. All Visual Basic Developers have been told to convert their -code to run under Microsoft's new .NET framework. In 2001 Microsoft -immediately stopped supporting its Visual J++ language, meant to be a -direct competitor with Java, because after settling a lawsuit with Sun -Microsystems. Microsoft is making these decisions because it makes -business sense for Microsoft, regardless of its effect on the businesses -who develop software using Microsoft proprietary software. +Basic. Any company which writes its software in Java or Visual Basic is +at the mercy of these large American companies. And this is a real, and +not theoretical threat. In 2002, Microsoft announced that it would no +longer be supporting Visual Basic 6.0. after the year 2005. All Visual +Basic Developers have been told to convert their code to run under +Microsoft's new .NET framework. In 2001 Microsoft immediately stopped +supporting its Visual J++ language, meant to be a direct competitor with +Java, after settling a lawsuit with Sun Microsystems. Microsoft is +making these decisions because they make business sense for Microsoft, +regardless of the effects on businesses who develop software using +Microsoft proprietary software. -In the face of these threats to the very survival of one's business, +In the face of these threats to the very survival of their businesses, European SMEs are moving to Free/Open Source languages such as Python. -In the year 2002, a group of SME's who rely on the Python program -language came together to form the Python Business Forum. +In the year 2002, a group of SME's who rely on the Python programming +language came together to form the Python Business Forum (www.python-in-business.org), at EuroPython, the European Python Community Conference (www.europython.org). @@ -55,15 +57,16 @@ applications designed for handhelds, and that the interpreter itself ran too slowly. -The developers of systems intended to run on tiny machines would like -a language with a 'smaller footprint'. They would like to strip out +The developers of systems intended to run on tiny machines would like a +language with a 'smaller footprint'. They would like to strip out everything which they do not need from the language and run with the -bare-bones minimum. This is hard to do in any language, and Python -was not implemented with this goal in mind. Another goal that was not -paramont was execution speed. Python is a dynamically-typed, -late-binding, interpreted language. Optimisation must be done at -run-time, and is notoriously more difficult to optimise than -statically typed, early-binding compiled languages such as C or C++. +bare-bones minimum. This is hard to do in any language, and Python was +not implemented with this goal in mind. Another goal that was not +paramount in Python's design and implementation was execution speed. +Python is a dynamically-typed, late-binding, interpreted language. +Optimisation must be done at run-time, and is notoriously more difficult +to optimise than statically typed, early-binding compiled languages such +as C or C++. Meanwhile, on the German Python mailing list, http://starship.python.net/mailman/listinfo/python-de an unrelated @@ -72,40 +75,43 @@ http://starship.python.net/pipermail/python-de/2003q1/002925.html include full text. -Academically trained Python -programmers were pondering writing an implementation of Python in -itself. This group included Armin Rigo, author of Psyco -http://psyco.sourceforge.net/introduction.html, a specialising -Just-In-Time compiler for Python, and Christian Tismer, author of -Stackless Python, http://www.stackless.com/ an implementation of -Python which does not use the C stack. These people knew and were -interested in doing exactly that as part of the design. +Academically trained Python programmers were pondering writing an +implementation of Python in itself. This group included Armin Rigo, +author of Psyco http://psyco.sourceforge.net/introduction.html, a +specialising Just-In-Time compiler for Python, and Christian Tismer, +author of Stackless Python, http://www.stackless.com/ an implementation +of Python which does not use the C stack. These people were intimately +familiar with both Python internals and advanced research in compilers +and runtime systems, and saw this self-hosted Python implementation as +a chance to put the two fields together. It is useful to quote Armin Rigo, stating the goals of Psyco in full, -from its website: +from the Psyco website: My goal in programming Psyco is to contribute to reduce the - following wide gap between academic computer science and industrial - programming tools. + following wide gap between academic computer science and + industrial programming tools. - While the former develops a number of programming languages - with very cool semantics and features, the latter stick with low-level - languages principally for performance reasons, on the ground that the - higher the level of a language, the slower it is. Althought clearly - justified in practice, this belief is theoretically false, and even - completely inverted --- for large, evolving systems like a whole - operating system and its applications, high-level programming can - deliver much higher performances. - - The new class of languages called "dynamic scripting - languages", of which Python is an example, is semantically close to - long-studied languages like Lisp. The constrains behind their designs - are however different: some high-level languages can be relatively - well statically compiled, we can do some type inference, and so on, - whereas with Python it is much harder --- the design goal was - different. We now have powerful machines to stick with interpretation - for a number of applications. This, of course, contributes to the - common belief that high-level languages are terribly slow. + While the former develops a number of programming languages with + very cool semantics and features, the latter stick with + low-level languages principally for performance reasons, on the + ground that the higher the level of a language, the slower it + is. Althought clearly justified in practice, this belief is + theoretically false, and even completely inverted --- for large, + evolving systems like a whole operating system and its + applications, high-level programming can deliver much higher + performances. + + The new class of languages called "dynamic scripting languages", + of which Python is an example, is semantically close to + long-studied languages like Lisp. The constraints behind their + designs are however different: some high-level languages can be + relatively well statically compiled, we can do some type + inference, and so on, whereas with Python it is much harder --- + the design goal was different. We now have powerful enough + machines to stick with interpretation for a number of + applications. This, of course, contributes to the common belief + that high-level languages are terribly slow. Psyco is both an academic and an industrial project. It is an academic experiment testing some new techniques in the field of @@ -142,17 +148,17 @@ Louvain-La-Neuve in Belgium. By the time the Louvain-La-Neuve Sprint, which was held June 20-24, -ended, the PyPy project had produced a working prototype. There -existed a working interpreter and standard implementation of 90% of -the python types, as well as such language features as nested scopes, -generators and metaclasses. We had begun work on a type inference -engine. Perhaps most importantly, we had the enthusiastic support of -the Python community. Guido van Rossum, author of Python, not only -endorsed the PyPy project, but attended our Louvain-La-Neuve Sprint. -No work had been done on actually Optimising the code, so it ran -FIXME 30,000 times(?) slower than existing Python. But for a -proof of concept, approximately three weeks work total, it was -a success. +ended, the PyPy project had produced a working prototype. There existed +a working interpreter and standard implementation of 90% of the python +types, as well as such language features as nested scopes, generators +and metaclasses. We had begun work on a type inference engine. Perhaps +most importantly, we had the enthusiastic support of the Python +community. Guido van Rossum, author of Python, not only endorsed the +PyPy project, but attended our Louvain-La-Neuve Sprint. No work had +been done on actually Optimising the code, so it ran FIXME 30,000 +times(?) slower than existing Python. But for a proof of concept, +approximately three weeks work total for a group of about a dozen +people, it was a success. It was time to look for funding. Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_relevance.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_relevance.txt Sun Sep 14 15:51:15 2003 @@ -51,9 +51,10 @@ integration, addressing non-functional aspects, complexity, autonomy and composability. -PyPy will advance the 'state-of-the-art' in specializing compilers, -with a new architecture for Object Libraries that provide for radical -improvements in composability, autonomy, and composability. +PyPy will advance the 'state-of-the-art' in specializing compilers, with +a new architecture for Object Libraries that provide for radical +improvements in (the management of) complexity, autonomy (self-hosting), +and composability (seamless modularity). ASK_STOCKHOLM DO I HAVE TO EXPLAIN THIS MORE? From lac at codespeak.net Sun Sep 14 19:02:03 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Sun, 14 Sep 2003 19:02:03 +0200 (MEST) Subject: [pypy-svn] rev 1293 - pypy/trunk/doc/funding Message-ID: <20030914170203.63DEF5AAE5@thoth.codespeak.net> Author: lac Date: Sun Sep 14 19:02:02 2003 New Revision: 1293 Added: pypy/trunk/doc/funding/B3.1_standards.txt Log: Will we effect standards Added: pypy/trunk/doc/funding/B3.1_standards.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B3.1_standards.txt Sun Sep 14 19:02:02 2003 @@ -0,0 +1,77 @@ +B.3.1 Contributions to standards: + +Describe contributions to national or international standards which +may be made by the project, if any. + +(Recommended length ? one page) + +There are currently two implementations of Python in common use. The +first one, which we will call CPython, but which is what the world +knows as Python, is a C implementation of the Python Programming +langauge which compiles to its own virtual machine. The second one is +Jython, a pure-Python implementation which compiles to the Java +virtual machine. There is no ANSI standard or similar for Python. +Right now the defacto standard for the programming language is +'whatever CPython does'. This is far from ideal, and questions arise, +especially from the developers of Jython as to which CPython language +behaviours are essential to the Python language itself, and which are +mere accidents of this particular implementation. + +For example, the garbage-collection behavior of CPython is implemented +by reference-counting ensuring that an object is finalized _as soon +as_ the last reference to it goes away. That would be extremely +inconvenient (close to impossible) to implement on standard Java +Virtual Machines, which have a deliberately under-specified garbage +collector (it can collect anything it pleases whenever it pleases...). +In this case, the Jython designers had to obtain an explicit ruling +from Guido van Rossum, Python's designer -- who ruled that the +behavior of CPython was 'accidental' in this case, and not intrinsic +to the Python language specification. + +Guido van Rossum has expressed interest in giving PyPy the status of +'implementation standard' (executable specification) of the Python +programming language. PyPy's ObjectSpaces flexibility will be crucial +in distinguishing "accidental" from "designed-in" characteristics. + +(FIXME: I sent Guido mail asking for something quotable. We will see +if he replies.) + +In order to do this we will have to do something which is called +'Submitting a PEP'. A PEP - Python Enhancement Proposal - is a design +document providing information to the Python community, or describing +a new feature for Python. There are two kinds of PEPs. A Standards +Track PEP describes a new feature or implementation for Python. An +Informational PEP describes a Python design issue, or provides general +guidelines or information to the Python community, but does not +propose a new feature. If we proposed to make PyPy the reference +standard of the Python language, we would, obviously, have to submit a +Standards Track PEP. + +PEP authors are responsible for collecting community feedback on a PEP +before submitting it for review. A PEP that has not been discussed on +python-list at python.org and/or python-dev at python.org will not be +accepted. After the authors believe that the PEP is ready, they must +inform the PEP editors (currently Barry Warsaw and David Goodger) that +it is ready for review. + +PEPs are reviewed by Guido van Rossum, the language author, and his chosen +consultants, who may accept or reject a PEP or send it back to the +author(s) for revision. + +If Guido van Rossum accepts the PEP, then its status is changed to +'Accepted'. If the reference implementation is not already complete, +it must then be completed. When the reference implementation is +complete and accepted (again by Guido van Rossum), the status will be +changed to 'Final'. In our case, we wouldn't even write the PEP +without having a complete reference implementation, since what we +would be proposing is to make PyPy and not CPython the complete +reference implementation of the language. Thus the expression of +interest from Guido van Rossum is of extreme significance. It is +likely, though not certain, that PyPy will become the standard +reference implementation of the Python language. + +The complete details of how to write a PEP are themselves an +Informational PEP -- PEP #1 in fact. Complete details are to +be found in this appendex: + +FIXME include http://www.python.org/peps/pep-0001.html From lac at codespeak.net Mon Sep 15 09:23:15 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Mon, 15 Sep 2003 09:23:15 +0200 (MEST) Subject: [pypy-svn] rev 1294 - pypy/trunk/doc/funding Message-ID: <20030915072315.DC2DD5AAE5@thoth.codespeak.net> Author: lac Date: Mon Sep 15 09:23:15 2003 New Revision: 1294 Added: pypy/trunk/doc/funding/B4.1_subcontracting.txt Log: A real short one. Just a question for Stockholm Added: pypy/trunk/doc/funding/B4.1_subcontracting.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B4.1_subcontracting.txt Mon Sep 15 09:23:15 2003 @@ -0,0 +1,17 @@ +B.4.1 Sub-contracting: + +If any part of the work is foreseen to be sub-contracted by the +participant responsible for it, describe the work involved and explain +why a sub-contract approach has been chosen for it. + +(Recommended length ? one page) + +I am going to run my accounting figures though the AB Strakt +bookkeeper, as part of my management duties. She is a contractor +who consults regularly to AB Strakt. Aside from that I don't +see that we will have any need for sub-contracting. Possibly to get +final stamps of approval on the books from KPMG our accounting +firm. This is the sort of thing that AB Strakt does all the time. + +ASK_STOCKHOLM Do I have to discuss this? + From lac at codespeak.net Mon Sep 15 12:29:13 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Mon, 15 Sep 2003 12:29:13 +0200 (MEST) Subject: [pypy-svn] rev 1295 - pypy/trunk/doc/funding Message-ID: <20030915102913.57B395AAE5@thoth.codespeak.net> Author: lac Date: Mon Sep 15 12:29:12 2003 New Revision: 1295 Added: pypy/trunk/doc/funding/B4.2_other_countries.txt Log: Just questions I want to not forget to ask Stockholm Added: pypy/trunk/doc/funding/B4.2_other_countries.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B4.2_other_countries.txt Mon Sep 15 12:29:12 2003 @@ -0,0 +1,20 @@ +B.4.2 Other countries : + +If one or more of the participants is based outside of the EU Member +and Associated states, explain in terms of the project's objectives +why this/these participants have been included, describe the level of +importance of their contribution to the project. + +(Recommended length ? one page). + +ASK_STOCKHOLM + +Samuele Pedroni is Swiss. But he says that he will move to do this +full time. What is the Status of Israel? Is a participant a +physical person, or a company? Does it matter where members sleep +at night? Could we just allocate funds to AB Strakt, say, and then +have Strakt go hire Samuele, letting him live and physically work +wherever he likes? What about Tim Peters, or some other American +we might want to invite? Is this sort of thing possible or +would Tim have to move? + From hpk at codespeak.net Mon Sep 15 12:29:42 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 15 Sep 2003 12:29:42 +0200 (MEST) Subject: [pypy-svn] rev 1296 - pypy/trunk/doc/funding Message-ID: <20030915102942.70CB15AAE5@thoth.codespeak.net> Author: hpk Date: Mon Sep 15 12:29:41 2003 New Revision: 1296 Modified: pypy/trunk/doc/funding/B2.0.0_background.txt pypy/trunk/doc/funding/B3.1_standards.txt Log: - fixed the standard-document to allows rest-generation - reworked the background document to be more religous and putting the EU at the center of PyPy development :-) Modified: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_background.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Mon Sep 15 12:29:41 2003 @@ -1,4 +1,8 @@ -Background: +B2.0.0 background +================= + +The relevance of Python +----------------------- Python is a portable, interpreted, object-oriented Very-High Level Language (VHLL). Its development started in 1990 at CWI, Centrum voor @@ -13,11 +17,12 @@ FIXME: Include this as a reference. http://www.opensource.org/licenses/PythonSoftFoundation.php -Python has tens, perhaps hundreds of thousands of active users +Python has tens, perhaps hundreds of thousands of active developers worldwide, which makes it one of the top ten most popular programming languages in the world. [FIXME Footnote -- see section XXX and Appendex XXX for support for this assertion]. Some other languages, specifically -C, C++, Java, Perl, and Visual Basic, have even larger user bases. But +C, C++, Java, Perl, and Visual Basic, have even larger user bases but they +are either proprietary or rather low-level languages. On the other hand, the languages which most excite the Computer Science Research community -- Self, Lisp, Haskell, Limbo, ML, and so on, are nowhere on this list -- yet they're the targets of most European academic research and @@ -39,6 +44,10 @@ regardless of the effects on businesses who develop software using Microsoft proprietary software. + +European SMEs are moving to Free/Open Source platforms +------------------------------------------------------ + In the face of these threats to the very survival of their businesses, European SMEs are moving to Free/Open Source languages such as Python. In the year 2002, a group of SME's who rely on the Python programming @@ -49,6 +58,9 @@ FIXME! include the bylaws of the PBF, its Board of Directors and the like as an Appendex. +Advancing the Python platform +----------------------------- + While each SME member of the Python Business Forum has sufficient faith in the Python programming language to use it for the development of its own projects, it was agreed that there are defects in the @@ -57,36 +69,34 @@ applications designed for handhelds, and that the interpreter itself ran too slowly. -The developers of systems intended to run on tiny machines would like a -language with a 'smaller footprint'. They would like to strip out +The developers of embedded systems intend to run on tiny machines would +like a language with a 'smaller footprint'. They would like to strip out everything which they do not need from the language and run with the bare-bones minimum. This is hard to do in any language, and Python was -not implemented with this goal in mind. Another goal that was not -paramount in Python's design and implementation was execution speed. -Python is a dynamically-typed, late-binding, interpreted language. -Optimisation must be done at run-time, and is notoriously more difficult -to optimise than statically typed, early-binding compiled languages such -as C or C++. - -Meanwhile, on the German Python mailing list, -http://starship.python.net/mailman/listinfo/python-de an unrelated -discussion was about to bear fruit. FOOTNOTE: -Here's the mail that started it all, -http://starship.python.net/pipermail/python-de/2003q1/002925.html -include full text. +not implemented with this goal in mind. -Academically trained Python programmers were pondering writing an -implementation of Python in itself. This group included Armin Rigo, -author of Psyco http://psyco.sourceforge.net/introduction.html, a -specialising Just-In-Time compiler for Python, and Christian Tismer, -author of Stackless Python, http://www.stackless.com/ an implementation -of Python which does not use the C stack. These people were intimately -familiar with both Python internals and advanced research in compilers -and runtime systems, and saw this self-hosted Python implementation as -a chance to put the two fields together. - -It is useful to quote Armin Rigo, stating the goals of Psyco in full, -from the Psyco website: +Another goal that was not paramount in Python's design and implementation +was execution speed. Python is a dynamically-typed, late-binding, +interpreted language. While this proved to provide extremely productive +development environments, execution speed sometimes is not fast enough. +Today, Optimisation of high-level languages must be done at run-time, and +is notoriously more difficult to optimise than statically typed, early-binding +compiled languages such as C or C++. + +Now a number of people and factors played together to start what is +now one of the most promising high-level-language projects. + +Some high-profile research +-------------------------- + +Independently some researchers who worked with Python were pondering +writing an implementation of Python in Python itself. This group included +Armin Rigo, author of PSYCO http://psyco.sourceforge.net/introduction.html, +a specialising Just-In-Time compiler for Python. He is intimately familiar +with both Python internals and advanced research in compilers and runtime +systems, and saw a Python implementation in Python itself as a chance to +put the two fields together. It is useful to quote from his webpage which +states the goals of his PSYCO in full: My goal in programming Psyco is to contribute to reduce the following wide gap between academic computer science and @@ -123,47 +133,126 @@ FIXME Footnote to: http://psyco.sourceforge.net/introduction.html -PyPy would be a more ambitious step, of course. +Although Armin Rigo proved with his PSYCO project that higher +level languages can actually be optimized to be as fast or faster than C, +he was very limited by the fact that Python is itself implemented in C +like almost all other languages in industrial use today. There was no +real-life project who tried to go all "optimize high-level down to machine-code" +way. Christian Tismer with his "Stackless" project had already come to a similar +conclusion from a more industrial viewpoint in that it is difficult to advance +language technology while relying on a large C-code base. + +Some mailing list discussions +----------------------------- + +On the German Python mailing list an independent discussion evolved +where developers and academics were pondering about the possibility +of developing a "minimal" python implemented in Python itself. Most +noticably Christian Tismer author of an industrial-use extension +("Stackless") of the Python language noted in a postscriptum of one +of his mails that having a minimized version of Python could provide +a new base to advance the language. Nevertheless, his extensions +to the language proved to be useful for companies who needed a way +to have millions of active objects and he had a branch of C-Python +to make this possible. + +Some organizational experience +------------------------------ + +Meanwhile Holger Krekel had joined the Python community in 2002. For some +years he had architected platforms and consulted CEO's for some large banking +centers in Europe. While participating two "coding sprints" of the Zope3 +web-platform (the to-be successor of the successful Zope web-platform) +he realized that "sprints" in combination with the rapid development language +Python provide an extremetly productive way of communicating about +and coding complex projects where proprietary methods often fail. + +Seeing the opportunity to setup the "PyPy" project with Armin Rigo +and Christian Tismer he offered to organize the first one-week meeting, +the "Sprint towards a minimal Python". Soon many interested developers +joined and intense academical and practical planning ensused. Just a +few weeks later the Sprint took place at "Trillke-Gut", a castle-like +building in Germany, bringing together some key developers, among them +Michael Hudson, the release manager of version 2.2.1 of Python. +Here is the mail that started this now rapidly evolving project +http://starship.python.net/pipermail/python-de/2003q1/002925.html + -Also instrumental was Holger Krekel, at TrillkeGut a (what the heck -do you call it?? in Hildesheim). Holger Krekel was experienced -in one of the latest in Agile Software methodologies, Sprint-driven -development, from his experience in the Zope HOLGER -WRITE ME A PARAGRAPH HERE PLEASE ABOUT WHY THIS IS COOL - - -This was precisely what the Python Business Forum members had been -looking for. Academics conversant in the latest in compiler theory -who were interested in producing the Python interpreter which they -needed. We began discussing the possibility of collaboration, both -on mailing lists, FIXME list our mailing list and in our first -face-to-face meeting at Hildesheim, Germany Feb 17-23 2003. - -We decided to make a prototype to test the proof of concept, and our -ability to work together. We would get together every six weeks in -our own time, and work on the prototype in intense work sessions which -are known as 'Sprints'. FIXME Etymology of Sprint? After the first -Sprint at TrillkeGut in Hildesheim, we held subsequent Sprints at AB -Strakt in Gothenburg, Sweden, and at the University of -Louvain-La-Neuve in Belgium. - -By the time the Louvain-La-Neuve Sprint, which was held June 20-24, -ended, the PyPy project had produced a working prototype. There existed -a working interpreter and standard implementation of 90% of the python -types, as well as such language features as nested scopes, generators -and metaclasses. We had begun work on a type inference engine. Perhaps -most importantly, we had the enthusiastic support of the Python -community. Guido van Rossum, author of Python, not only endorsed the -PyPy project, but attended our Louvain-La-Neuve Sprint. No work had -been done on actually Optimising the code, so it ran FIXME 30,000 -times(?) slower than existing Python. But for a proof of concept, -approximately three weeks work total for a group of about a dozen -people, it was a success. +Some promising open-source developments tools +--------------------------------------------- + +Moreover, from the start the PyPy developers were committed to bringing +together and utilizing most promising open-source technologies. Here +Jens-Uwe Mager, a retired CTO from Helios (http://www.helios.de) came in and +helped setting up a state-of-the-art open-source development environment. +With his 12-year experience of setting up and leading a SME-company which +is one of the worldwide leaders in Print-Preprocessing technology +he helped organizing the development and net-connectivity for the +various web services needed by the PyPy developers. + +Research, pragmatism and industry experience combined +----------------------------------------------------- + +This combination of research, applicability and industrial experience +was precisely what the Python Business Forum members had been +looking for. Academics, developers and practioners conversant in the +latest in language and platform architecture who were interested in +producing the Python interpreter which they needed. Laura Creighton and +Jacob Hallen from the PBF attended the sprint and began to communicate +and participate with the project. + +The group decided soon to test their proof of concept by developing +a working prototype and to test their ability to work together. It would +get together in their own private time, and work on the prototype in intense +pair-progamming work sessions. After the first Sprint at TrillkeGut in +Hildesheim, Jacob Hallen and Laura Creighton organized the next Sprint +at AB Strakt in Gothenburg, Sweden which helped to keep up the momentum +that the PyPy development group had gained. Also Samuele Pedroni, the +lead developer of Jython (a tight industrial-use integration of Java and +Python), joined the project out of research and practical interest too see +that PyPy would help with his own java-integration project. With all +these experienced people, by the end of the week you could already run +simple Python programs within PyPy. + +The third sprint followed close just before the "EuroPython" conference +in Belgium. the sprint was organized by interested developers in Belgium in +Louvain-La-Neuve, a university city. In another move, Laura Creighton took +the opportunity during her visit in Washington D.C. (XXX) to invite Guido van Rossum, +the inventor and principal of Python. He not only attended the belgium sprint +but announced a week later at the EuroPython conference that "PyPy" has a high +priority on his "dreams hopefully coming true"-list and he enjoyed it a lot. + +By the end of the third one-week sprint in Louvain-La-Neuve, which was held June 20-24, +the PyPy project had already produced a fully working prototype. There +existed a working interpreter and standard implementation of 90% of the python +types, as well as advanced language features such as nested scopes, generators +and metaclasses. Armin Rigo and Guido van Rossum started working on a type-inference +engine which is the foundation for the final missing step: generating a native +machine-level version of Python from its high-level PyPy-implementation. +Perhaps most importantly, we had the enthusiastic support of the Python +community on the EuroPython conference. + +No work had been done on actually optimising the code so it ran around 30,000 +times slower than the existing CPython-implementation but this was to be expected +from the start. Nevertheless, for a proof of concept, approximately four weeks work +total for a group of about a dozen people, it was clearly a success. + +Breaking the last barrier +------------------------- + +However, almost all of the participants realized after three one-week sprints +in just six month that their other obligations (full-time jobs in mostly unrelated +areas!) would not allow them to continue at this pace. But everyone involved understood +that this new Python implementation opened up extremely promising possibilities. It was time to look for funding. -On 17th of June, that is to say exactly one week before, the 2nd Call -of the Information Society Technologies [IST] Priority went out. -Included in it was IST-2002-2.3.2.3 - Open development platforms -for software and services. We believe that what we intend to do -is perfectly addressed by this Call. +While some people suggested to apply to DARPA for funding we realized +that most of us were rooted in Europe and it would make more sense to +look for european possibilities. + +In June the maybe most important development took place. The 2nd Call of the +Information Society Technologies [IST] Priority went out. Included in it was +IST-2002-2.3.2.3 - Open development platforms for software and services. + +We believe that what we intend to do is a perfect fit for the goals of this call. Modified: pypy/trunk/doc/funding/B3.1_standards.txt ============================================================================== --- pypy/trunk/doc/funding/B3.1_standards.txt (original) +++ pypy/trunk/doc/funding/B3.1_standards.txt Mon Sep 15 12:29:41 2003 @@ -1,4 +1,5 @@ -B.3.1 Contributions to standards: +B.3.1 Contributions to standards +----------------------------------- Describe contributions to national or international standards which may be made by the project, if any. @@ -18,8 +19,8 @@ mere accidents of this particular implementation. For example, the garbage-collection behavior of CPython is implemented -by reference-counting ensuring that an object is finalized _as soon -as_ the last reference to it goes away. That would be extremely +by reference-counting ensuring that an object is finalized as soon +as the last reference to it goes away. That would be extremely inconvenient (close to impossible) to implement on standard Java Virtual Machines, which have a deliberately under-specified garbage collector (it can collect anything it pleases whenever it pleases...). From lac at codespeak.net Mon Sep 15 12:45:54 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Mon, 15 Sep 2003 12:45:54 +0200 (MEST) Subject: [pypy-svn] rev 1297 - pypy/trunk/doc/funding Message-ID: <20030915104554.653AD5AAE5@thoth.codespeak.net> Author: lac Date: Mon Sep 15 12:45:53 2003 New Revision: 1297 Modified: pypy/trunk/doc/funding/B4.2_other_countries.txt Log: Thought of another question. Modified: pypy/trunk/doc/funding/B4.2_other_countries.txt ============================================================================== --- pypy/trunk/doc/funding/B4.2_other_countries.txt (original) +++ pypy/trunk/doc/funding/B4.2_other_countries.txt Mon Sep 15 12:45:53 2003 @@ -16,5 +16,6 @@ have Strakt go hire Samuele, letting him live and physically work wherever he likes? What about Tim Peters, or some other American we might want to invite? Is this sort of thing possible or -would Tim have to move? +would Tim have to move? If Samuele is moving, can we get the +Marie Curie funds to pay for it? From hpk at codespeak.net Mon Sep 15 12:53:10 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 15 Sep 2003 12:53:10 +0200 (MEST) Subject: [pypy-svn] rev 1299 - pypy/trunk/doc/EU_funding Message-ID: <20030915105310.2872F5AAE5@thoth.codespeak.net> Author: hpk Date: Mon Sep 15 12:53:09 2003 New Revision: 1299 Removed: pypy/trunk/doc/EU_funding/ Log: got rid of EU_funding directory (and realized i could have done it with the last commit if i had hand-removed some non-versioned files in that directory) From lac at codespeak.net Mon Sep 15 14:12:22 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Mon, 15 Sep 2003 14:12:22 +0200 (MEST) Subject: [pypy-svn] rev 1301 - pypy/trunk/doc/funding Message-ID: <20030915121222.712095AAE5@thoth.codespeak.net> Author: lac Date: Mon Sep 15 14:12:21 2003 New Revision: 1301 Added: pypy/trunk/doc/funding/strep_boilerplate.txt Log: This is the boilerplate for STRP section B Added: pypy/trunk/doc/funding/strep_boilerplate.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/strep_boilerplate.txt Mon Sep 15 14:12:21 2003 @@ -0,0 +1,213 @@ +Annex 2 - Proposal Part B +IST Priority - ? 2nd call - Guide for Proposers ? Specific Targeted Research +Projects ? June 2003 + +Instructions for preparing proposal Part B for +Specific targeted research projects in the IST Priority + +In addition to the detailed technical information provided in Part B, +a proposal must also contain a Part A, containing basic information on +the proposal and the consortium making the proposal1. The forms for +Part A are provided elsewhere in this Guide. Incomplete proposals are +not eligible and will not be evaluated + + +Specific targeted research projects will aim at improving European +competitiveness. They will be sharply focused and will take either of +the following two forms, or a combination of the two: + +(a) a research and technological development project designed to gain + new knowledge either to improve considerably or to develop new products, + processes or services or to meet other needs of society and community + policies; + +(b) a demonstration project designed to prove the viability of new + technologies offering potential economic advantage but which cannot + be commercialised directly. + +Proposers should particularly note that only research, demonstration and +project management activities are funded in STREPs. Training activities +cannot be included among the eligible costs of projects. + +Information which fully details what a STREP comprises and how such a +project should be implemented can be found at +http://www.cordis.lu/fp6/instrument-strp/. Proposers should study this +information thoroughly before commencing the preparation of their proposal + +Part B. + +Front page +Proposal full title +Proposal acronym +Date of preparation +Type of instrument in this case: Specific targeted research project +List of participants + +Participant no. Participant name Participant short name +1 (coordinator) +2 +3 +4 + etc +Coordinator name +Coordinator organisation name +Coordinator email +Coordinator fax + +Contents page + show contents list + +Proposal summary page +Proposal full title +Proposal acronym +Strategic objectives addressed + (If more than one objective , indicate their order of importance + to the project). +Proposal abstract copied from Part A (if not in English, include an + English translation) + +B.1 Scientific and technological objectives of the project and state of + the art In the event of inconsistency between information given in + Part A and that given in Part B, the Part A version will prevail + +Describe in detail the proposed project's S&T objectives. The +objectives should be those achievable within the project, not through +subsequent development, and should be stated in a measurable and +verifiable form. The progress of the project work will be measured +against these goals in later reviews and assessments. Describe the +state-of-the-art in the area concerned and how the proposed project +will enhance the state-of-the-art in that area. + +(Recommended length ? three pages) + +B.2 Relevance to the objectives of the IST Priority + +Describe the manner in which the proposed project's goals address the +scientific, technical, wider societal and policy objectives of the IST +Priority in the areas concerned. + +(Recommended length ? three pages) + +B.3 Potential impact + +Describe the strategic impact of the proposed project, for example in +reinforcing competitiveness or on solving societal problems. Describe +the innovation-related activities. Describe the exploitation and/or +dissemination plans which are foreseen to ensure use of the project +results. Describe the added-value in carrying out the work at a +European level. Indicate what account is taken of other national or +international research activities. + +(Recommended length ? three pages) + +B.3.1 Contributions to standards: + +Describe contributions to national or international standards which +may be made by the project, if any. + +(Recommended length ? one page) + +B.4 The consortium and project resources + +Describe the role of the participants and the specific skills of each +of them. Show how the participants are suited and committed to the +tasks assigned to them; show the complementarity between +participants. Describe how the opportunity of involving SMEs has been +addressed. Describe the resources, human and material, that will be +deployed for the implementation of the project. Include a STREP +Project Effort Form, as shown below, covering the full duration of the +project. Demonstrate how the project will mobilise the critical mass +of resources (personnel, equipment, finance... ) necessary for +success; and show that the overall financial plan for the project is +adequate. + +(Recommended length ? five pages) + +B.4.1 Sub-contracting: + +If any part of the work is foreseen to be sub-contracted by the +participant responsible for it, describe the work involved and explain +why a sub-contract approach has been chosen for it. + +(Recommended length ? one page) + +B.4.2 Other countries : + +If one or more of the participants is based outside of the EU Member +and Associated states, explain in terms of the project's objectives +why this/these participants have been included, describe the level of +importance of their contribution to the project. + +(Recommended length ? one page). + +B.5 Project management + +Describe the organisation, management and decision making structures +of the project. Describe the plan for the management of knowledge, of +intellectual property and of other innovation-related activities +arising in the project. + +(Recommended length ? three pages) + +B.6 Detailed Implementation plan + +This section describes in detail the work planned to achieve the +objectives for the full duration of the of the proposed project. The +recommended length, excluding the forms specified below, is up to 15 +pages. An introduction should explain the structure of this workplan +plan and how the plan will lead the participants to achieve the +objectives. The workplan should be broken down according to types of +activities: Research, technological development and innovation related +activities, demonstration activities and project management +activities. It should identify significant risks, and contingency +plans for these. The plan must for each type of activity be broken +down into workpackages (WPs) which should follow the logical phases of +the project, and include management of the project and assessment of +progress and results. + +Essential elements of the plan are: + +a) Detailed Implementation plan introduction ? explaining the structure + of this plan and the overall methodology used to achieve the objectives. + +b) Work planning, showing the timing of the different WPs and their + components (Gantt chart or similar) + +c) Graphical presentation of the components showing their + interdependencies (Pert diagram or similar) + +d) Detailed work description broken down into workpackages: + Workpackage list (use Workpackage list form below); + Deliverables list (use Deliverables list form below); + Description of each workpackage (use Workpackage description form + below, one per workpackage): + +Note: The number of workpackages used must be appropriate to the +complexity of the work and the overall value of the proposed +project. Each workpackage should be a major sub-division of the +proposed project and should also have a verifiable end-point (normally +a deliverable or an important milestone in the overall project). The +planning should be sufficiently detailed to justify the proposed +effort and allow progress monitoring by the Commission ? the +day-to-day management of the project by the consortium may require a +more detailed plan. + +(recommended length, excluding the forms specified above,- up to 15 pages) + +B.7 Other issues + +B.7.1. If there are ethical or gender issues associated with the +subject of the proposal, show they have been adequately taken into +account - indicate which national and international regulations are +applicable and explain how they will be respected. Explore potential +ethical aspects of the implementation of project results. Include the +Ethical issues form given below. + +B.7.2 Are there other EC-policy related issues, and are they taken +into account? Demonstrate a readiness to engage with actors beyond the +research to help spread awareness and knowledge and to explore the +wider societal implications of the proposed work; if relevant set out +synergies with education at all levels. + +(No recommended length ? depends on the number of such other + issues which the project involves). From lac at codespeak.net Mon Sep 15 14:21:56 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Mon, 15 Sep 2003 14:21:56 +0200 (MEST) Subject: [pypy-svn] rev 1302 - pypy/trunk/doc/funding Message-ID: <20030915122156.C441D5AAE5@thoth.codespeak.net> Author: lac Date: Mon Sep 15 14:21:56 2003 New Revision: 1302 Modified: pypy/trunk/doc/funding/B2.0.0_background.txt Log: tiny changes Modified: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_background.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Mon Sep 15 14:21:56 2003 @@ -133,21 +133,24 @@ FIXME Footnote to: http://psyco.sourceforge.net/introduction.html -Although Armin Rigo proved with his PSYCO project that higher -level languages can actually be optimized to be as fast or faster than C, -he was very limited by the fact that Python is itself implemented in C -like almost all other languages in industrial use today. There was no -real-life project who tried to go all "optimize high-level down to machine-code" -way. Christian Tismer with his "Stackless" project had already come to a similar -conclusion from a more industrial viewpoint in that it is difficult to advance -language technology while relying on a large C-code base. +Although Armin Rigo proved with his 'Psyco' project that higher level +languages can actually be optimized to be as fast or faster than C, he +was very limited by the fact that Python is itself implemented in C +like almost all other languages in industrial use today. There was no +real-life project who tried to go all 'optimize high-level down to +machine-code' way. Christian Tismer with his 'Stackless' project had +already come to a similar conclusion from a more industrial viewpoint +in that it is difficult to advance language technology while relying +on a large C-code base. + +FIXME Footnote to: Some mailing list discussions ----------------------------- On the German Python mailing list an independent discussion evolved where developers and academics were pondering about the possibility -of developing a "minimal" python implemented in Python itself. Most +of developing a 'minimal' python implemented in Python itself. Most noticably Christian Tismer author of an industrial-use extension ("Stackless") of the Python language noted in a postscriptum of one of his mails that having a minimized version of Python could provide @@ -222,15 +225,17 @@ but announced a week later at the EuroPython conference that "PyPy" has a high priority on his "dreams hopefully coming true"-list and he enjoyed it a lot. -By the end of the third one-week sprint in Louvain-La-Neuve, which was held June 20-24, -the PyPy project had already produced a fully working prototype. There -existed a working interpreter and standard implementation of 90% of the python -types, as well as advanced language features such as nested scopes, generators -and metaclasses. Armin Rigo and Guido van Rossum started working on a type-inference -engine which is the foundation for the final missing step: generating a native -machine-level version of Python from its high-level PyPy-implementation. -Perhaps most importantly, we had the enthusiastic support of the Python -community on the EuroPython conference. +By the end of the third one-week sprint at the University in +Louvain-La-Neuve, which was held June 20-24, the PyPy project had +already produced a fully working prototype. There existed a working +interpreter and standard implementation of 90% of the python types, as +well as advanced language features such as nested scopes, generators +and metaclasses. Armin Rigo and Guido van Rossum started working on a +type-inference engine which is the foundation for the final missing +step: generating a native machine-level version of Python from its +high-level PyPy-implementation. Perhaps most importantly, we had the +enthusiastic support of the Python community on the EuroPython +conference. No work had been done on actually optimising the code so it ran around 30,000 times slower than the existing CPython-implementation but this was to be expected From lac at codespeak.net Mon Sep 15 14:37:09 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Mon, 15 Sep 2003 14:37:09 +0200 (MEST) Subject: [pypy-svn] rev 1303 - pypy/trunk/doc/funding Message-ID: <20030915123709.D30865AAE5@thoth.codespeak.net> Author: lac Date: Mon Sep 15 14:37:09 2003 New Revision: 1303 Added: pypy/trunk/doc/funding/B1.0_objectives.txt Log: Base Document Added: pypy/trunk/doc/funding/B1.0_objectives.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B1.0_objectives.txt Mon Sep 15 14:37:09 2003 @@ -0,0 +1,14 @@ +B.1 Scientific and technological objectives of the project and state of + the art In the event of inconsistency between information given in + Part A and that given in Part B, the Part A version will prevail + +Describe in detail the proposed project's S&T objectives. The +objectives should be those achievable within the project, not through +subsequent development, and should be stated in a measurable and +verifiable form. The progress of the project work will be measured +against these goals in later reviews and assessments. Describe the +state-of-the-art in the area concerned and how the proposed project +will enhance the state-of-the-art in that area. + +(Recommended length ? three pages) + From lac at codespeak.net Mon Sep 15 17:28:26 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Mon, 15 Sep 2003 17:28:26 +0200 (MEST) Subject: [pypy-svn] rev 1304 - pypy/trunk/doc/funding Message-ID: <20030915152826.9C5B25AAE5@thoth.codespeak.net> Author: lac Date: Mon Sep 15 17:28:25 2003 New Revision: 1304 Modified: pypy/trunk/doc/funding/B2.0.0_background.txt Log: Make it a little less obvious to everybody who reads this that somewhere there was a German involved ... :-) Fix PSYCO, and probably do damage to the "s. Add some suggestions, and Spell Jacob's last name correctly. Modified: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_background.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Mon Sep 15 17:28:25 2003 @@ -91,12 +91,12 @@ Independently some researchers who worked with Python were pondering writing an implementation of Python in Python itself. This group included -Armin Rigo, author of PSYCO http://psyco.sourceforge.net/introduction.html, +Armin Rigo, author of Psyco http://psyco.sourceforge.net/introduction.html, a specialising Just-In-Time compiler for Python. He is intimately familiar with both Python internals and advanced research in compilers and runtime systems, and saw a Python implementation in Python itself as a chance to put the two fields together. It is useful to quote from his webpage which -states the goals of his PSYCO in full: +states the goals of his Psyco in full: My goal in programming Psyco is to contribute to reduce the following wide gap between academic computer science and @@ -112,7 +112,7 @@ applications, high-level programming can deliver much higher performances. - The new class of languages called "dynamic scripting languages", + The new class of languages called 'dynamic scripting languages', of which Python is an example, is semantically close to long-studied languages like Lisp. The constraints behind their designs are however different: some high-level languages can be @@ -143,16 +143,16 @@ in that it is difficult to advance language technology while relying on a large C-code base. -FIXME Footnote to: +FIXME Footnote to: http://www.stackless.com/ Some mailing list discussions ----------------------------- On the German Python mailing list an independent discussion evolved where developers and academics were pondering about the possibility -of developing a 'minimal' python implemented in Python itself. Most +of developing a 'minimal' Python implemented in Python itself. Most noticably Christian Tismer author of an industrial-use extension -("Stackless") of the Python language noted in a postscriptum of one +('Stackless') of the Python language noted in a postscriptum of one of his mails that having a minimized version of Python could provide a new base to advance the language. Nevertheless, his extensions to the language proved to be useful for companies who needed a way @@ -162,36 +162,46 @@ Some organizational experience ------------------------------ -Meanwhile Holger Krekel had joined the Python community in 2002. For some -years he had architected platforms and consulted CEO's for some large banking -centers in Europe. While participating two "coding sprints" of the Zope3 -web-platform (the to-be successor of the successful Zope web-platform) -he realized that "sprints" in combination with the rapid development language -Python provide an extremetly productive way of communicating about -and coding complex projects where proprietary methods often fail. - -Seeing the opportunity to setup the "PyPy" project with Armin Rigo -and Christian Tismer he offered to organize the first one-week meeting, -the "Sprint towards a minimal Python". Soon many interested developers -joined and intense academical and practical planning ensused. Just a -few weeks later the Sprint took place at "Trillke-Gut", a castle-like -building in Germany, bringing together some key developers, among them -Michael Hudson, the release manager of version 2.2.1 of Python. -Here is the mail that started this now rapidly evolving project -http://starship.python.net/pipermail/python-de/2003q1/002925.html +Meanwhile Holger Krekel had joined the Python community in 2002. For +some years he had designed the architecture for platforms and +consulted for CEO's of some large banking centers in Europe. While +participating in two 'coding sprints' of the Zope3 web-platform (the +to-be successor of the successful Zope web-platform) he realized that +agile 'sprints' in combination with the rapid development language +Python provide an extremetly productive way of communicating about and +coding complex projects where traditional, slow-moving methods often fail. + +At a Sprint a group of people assemble together to write code and +practice Agile software methodological techniques, such as +Pair-Programming. Not only is this a lot of fun, but it is a way to +transmit knowledge and enthusiasm thoughout the community. + +ASK_STOCKHOLM DO we need to discuss Sprints more? + +Holger Krekel, seeing the opportunity to launch the PyPy project with +Armin Rigo and Christian Tismer offered to organize the first +one-week meeting, the 'Sprint towards a minimal Python'. Soon many +interested developers joined and intense academical and practical +planning ensused. Just a few weeks later the Sprint took place at +'Trillke-Gut', a castle-like building in Germany, bringing together +some key developers, among them Michael Hudson, the release manager of +version 2.2.1 of Python. +Here is the mail that started this now rapidly evolving project +http://starship.python.net/pipermail/python-de/2003q1/002925.html -Some promising open-source developments tools ---------------------------------------------- +Some promising open-source development tools +-------------------------------------------- -Moreover, from the start the PyPy developers were committed to bringing -together and utilizing most promising open-source technologies. Here -Jens-Uwe Mager, a retired CTO from Helios (http://www.helios.de) came in and -helped setting up a state-of-the-art open-source development environment. -With his 12-year experience of setting up and leading a SME-company which -is one of the worldwide leaders in Print-Preprocessing technology -he helped organizing the development and net-connectivity for the -various web services needed by the PyPy developers. +From the beginning, the PyPy developers were committed using and +integrating the most promising open-source technologies. Jens-Uwe +Mager, the retired CTO from Helios (http://www.helios.de) attended the +Sprint and helped set up a state-of-the-art open-source development +environment. With his 12-year experience of setting up and leading a +SME-company which is one of the worldwide leaders in +Print-Preprocessing technology he helped organise the development and +net-connectivity for the various web services needed by the PyPy +developers. Research, pragmatism and industry experience combined ----------------------------------------------------- @@ -201,63 +211,70 @@ looking for. Academics, developers and practioners conversant in the latest in language and platform architecture who were interested in producing the Python interpreter which they needed. Laura Creighton and -Jacob Hallen from the PBF attended the sprint and began to communicate -and participate with the project. +Jacob Hall?n from the PBF attended the sprint and began participating +in the project. -The group decided soon to test their proof of concept by developing -a working prototype and to test their ability to work together. It would -get together in their own private time, and work on the prototype in intense -pair-progamming work sessions. After the first Sprint at TrillkeGut in -Hildesheim, Jacob Hallen and Laura Creighton organized the next Sprint -at AB Strakt in Gothenburg, Sweden which helped to keep up the momentum -that the PyPy development group had gained. Also Samuele Pedroni, the -lead developer of Jython (a tight industrial-use integration of Java and -Python), joined the project out of research and practical interest too see -that PyPy would help with his own java-integration project. With all -these experienced people, by the end of the week you could already run -simple Python programs within PyPy. - -The third sprint followed close just before the "EuroPython" conference -in Belgium. the sprint was organized by interested developers in Belgium in -Louvain-La-Neuve, a university city. In another move, Laura Creighton took -the opportunity during her visit in Washington D.C. (XXX) to invite Guido van Rossum, -the inventor and principal of Python. He not only attended the belgium sprint -but announced a week later at the EuroPython conference that "PyPy" has a high -priority on his "dreams hopefully coming true"-list and he enjoyed it a lot. +The group decided soon to test their proof of concept by developing a +working prototype and to test their ability to work together. They +would meet for Sprints in their own private time, and work on the +prototype. + +After the first Sprint at TrillkeGut in Hildesheim, Jacob Hall?n and +Laura Creighton organized the next Sprint at AB Strakt in Gothenburg, +Sweden. At this Sprint, Samuele Pedroni, the lead developer of Jython +(a tight industrial-use integration of Java and Python), joined the +project not only because such a project was relavant to his own +computer science research interests, but also to see if PyPy would +help with his own java-integration project. Prototyping went quickly, +with the work of all these experienced people, and by the end of the +week you could already run simple Python programs within PyPy. PyPy +had gone from being 'a nice idea' to 'something we knew we could do'. + + + + + +The third sprint was organized by interested developers in Belgium at +the University in Louvain-La-Neuve and held June 20-24. We invited +Guido van Rossum, the inventor of Python to attend. He not only +attended the Belgium sprint but announced a few days later at the +EuroPython conference that PyPy had a high priority on his list of +'dreams he hoped would come true' and he enjoyed Sprinting with us a +lot. By the end of the third one-week sprint at the University in -Louvain-La-Neuve, which was held June 20-24, the PyPy project had -already produced a fully working prototype. There existed a working -interpreter and standard implementation of 90% of the python types, as -well as advanced language features such as nested scopes, generators -and metaclasses. Armin Rigo and Guido van Rossum started working on a -type-inference engine which is the foundation for the final missing -step: generating a native machine-level version of Python from its -high-level PyPy-implementation. Perhaps most importantly, we had the -enthusiastic support of the Python community on the EuroPython +Louvain-La-Neuve, the PyPy project had already produced a fully +working prototype. There existed a working interpreter and standard +implementation of 90% of the python types, as well as advanced +language features such as nested scopes, generators and metaclasses. +Armin Rigo and Guido van Rossum had started work on a type-inference +engine which is the foundation for the final missing step: generating +a native machine-level version of Python from its high-level +PyPy-implementation. Perhaps most importantly, we had the +enthusiastic support of the Python community at the EuroPython conference. -No work had been done on actually optimising the code so it ran around 30,000 -times slower than the existing CPython-implementation but this was to be expected -from the start. Nevertheless, for a proof of concept, approximately four weeks work -total for a group of about a dozen people, it was clearly a success. - -Breaking the last barrier -------------------------- - -However, almost all of the participants realized after three one-week sprints -in just six month that their other obligations (full-time jobs in mostly unrelated -areas!) would not allow them to continue at this pace. But everyone involved understood -that this new Python implementation opened up extremely promising possibilities. +No work had been done on actually optimising the code so it ran around +30,000 times slower than the existing CPython-implementation but this +was to be expected from the start. Nevertheless, for a proof of +concept, approximately four weeks work total for a group of about a +dozen people, it was clearly a success. It was time to look for funding. +< While some people suggested to apply to DARPA for funding we realized that most of us were rooted in Europe and it would make more sense to look for european possibilities. -In June the maybe most important development took place. The 2nd Call of the -Information Society Technologies [IST] Priority went out. Included in it was -IST-2002-2.3.2.3 - Open development platforms for software and services. +I don't think that this is wise. Why remind them that we could get +somebody else to fund it, maybe? Besides, everybody on the list +of people they are getting are in Europe, not just most ... +> + +On June 17th, the 2nd Call of the Information Society Technologies +[IST] Priority went out. Included in it was IST-2002-2.3.2.3 - Open +development platforms for software and services. -We believe that what we intend to do is a perfect fit for the goals of this call. +We believe that what we intend to do is a perfect fit for the +goals of this call. From tismer at codespeak.net Mon Sep 15 17:47:44 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 15 Sep 2003 17:47:44 +0200 (MEST) Subject: [pypy-svn] rev 1306 - pypy/trunk/doc/funding Message-ID: <20030915154744.B0B7A5AAE5@thoth.codespeak.net> Author: tismer Date: Mon Sep 15 17:47:44 2003 New Revision: 1306 Added: pypy/trunk/doc/funding/B6.1_plan_introduction.txt Log: Added an atte,pt of an introduction. This needs to be chngd quite a lot. At the moment, it contains all the working packages. Armin, I need your input, please see 'Low-level targets, tools and releases' Added: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 15 17:47:44 2003 @@ -0,0 +1,358 @@ +B 6.1 Workplan Introduction +=============================== + +The PyPy project has a number of objectives, which +can be categorized into two major groups: + +- development of PyPy itself, as a Python written in Python. + This includes code generation for various targets, re-implementation + of all builtin Python objects and some extension modules, + and the development of several object spaces, testing and documentation. + +- application level code which shows the results of PyPy as a usability + study. Generating down-sized code for embedded systems, load balancing + in a distributed network, code generators optimized for number crunching + on some processor architectures and re-writing numerical Python + packages should be able to prove that PyPy is more than an + academical exercise but very apropriate for industrial strength + applications. + + +*XXX: AT the moment, I'm listing the working packages here. +I think this is no good. The WPs are now mved out of here. +The text here gets shortened, the text outside gets +extended. + + +Working Packages +=================== + +The PyPy Interpreter +--------------------------- + +The goal is to make a complete Python interpreter that runs under any +existing Python implementation. + +[[WP1]]: Development and Completion of the PyPy Interpreter +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The PyPy interpreter itself should be developed and completed +as a regular Python program. This package includes all +parts of CPython tht we don't want to move to [[WP2]]. +Further investigate the unorthodox multimethod +concepts that the standard object space is based on, and how to hook in +the bytecode compiler. + +[[WP2]]: Translation of CPython into regular Python +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Translate all other parts of CPython which we don't want to +implement in [[WP1]] into regular Python libraries. +These ones should also work without PyPy, being just plain-Python +replacements for existing CPython functionality. +This includes the bytecode compiler, which definately should +become a regular Python program, instead of being built +into the interpreter. +. + +The PyPy Compiler +--------------------------- + + +[[WP3]]: Translation of RPython +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +RPython is a restricted version of Python, which is much +more statically defined than standard Python. It allows +to write code that is rather easily translated into something +else, like C source, Pyrex code, or even passed to +an assembly code generator. +The Definition of RPython is slightly in flux and might +be adjusted during [[WP3]]. + +The goal is to be able to translate arbitrary RPython source code (e.g. +the one produced in [[WP1]] into low-level code (C, Pyrex, Java, others). +This includes making a stand-alone, not-PyPy-related tool for general +optimization of arbitrary but suitably restricted Python application or +parts thereof. + +**Part a): Code Analysis** + +Analyse code to produce the relevant typing information. Investigate +if we can use the annotation object space only or if additional +AST-based control flow analysis is needed. Give a formal definition +of RPython. + +**Part b): Code Production** + +Produce low-level code out of the data gathered in Part a). Again +investigate how this is best done (AST-guided translation or +reverse-engeneering of the low-level control flow gathered by the +annotation object space). Compare different low-level environments that +we could target (C, Pyrex, others?). + + +[[WP4]]: _`Bootstrapping` PyPy +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal is to put interpreter ([[WP1]], [[WP2]]) and translator +([[WP3]]) together. + +The interpreter is written as an RPython program. The translator +has to translate this program into some low-level language. +The resulting program then needs to be supported by a special +runtime library. The work-flow of [[WP9]] is repetitive, +since it will not be possible to "get it right" in the +first attempt. Analysis and redesign will have to be +repeated until the result is satisfactory. + +**Part a): Specific Analysis and Redesign** + +The global translation of PyPy is going to raise particular +problems, other than more general RPython programs do. +Since translation of RPython is the core idea of the +bootstrap process and the main target of the translator, +we need to investigate and isolate the +particular problems, and redesign specific parts of PyPy +to support translation, code generation and optimzation +better. This will also include a re-iteration of the interface +design between application level and interpreter level +until we reach overall convergence. + +**Part b): Low Level PyPy Runtime** + +In order to give a working environment to the translated RPython +program, we need to build the low-level-specific **run-time** +components of PyPy. +Most notably are the object layout, the memory management, +and possibly threading support, as well as an +efficient multimethod dispatch. +The target language is not yet decided on. We might go different +paths at the same time. +If producing C code is a target, important +parts can be directly re-used from CPython. + + +_`High-performance` PyPy-Python +----------------------------------- +The goal is to optimize `Bootstrapping`_ in possibly various ways, +building on its flexibility to go beyond CPython. +The main lack of flexibility in CPython stems from the +fact that all structures are hard-coded in C, and there is +no abstraction layer. PyPy does provide this abstraction layer, +since it's RPython implementation is not meant to be +executed directly, but goes though a code generator which +produces the actual machine code. This layer is highly configurable. + +The associated working packages +should be done in parallel,since the results are rather dependant +from each other. + +[[WP5]]: Several Object Implementations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +develop **several object implementations** for the same types, as +explicitly allowed by the standard object space, and develop heuristics +to switch between implementations during execution. (see `Annex SOI`_) +Study the efficiency of different approaches. + +[[WP6]]: Translator Optimisations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +It should be identified, which **optimizations** would benefit +from support from the translator. These are the optimizations +not easily available to CPython because they would require +large-scale code rewrites. (see `Annex OPT`) +This includes design-cosiderations including the decision whether +to use reference counting together with garbage collection, +or to go for garbage collection, only. + +*XXX use a different WP for the next?* + +For each issue, work on **several solutions** when no one is obviously +better than the other ones. The meta-programming underlying [[WP10]] +-- namely the work on the translator instead of on the resulting code -- is +what gives us the possibility of actually implementing several very +different schemes. The outcome of this effort is of course not unique, +but depends on the specific target of the optimization. +There will be at least two efforts at the same time: + +- optimization towards maximum performance fo an integrated PyPy system + +- optimization towards minimal code size for embedded systems. + +*add a lot more about this* + +[[WP7]]: Integration of Existing Technology +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*XXX should this be split into two WPs?* + +There are existing projects, notably Psyco and Stackless, +which have been traditionally depended on closely +following CPython's code base. + +Psyco should be re-implemented in pure Python, and maybe +partially in RPython. The intent is to be able to create +specialized code at run-time. All prior knowledge of the Psyco +project will be integrated into PyPy. + +Stackless Python has implemented high speed multitasking in +a single thread for C Python in two different ways: +Continuation passing and stack switching. Both ways of +task switching can be integrated into PyPy. Furthermore, +pickling of running programs has been implemented in +Stackless Python and should enable PyPy for load-balancing +between different machines. + +Rewrite each one as a meta-component that hooks into the +translator plus a dedicated run-time component. Further +develop these technologies based on the results gathered in [[WP11]], +e.g. identify when these technologies would guide specific +choices among the solutions developed in [[WP10]] and studying +several solutions in [[WP11]]. + + +*XXX The following paragraph is not clear to me, +how to group it into packages. Armin?* + +Low-level targets, tools and releases +-------------------------------------------- + +The goal is to identify, among those low-level targets that are in +widespread use (e.g. workstation usage vs. web server vs. +high-performance computing vs. memory-starved hand-held device; C/Unix +vs. Java vs. .NET environment), which ones would benefit most from a +high-performance Python interpreter. For each of these, focus will be +given to: + +a) develop the translation process, run-time and those optimizations + that depend on low-level details. + +b) design interfaces for extension modules. Some can be very general + (e.g. a pure Python one that should allow generic third-party code to + hook into the PyPy interpreter source code without worrying about the + translation process). Others depend on the low-level environment and on + the choices made for the issues of `High-performance`. + +c) (_`APP`)combine different solutions for the different issues discussed in + `High-performance`_. Gather statistics with real-work Python applications. Compare the + results. This is where the flexibility of the whole project is vital. + Typically, very different trade-offs need to be made on different + environments. + +d) most importantly, develop tools to easily allow third-parties to + repeat (APP_) in their own domain and build their own tailored versions of + PyPy. + +e) release a few official versions pre-tailored for various common + environments. Develop in particular a version whose goal is to simulate + the existing CPython interpreter to support legacy extension modules. + Investigate if the PyPy core can make internal choices that are very + different from CPython's without sacrifying legacy extension modules + compatibility. + + +Application Level Targets +----------------------------- + +[[WPxxx]] Load Balancing in a Multi-Processor environment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*XXX to be finished, collecting snippets* + +[[WPxxx]] Specialized Code for Embedded Devices +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*XXX to be finished, collecting snippets* + +[[WPxxx]] Numerical Applications +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +PyPy is extremely flexible, since it supports static +and dynamic code generation by pluggable processor +architecture modules. PyPy can use special hardware +by itself, or it can cross-compile code for target +machines which will run only a downsized PyPy +instantiation which is runtime only. + +One advantage of using PyPy instead of other compiled +libraries is its ability to not only support the +special hardware, but also to adjust to the given +machine properties, like cache sizes, number of +parallel FPUs, memory access speed and memory size. +PyPy is able to probe these parameters and to choose +an optimum implementation for the particular hardware +configuration during startup time. + +Write specialized code generators which +support SIMD instruction sets, like SSE2 and AltiVec. + +Rewrite a numerical Python package like NumPy +using RPython. Identify the numerical operations +in the package which are candidates for parallelization. +Write an interface that allows to implement these operations +in either a traditional way or using SIMD instructions, +dependant from the available capabilities of the current +code generator. +The ability of special instructions and other extensions +should be probed using run-time checks. Psyho should be +extended to be aware of paralle instructions, and be +enabled to emit optimized code for that. + + +[[WPxxx]] An Operating System in PyPy +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*XXX this is what I'm working on at the moment -- the text is +not yet worked out enough. Another hour...* + + +.. _Annex SOI: + +Annex SOI and others: *where do we put this?* + + + +Infrastructure +------------------ + +*XXX: Numbering these beginning with 100 for the moment +until we know the exact number of packages* + +[[WP101]]: Coordination and Management +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +WP1 continues throughout the duration of the project and is carried out +by the project coordinator. It involves the collection and monitoring of +monthly status reports, reporting to the EU, organising meetings +and maintaining an internal web site. The webiste will make heavy +use of colaborative tools like Wiki pages, a content management system, +and issue trackers. + +[[WP102]]: Project Documentation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +During the whole project, a set of documents will be maintained +that documents the current status of the project, results of +discussions, the planning of new sprints and their status reports, +as well as the preparation of papers for presentation on +various congresses. + +[[WP103]]: Maintenance of Tools +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +PyPy's own development needs an infrastructure that must continuously +be kept up-to-date and further developed. This includes maintenance +of the SubVersion package and tracking of its ongoing development, +maintenance of the SubVersion respository, extensions like automatic +documentation extraction, and change notification via mailing lists. + + +[[WP104]]: Quality Control +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The PyPy project has included an exhaustive set of tests for +all software modules. This has to be extended and maintained +throughout the whole project. The intent is to be able to +run all of our internal tests frequently, and to be able to +run the standard Python test suite, as far as applicable. + +[[WP105]]: Synchronisation with Standard Python +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Since Python is being further developed all the time, there is +a continuous need of keeping PyPy in sync with the future developments +of CPython, e.g. ways to relate pieces of PyPy source and pieces of CPython +source. This work is carried out by hand at the moment. We +do look for existing solutions which allow to automate this +effort at least partially. From tismer at codespeak.net Mon Sep 15 18:01:01 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 15 Sep 2003 18:01:01 +0200 (MEST) Subject: [pypy-svn] rev 1307 - pypy/trunk/doc/funding Message-ID: <20030915160101.7E9655AAE5@thoth.codespeak.net> Author: tismer Date: Mon Sep 15 18:01:00 2003 New Revision: 1307 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt Log: smallish changes Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 15 18:01:00 2003 @@ -51,7 +51,7 @@ This includes the bytecode compiler, which definately should become a regular Python program, instead of being built into the interpreter. -. + The PyPy Compiler --------------------------- @@ -179,7 +179,7 @@ *add a lot more about this* [[WP7]]: Integration of Existing Technology -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *XXX should this be split into two WPs?* @@ -231,10 +231,10 @@ the choices made for the issues of `High-performance`. c) (_`APP`)combine different solutions for the different issues discussed in - `High-performance`_. Gather statistics with real-work Python applications. Compare the - results. This is where the flexibility of the whole project is vital. - Typically, very different trade-offs need to be made on different - environments. + `High-performance`_. Gather statistics with real-work Python applications. + Compare the results. This is where the flexibility of the whole + project is vital. Typically, very different trade-offs need to be + made on different environments. d) most importantly, develop tools to easily allow third-parties to repeat (APP_) in their own domain and build their own tailored versions of @@ -312,7 +312,7 @@ ------------------ *XXX: Numbering these beginning with 100 for the moment -until we know the exact number of packages* +until we know the exact number of packages that come before* [[WP101]]: Coordination and Management ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From tismer at codespeak.net Mon Sep 15 18:26:51 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 15 Sep 2003 18:26:51 +0200 (MEST) Subject: [pypy-svn] rev 1308 - pypy/trunk/doc/funding Message-ID: <20030915162651.1A2D85AAE5@thoth.codespeak.net> Author: tismer Date: Mon Sep 15 18:26:50 2003 New Revision: 1308 Modified: pypy/trunk/doc/funding/B0.0_preamble.txt pypy/trunk/doc/funding/B6.0_detailed_implementation.txt pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/strep_boilerplate.txt Log: small corrections to make several files work with docutils Modified: pypy/trunk/doc/funding/B0.0_preamble.txt ============================================================================== --- pypy/trunk/doc/funding/B0.0_preamble.txt (original) +++ pypy/trunk/doc/funding/B0.0_preamble.txt Mon Sep 15 18:26:50 2003 @@ -48,22 +48,22 @@ 2 3 4 - etc +etc Coordinator name Coordinator organisation name Coordinator email Coordinator fax -Contents page - show contents list +Contents page *show contents list* Proposal summary page Proposal full title Proposal acronym Strategic objectives addressed + (If more than one objective , indicate their order of importance to the project). -Proposal abstract copied from Part A (if not in English, include an - English translation) +Proposal abstract copied from Part A +(if not in English, include an English translation) Modified: pypy/trunk/doc/funding/B6.0_detailed_implementation.txt ============================================================================== --- pypy/trunk/doc/funding/B6.0_detailed_implementation.txt (original) +++ pypy/trunk/doc/funding/B6.0_detailed_implementation.txt Mon Sep 15 18:26:50 2003 @@ -29,7 +29,7 @@ Workpackage list (use Workpackage list form below); Deliverables list (use Deliverables list form below); Description of each workpackage (use Workpackage description form - below, one per workpackage): + below, one per workpackage): Note: The number of workpackages used must be appropriate to the complexity of the work and the overall value of the proposed Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 15 18:26:50 2003 @@ -19,9 +19,9 @@ *XXX: AT the moment, I'm listing the working packages here. -I think this is no good. The WPs are now mved out of here. +I think this is no good. The WPs are now moved out of here. The text here gets shortened, the text outside gets -extended. +extended.* Working Packages Modified: pypy/trunk/doc/funding/strep_boilerplate.txt ============================================================================== --- pypy/trunk/doc/funding/strep_boilerplate.txt (original) +++ pypy/trunk/doc/funding/strep_boilerplate.txt Mon Sep 15 18:26:50 2003 @@ -1,4 +1,5 @@ Annex 2 - Proposal Part B + IST Priority - ? 2nd call - Guide for Proposers ? Specific Targeted Research Projects ? June 2003 @@ -48,23 +49,25 @@ 2 3 4 - etc +etc + Coordinator name Coordinator organisation name Coordinator email Coordinator fax -Contents page - show contents list +Contents page *show contents list* Proposal summary page Proposal full title Proposal acronym Strategic objectives addressed + (If more than one objective , indicate their order of importance to the project). + Proposal abstract copied from Part A (if not in English, include an - English translation) +English translation) B.1 Scientific and technological objectives of the project and state of the art In the event of inconsistency between information given in @@ -180,7 +183,7 @@ Workpackage list (use Workpackage list form below); Deliverables list (use Deliverables list form below); Description of each workpackage (use Workpackage description form - below, one per workpackage): + below, one per workpackage): Note: The number of workpackages used must be appropriate to the complexity of the work and the overall value of the proposed From tismer at codespeak.net Mon Sep 15 20:00:04 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 15 Sep 2003 20:00:04 +0200 (MEST) Subject: [pypy-svn] rev 1309 - pypy/trunk/doc/funding Message-ID: <20030915180004.B687E5B9CA@thoth.codespeak.net> Author: tismer Date: Mon Sep 15 20:00:03 2003 New Revision: 1309 Added: pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt Log: Trying to make stand-alone working package files. Please review and tell me whether it makes sense. Added: pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt Mon Sep 15 20:00:03 2003 @@ -0,0 +1,43 @@ +Workpackage Description +======================== + +================================ ========================== +**Workpackage number** 01 +-------------------------------- -------------------------- +**Start date or starting event** 0 +================================ ========================== + +================================= ======= ======= ======= ======= ======= ======= ======= ======= +**Participant id** X Y Z +--------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- +**Person-months per participant** 3 3 3 +================================= ======= ======= ======= ======= ======= ======= ======= ======= + + +**Objectives** + +Title: Development and Completion of the PyPy Interpreter + +The PyPy interpreter itself should be developed and completed +as a regular Python program. This package includes all +parts of CPython that we don't want to move to WP02_. + +**Description of work** + +- Coordinate with WP02_ which parts of CPython should be implemented in RPython +- Implementation of all C modules and C objects +- Further investigation of the unorthodox multimethod + concepts that the standard object space is based on, and how to hook in + the bytecode compiler. + + +**Deliverables** + +- DL01: First complete Python Implementation + + +**Milestones and Expected Result** + +- Initial Prototype + +.. _WP02: B6.wp02_cpython_to_python.html Added: pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt Mon Sep 15 20:00:03 2003 @@ -0,0 +1,45 @@ +Workpackage Description +======================== + +================================ ========================== +**Workpackage number** 02 +-------------------------------- -------------------------- +**Start date or starting event** 0 +================================ ========================== + +================================= ======= ======= ======= ======= ======= ======= ======= ======= +**Participant id** X Y Z +--------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- +**Person-months per participant** 3 3 3 +================================= ======= ======= ======= ======= ======= ======= ======= ======= + + +**Objectives** + +Title: Translation of CPython into regular Python + +Translate all parts of CPython which we don't want to +implement in WP01_ into regular Python libraries. +Note: These ones should also work without PyPy, being just plain-Python +replacements for existing CPython functionality. + + +**Description of work** + +- Coordinate with WP01_ which parts of CPython should be implemented in Python +- Implementation of all C modules and C objects +- Implement the Python bytecode compiler, which definately should + become a regular Python program, instead of being built + into the interpreter. + + +**Deliverables** + +- DL01: First complete Python Implementation + + +**Milestones and Expected Result** + +- Initial Prototype + +.. _WP01: B6.wp01_interpreter_completion.html From lac at codespeak.net Mon Sep 15 20:15:52 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Mon, 15 Sep 2003 20:15:52 +0200 (MEST) Subject: [pypy-svn] rev 1310 - pypy/trunk/doc/funding Message-ID: <20030915181552.66DB95B9CA@thoth.codespeak.net> Author: lac Date: Mon Sep 15 20:15:51 2003 New Revision: 1310 Modified: pypy/trunk/doc/funding/B2.0.0_background.txt Log: fix typo Modified: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_background.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Mon Sep 15 20:15:51 2003 @@ -35,7 +35,7 @@ Basic. Any company which writes its software in Java or Visual Basic is at the mercy of these large American companies. And this is a real, and not theoretical threat. In 2002, Microsoft announced that it would no -longer be supporting Visual Basic 6.0. after the year 2005. All Visual +longer be supporting Visual Basic 6.0 after the year 2005. All Visual Basic Developers have been told to convert their code to run under Microsoft's new .NET framework. In 2001 Microsoft immediately stopped supporting its Visual J++ language, meant to be a direct competitor with From tismer at codespeak.net Mon Sep 15 20:33:41 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 15 Sep 2003 20:33:41 +0200 (MEST) Subject: [pypy-svn] rev 1311 - pypy/trunk/doc/funding Message-ID: <20030915183341.D75FF5B9CA@thoth.codespeak.net> Author: tismer Date: Mon Sep 15 20:33:41 2003 New Revision: 1311 Added: pypy/trunk/doc/funding/crossreferences.txt Log: Moved all hyperlinks into crossreferences.txt, which gets included into all working package files. First usage of the ".. include::" directive. Added: pypy/trunk/doc/funding/crossreferences.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/crossreferences.txt Mon Sep 15 20:33:41 2003 @@ -0,0 +1,4 @@ +.. _WP01: B6.wp01_interpreter_completion.html +.. _WP02: B6.wp02_cpython_to_python.html +.. _WP03: B6.wp03_translation_of_rpython.html +.. _WP04: B6.wp04_bootstrapping_pypy.html From tismer at codespeak.net Mon Sep 15 20:34:34 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 15 Sep 2003 20:34:34 +0200 (MEST) Subject: [pypy-svn] rev 1312 - pypy/trunk/doc/funding Message-ID: <20030915183434.D484A5B9CA@thoth.codespeak.net> Author: tismer Date: Mon Sep 15 20:34:34 2003 New Revision: 1312 Added: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Log: Added new working package. Added: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Mon Sep 15 20:34:34 2003 @@ -0,0 +1,58 @@ +.. include:: crossreferences.txt + +Workpackage Description +======================== + +================================ ========================== +**Workpackage number** 03 +-------------------------------- -------------------------- +**Start date or starting event** 0 +================================ ========================== + +================================= ======= ======= ======= ======= ======= ======= ======= ======= +**Participant id** X Y +--------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- +**Person-months per participant** 3 3 +================================= ======= ======= ======= ======= ======= ======= ======= ======= + + +**Objectives** + +Title: Translation of RPython + +- Ability to translate arbitrary RPython source code (e.g. + the one produced in WP01_ into low-level code (C, Pyrex, Java, others). +- Creation of a stand-alone, not-PyPy-related tool for general + optimization of arbitrary but suitably restricted Python application or + parts thereof. + + +**Description of work** + +**Part a): Code Analysis** + +- Analyse code to produce the relevant typing information. +- Investigate whether we can use the annotation object space + only or if additional AST-based control flow analysis is needed. +- Give a formal definition of RPython. + +**Part b): Code Production** + +- Produce low-level code out of the data gathered in Part a). +- Again investigate how this is best done (AST-guided translation or + reverse-engeneering of the low-level control flow gathered by the + annotation object space). +- Compare different low-level environments that + we could target (C, Pyrex, others?). +- Provide some measure for the efficiency of the generated + code, in comparison to Standard Python. + +**Deliverables** + +- DL02: Stand-alone Python Interpreter +- DL03: Tool for Analysis of restricted Python Programs + +**Milestones and Expected Result** + +- Initial Complete Self-COntained PyPy Interpreter +- Initial Measures of Efficiency From tismer at codespeak.net Mon Sep 15 20:35:29 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 15 Sep 2003 20:35:29 +0200 (MEST) Subject: [pypy-svn] rev 1313 - pypy/trunk/doc/funding Message-ID: <20030915183529.CF2F15B9CA@thoth.codespeak.net> Author: tismer Date: Mon Sep 15 20:35:29 2003 New Revision: 1313 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt Log: Corrections and small changes. All references use crossreferences.txt now. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 15 20:35:29 2003 @@ -1,3 +1,5 @@ +.. include:: crossreferences.txt + B 6.1 Workplan Introduction =============================== @@ -33,19 +35,20 @@ The goal is to make a complete Python interpreter that runs under any existing Python implementation. -[[WP1]]: Development and Completion of the PyPy Interpreter +WP01_: Development and Completion of the PyPy Interpreter ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The PyPy interpreter itself should be developed and completed as a regular Python program. This package includes all -parts of CPython tht we don't want to move to [[WP2]]. +parts of CPython tht we don't want to move to WP02_. Further investigate the unorthodox multimethod concepts that the standard object space is based on, and how to hook in the bytecode compiler. -[[WP2]]: Translation of CPython into regular Python + +WP02_: Translation of CPython into regular Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Translate all other parts of CPython which we don't want to -implement in [[WP1]] into regular Python libraries. +implement in WP01 into regular Python libraries. These ones should also work without PyPy, being just plain-Python replacements for existing CPython functionality. This includes the bytecode compiler, which definately should @@ -53,11 +56,12 @@ into the interpreter. + The PyPy Compiler --------------------------- -[[WP3]]: Translation of RPython +WP03_: Translation of RPython ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ RPython is a restricted version of Python, which is much @@ -65,11 +69,11 @@ to write code that is rather easily translated into something else, like C source, Pyrex code, or even passed to an assembly code generator. -The Definition of RPython is slightly in flux and might -be adjusted during [[WP3]]. +The Definition of RPython is slightly in flux and should +be adjusted during WP03_. The goal is to be able to translate arbitrary RPython source code (e.g. -the one produced in [[WP1]] into low-level code (C, Pyrex, Java, others). +the one produced in WP01_ into low-level code (C, Pyrex, Java, others). This includes making a stand-alone, not-PyPy-related tool for general optimization of arbitrary but suitably restricted Python application or parts thereof. @@ -90,15 +94,15 @@ we could target (C, Pyrex, others?). -[[WP4]]: _`Bootstrapping` PyPy +WP04_: _`Bootstrapping` PyPy ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The goal is to put interpreter ([[WP1]], [[WP2]]) and translator -([[WP3]]) together. +The goal is to put interpreter (WP01_, WP02_) and translator +(WP03_) together. The interpreter is written as an RPython program. The translator has to translate this program into some low-level language. The resulting program then needs to be supported by a special -runtime library. The work-flow of [[WP9]] is repetitive, +runtime library. The work-flow of WP04_ is repetitive, since it will not be possible to "get it right" in the first attempt. Analysis and redesign will have to be repeated until the result is satisfactory. Modified: pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt (original) +++ pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt Mon Sep 15 20:35:29 2003 @@ -1,3 +1,5 @@ +.. include:: crossreferences.txt + Workpackage Description ======================== @@ -39,5 +41,3 @@ **Milestones and Expected Result** - Initial Prototype - -.. _WP02: B6.wp02_cpython_to_python.html Modified: pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt (original) +++ pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt Mon Sep 15 20:35:29 2003 @@ -1,3 +1,5 @@ +.. include:: crossreferences.txt + Workpackage Description ======================== @@ -42,4 +44,3 @@ - Initial Prototype -.. _WP01: B6.wp01_interpreter_completion.html From tismer at codespeak.net Mon Sep 15 21:16:12 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 15 Sep 2003 21:16:12 +0200 (MEST) Subject: [pypy-svn] rev 1314 - pypy/trunk/doc/funding Message-ID: <20030915191612.D73F45B9CA@thoth.codespeak.net> Author: tismer Date: Mon Sep 15 21:16:12 2003 New Revision: 1314 Added: pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt pypy/trunk/doc/funding/crossreferences.txt Log: intermediate result, still editing quite much, but need to go home :-) Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 15 21:16:12 2003 @@ -149,20 +149,20 @@ should be done in parallel,since the results are rather dependant from each other. -[[WP5]]: Several Object Implementations +WP05_: Several Object Implementations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ develop **several object implementations** for the same types, as explicitly allowed by the standard object space, and develop heuristics to switch between implementations during execution. (see `Annex SOI`_) Study the efficiency of different approaches. -[[WP6]]: Translator Optimisations +WP06_: Translator Optimisations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It should be identified, which **optimizations** would benefit from support from the translator. These are the optimizations not easily available to CPython because they would require large-scale code rewrites. (see `Annex OPT`) -This includes design-cosiderations including the decision whether +This includes design-considerations including the decision whether to use reference counting together with garbage collection, or to go for garbage collection, only. @@ -182,7 +182,7 @@ *add a lot more about this* -[[WP7]]: Integration of Existing Technology +WP07_: Integration of Existing Technology ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *XXX should this be split into two WPs?* @@ -206,10 +206,10 @@ Rewrite each one as a meta-component that hooks into the translator plus a dedicated run-time component. Further -develop these technologies based on the results gathered in [[WP11]], +develop these technologies based on the results gathered in WP04_, e.g. identify when these technologies would guide specific -choices among the solutions developed in [[WP10]] and studying -several solutions in [[WP11]]. +choices among the solutions developed in WP05_ and studying +several solutions in WP06_. *XXX The following paragraph is not clear to me, Modified: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt (original) +++ pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Mon Sep 15 21:16:12 2003 @@ -54,5 +54,5 @@ **Milestones and Expected Result** -- Initial Complete Self-COntained PyPy Interpreter +- Initial Complete Self-Contained PyPy Interpreter - Initial Measures of Efficiency Added: pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt Mon Sep 15 21:16:12 2003 @@ -0,0 +1,77 @@ +.. include:: crossreferences.txt + +Workpackage Description +======================== + +================================ ========================== +**Workpackage number** 04 +-------------------------------- -------------------------- +**Start date or starting event** 0 +================================ ========================== + +================================= ======= ======= ======= ======= ======= ======= ======= ======= +**Participant id** X Y +--------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- +**Person-months per participant** 3 3 +================================= ======= ======= ======= ======= ======= ======= ======= ======= + + +**Objectives** + +Title: Bootstrapping PyPy + +- Putting interpreter (WP01_, WP02_) and translator (WP03_) together. + +- Ability to translate arbitrary RPython source code (e.g. + the one produced in WP01_ into low-level code (C, Pyrex, Java, others). +- Creation of a stand-alone, not-PyPy-related tool for general + optimization of arbitrary but suitably restricted Python application or + parts thereof. + + +**Description of work** + + + +**Deliverables** + + + +**Milestones and Expected Result** + +*XXX: This check-in is just temporary* + +The interpreter is written as an RPython program. The translator +has to translate this program into some low-level language. +The resulting program then needs to be supported by a special +runtime library. The work-flow of WP04_ is repetitive, +since it will not be possible to "get it right" in the +first attempt. Analysis and redesign will have to be +repeated until the result is satisfactory. + +**Part a): Specific Analysis and Redesign** + +The global translation of PyPy is going to raise particular +problems, other than more general RPython programs do. +Since translation of RPython is the core idea of the +bootstrap process and the main target of the translator, +we need to investigate and isolate the +particular problems, and redesign specific parts of PyPy +to support translation, code generation and optimzation +better. This will also include a re-iteration of the interface +design between application level and interpreter level +until we reach overall convergence. + +**Part b): Low Level PyPy Runtime** + +In order to give a working environment to the translated RPython +program, we need to build the low-level-specific **run-time** +components of PyPy. +Most notably are the object layout, the memory management, +and possibly threading support, as well as an +efficient multimethod dispatch. +The target language is not yet decided on. We might go different +paths at the same time. +If producing C code is a target, important +parts can be directly re-used from CPython. + Modified: pypy/trunk/doc/funding/crossreferences.txt ============================================================================== --- pypy/trunk/doc/funding/crossreferences.txt (original) +++ pypy/trunk/doc/funding/crossreferences.txt Mon Sep 15 21:16:12 2003 @@ -2,3 +2,6 @@ .. _WP02: B6.wp02_cpython_to_python.html .. _WP03: B6.wp03_translation_of_rpython.html .. _WP04: B6.wp04_bootstrapping_pypy.html +.. _WP05: B6.wp05_several_object_implementations.html +.. _WP06: B6.wp06_translator_optimisations.html +.. _WP07: B6.wp07_psyco_and_stackless.html From lac at codespeak.net Mon Sep 15 21:21:42 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Mon, 15 Sep 2003 21:21:42 +0200 (MEST) Subject: [pypy-svn] rev 1315 - pypy/trunk/doc/funding Message-ID: <20030915192142.C556F5B9CA@thoth.codespeak.net> Author: lac Date: Mon Sep 15 21:21:42 2003 New Revision: 1315 Modified: pypy/trunk/doc/funding/B3.0_impact.txt Log: first draft Modified: pypy/trunk/doc/funding/B3.0_impact.txt ============================================================================== --- pypy/trunk/doc/funding/B3.0_impact.txt (original) +++ pypy/trunk/doc/funding/B3.0_impact.txt Mon Sep 15 21:21:42 2003 @@ -10,23 +10,159 @@ (Recommended length ? three pages) -One of the greatest threats to European competitiveness is its -dependence upon Proprietary Closed Source Software, mostly made in the -United States. This is not only the matter of money being spent -in the United States is money that is not being spent here, although -that effect matters as well. There are two more serious risks. -The first is a threat-in-the-present. +This project has direct relevance to European competitiveness. Europe +is the acknowledged world leader in handheld, mobile, and embedded +devices. But people working in such industries have long desired a +high level language with a very small footprint. They wish to use +only the language features which they need. But modern languages +are chock full of features designed for the developer to use when +developing software. The new innovative concept of Object Spaces, +pioneered by PyPy makes the construction of tiny Object Spaces, +suitable for running on the smallest devices straight-forward. +A Python with greater speed will seemlessly improve the offerings of +those European Companies who develop using Python. These two reasons +make the Python Business Forum quite excited about PyPy. + +Morever, a great many companies, in deciding what language to develop +in, reject Very High Level Languages, despite their known advantages +for programmer productivity and code-reuse and maintainablity because +the code produced simply does not run fast enough. If we give them +a *fast* VHLL, they will switch. + +Furthermore, one of the greatest threats to European competitiveness +is its dependence upon Proprietary Closed Source Software, mostly made +in the United States. This is not only the matter of money being +spent in the United States is money that is not being spent here, +although that effect matters as well. There are two more serious +risks. The first is a threat in the present. Any company which writes its software in a Proprietary, Closed Source -language is at the mercy of its software provider. +language is dependent upon its software provider. If you have a bug, +you must wait for them to fix it. If this bug is not a high priority +for them, you can wait a long time. If you have access to the source +you always have the option of fixing it yourself, or hiring somebody +else to do that. But this is not the greatest of your worries. You +are at constant risk of having your software provider discontinue +support for your language. This is a real threat, not a theoretical +one. In 2002, Microsoft announced that it would no longer be +supporting Visual Basic 6.0 after the year 2005. All Visual Basic +Developers have been told to convert their code to run under +Microsoft's new .NET framework. Before that, in 2001 Microsoft +immediately stopped supporting its Visual J++ language, meant to be a +direct competitor with Java, after settling a lawsuit with Sun +Microsystems. No migration path was specified. Microsoft is making +these decisions because they make business sense for Microsoft, +regardless of the effects on European software developers. + +The second threat Closed Source makes to European competitiveness is +more insidious, and more long term. A good workman knows his tools. +This is much more than the theoretical knowledge of how his tools +ought to work, according to principles learned in school. The way +car mechanics know how cars work is distinctly different from the +what you would know if you had attended classes on 'the principles of +the internal combustion engine', let alone what you need to know to +just drive the thing. + +Right now, in Europe, we don't have enough of the software equivalents +of car-mechanics. And most of them live in academia, where they know +the intimate details of languages that never get used in industrial +applications. The world needs Formula-One race car mechanics, indeed, +but it has a much greater need for people who know how to repair the +family car. + +It is not as if there is a shortage of people who would be interested +in learning such things, if the source were made available. Many people +have taken this step by learning how CPython does its stuff. But still +there is a barrier. If you want to know how C Python does things, +you need to learn C. C is a notoriously difficult language to learn. + +But let me quote from an article posted to the Python-in-Education +mailing list. + +FIXME -- I promised Arthur I would fix any typos. + +Date: Sun, 14 Sep 2003 11:52:05 -0400 +From: Arthur +To: edu-sig at python.org +Subject: [Edu-sig] re : If the PyPy Project ... + +List-Id: Python in education + +Terry - + +>Since I presume the goal of PyPy is to implement *Python* in Python, +>wouldn't the implementation language be rather insignificant to an +>end-user such as an educator? Why would it be "better" than >CPython? + +For whatever reason, the complex built_in and the cmath module, implemented +in Python, are part of the early pypy codebase. As I had been spending some +time in the complex realm with PyGeo - a simple verions of the complex +realm, as these things go - Laura's post gave me the impetus to try to +plugin the pypy implmentations. + +Only got stuck on the typing issue. My code tests for +instance(object,complex). The pypy complexobject, unadorned, is a class - +and fails the test. But that leads me into a deeper look at some of the +pypy codebase, trying to understand a little bit of how this kind of issue +are to be dealt with. Not that I got there, yet - but I did seem to have an +avenue to explore I would not have with CPython - as someone who doesn't C, +and has no intention of trying, seriously, to do so. + +As someone living within the limits of having Python as my only real +language, I think that pypy should open things up for me considerably. It +will make Python, I believe, a more attractive educational language, because +it will make someone with a strong foundation in Python - as the language of +choice - a more self-sufficient programmer. + +Presumably - the point is - there will be less cases where the right +approach would be an extension module in C or C++, and a sense of +fundamental compromise sould one not be equipped to go there. Many +thousands of folks - using VB and the like - already do involed, highly +performing realworld applications and make nice livings doing so, without +being equipped to do C. I am thinking that pypy would put Python more +squarely in that "space". + +Is any of this so, or just hope? + +Art + + +_______________________________________________ +Edu-sig mailing list +Edu-sig at python.org +http://mail.python.org/mailman/listinfo/edu-sig +------------------------------------------------------ + +Here is somebody who is hoping we can give him a language he can understand. +Python already is an excellent teaching language. PyPy will be a better one. + +This project has to be done at the European or the International level. +That's where we all live. + +Since Education is a primary goal of the project, we will take every +opportunity to disseminate PyPy. The source will always be freely +available from our website. We will continue to give talks about PyPy +at EuroPython, Python-UK, OSCON (the International Open Source +Conference) the International Python Conference, and others. ADD SOME GOOD ONES THAT ARE FOR CSC ACADEMICS + +ASK STOCKHOLM -- will the EU pay for us to go to conferences? even +ones we were going to attend anyway? + +These talks will create interest as well as teach +techniques. We will submit a PEP and push to get PyPy made the reference +implementation of the Python programming language. We will continue +to discuss PyPy on our own mailing lists, as well as other Python +mailing lists such as the Python-in-Education list, and the Usenet +Newsgroup comp.lang.python. We already have an #irc channel, +irc.freenode.net where live online discussions happen, and where +we communicate with each other while we are apart. Beyond that -- +we are willing to take any actions the EU would like to fund. + +Ask Stockholm We think you get no money for this in a STREP. True? -these large American companies. And this is a real, and -not theoretical threat. In 2002, Microsoft announced that it would no -longer be supporting Visual Basic 6.0. after the year 2005. All Visual -Basic Developers have been told to convert their code to run under -Microsoft's new .NET framework. In 2001 Microsoft immediately stopped -supporting its Visual J++ language, meant to be a direct competitor with -Java, after settling a lawsuit with Sun Microsystems. Microsoft is -making these decisions because they make business sense for Microsoft, -regardless of the effects on businesses who develop software using -Microsoft proprietary software. +ASK STOCKHOLM -- +can we get some money to give Freenode? +re: 'Indicate what account is taken of other national or international +research activities.' -- Samuele and Armin read the literature all +the time. What do I say? From tismer at codespeak.net Tue Sep 16 12:01:04 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 12:01:04 +0200 (MEST) Subject: [pypy-svn] rev 1316 - pypy/trunk/doc/funding Message-ID: <20030916100104.8C8C75BD44@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 12:01:03 2003 New Revision: 1316 Modified: pypy/trunk/doc/funding/B3.0_impact.txt Log: made it build (email footers are no good in reST) Modified: pypy/trunk/doc/funding/B3.0_impact.txt ============================================================================== --- pypy/trunk/doc/funding/B3.0_impact.txt (original) +++ pypy/trunk/doc/funding/B3.0_impact.txt Tue Sep 16 12:01:03 2003 @@ -126,12 +126,13 @@ Art - -_______________________________________________ -Edu-sig mailing list -Edu-sig at python.org -http://mail.python.org/mailman/listinfo/edu-sig ------------------------------------------------------- +:: + + _______________________________________________ + Edu-sig mailing list + Edu-sig at python.org + http://mail.python.org/mailman/listinfo/edu-sig + ------------------------------------------------------ Here is somebody who is hoping we can give him a language he can understand. Python already is an excellent teaching language. PyPy will be a better one. From tismer at codespeak.net Tue Sep 16 12:10:05 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 12:10:05 +0200 (MEST) Subject: [pypy-svn] rev 1317 - pypy/trunk/doc/funding Message-ID: <20030916101005.EB5AC5BD44@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 12:10:04 2003 New Revision: 1317 Added: pypy/trunk/doc/funding/crossreferences.asc - copied unchanged from rev 1315, pypy/trunk/doc/funding/crossreferences.txt Removed: pypy/trunk/doc/funding/crossreferences.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt Log: renamed crossreferences.txt into crossreferences.asc because we don't want this file to be rendered. It is for including, only. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 16 12:10:04 2003 @@ -1,4 +1,4 @@ -.. include:: crossreferences.txt +.. include:: crossreferences.asc B 6.1 Workplan Introduction =============================== Modified: pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt (original) +++ pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt Tue Sep 16 12:10:04 2003 @@ -1,4 +1,4 @@ -.. include:: crossreferences.txt +.. include:: crossreferences.asc Workpackage Description ======================== Modified: pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt (original) +++ pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt Tue Sep 16 12:10:04 2003 @@ -1,4 +1,4 @@ -.. include:: crossreferences.txt +.. include:: crossreferences.asc Workpackage Description ======================== Modified: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt (original) +++ pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Tue Sep 16 12:10:04 2003 @@ -1,4 +1,4 @@ -.. include:: crossreferences.txt +.. include:: crossreferences.asc Workpackage Description ======================== Modified: pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt (original) +++ pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt Tue Sep 16 12:10:04 2003 @@ -1,4 +1,4 @@ -.. include:: crossreferences.txt +.. include:: crossreferences.asc Workpackage Description ======================== Deleted: /pypy/trunk/doc/funding/crossreferences.txt ============================================================================== --- /pypy/trunk/doc/funding/crossreferences.txt Tue Sep 16 12:10:04 2003 +++ (empty file) @@ -1,7 +0,0 @@ -.. _WP01: B6.wp01_interpreter_completion.html -.. _WP02: B6.wp02_cpython_to_python.html -.. _WP03: B6.wp03_translation_of_rpython.html -.. _WP04: B6.wp04_bootstrapping_pypy.html -.. _WP05: B6.wp05_several_object_implementations.html -.. _WP06: B6.wp06_translator_optimisations.html -.. _WP07: B6.wp07_psyco_and_stackless.html From tismer at codespeak.net Tue Sep 16 13:17:47 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 13:17:47 +0200 (MEST) Subject: [pypy-svn] rev 1318 - pypy/trunk/doc/funding Message-ID: <20030916111747.6F0275AAF6@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 13:17:46 2003 New Revision: 1318 Modified: pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt Log: going to split this one into two working packages. There are two major tasks which have little to do with each other. Modified: pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt (original) +++ pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt Tue Sep 16 13:17:46 2003 @@ -21,46 +21,21 @@ Title: Bootstrapping PyPy - Putting interpreter (WP01_, WP02_) and translator (WP03_) together. - -- Ability to translate arbitrary RPython source code (e.g. - the one produced in WP01_ into low-level code (C, Pyrex, Java, others). -- Creation of a stand-alone, not-PyPy-related tool for general - optimization of arbitrary but suitably restricted Python application or - parts thereof. +- Enhancing the PyPy Design. **Description of work** - - -**Deliverables** - - - -**Milestones and Expected Result** - -*XXX: This check-in is just temporary* - -The interpreter is written as an RPython program. The translator -has to translate this program into some low-level language. -The resulting program then needs to be supported by a special -runtime library. The work-flow of WP04_ is repetitive, -since it will not be possible to "get it right" in the -first attempt. Analysis and redesign will have to be -repeated until the result is satisfactory. **Part a): Specific Analysis and Redesign** -The global translation of PyPy is going to raise particular -problems, other than more general RPython programs do. -Since translation of RPython is the core idea of the -bootstrap process and the main target of the translator, -we need to investigate and isolate the -particular problems, and redesign specific parts of PyPy -to support translation, code generation and optimzation -better. This will also include a re-iteration of the interface -design between application level and interpreter level -until we reach overall convergence. +- Investigate the particular problems that the + global translation of PyPy is going to raise. +- Enhance and redesign parts of PyPy as needed + to give best support to the translation of RPython. +- Re-iterate this until we agree to have reached + an optimum solution. + **Part b): Low Level PyPy Runtime** @@ -75,3 +50,11 @@ If producing C code is a target, important parts can be directly re-used from CPython. + + +**Deliverables** + + + +**Milestones and Expected Result** + From tismer at codespeak.net Tue Sep 16 13:19:38 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 13:19:38 +0200 (MEST) Subject: [pypy-svn] rev 1319 - pypy/trunk/doc/funding Message-ID: <20030916111938.ADB5B5AAF6@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 13:19:38 2003 New Revision: 1319 Added: pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt - copied unchanged from rev 1318, pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt Log: splitted the bootstrap thing into two. Text will change now. From arigo at codespeak.net Tue Sep 16 13:30:24 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 16 Sep 2003 13:30:24 +0200 (MEST) Subject: [pypy-svn] rev 1320 - pypy/trunk/doc/funding Message-ID: <20030916113024.0A9AA5AAF6@thoth.codespeak.net> Author: arigo Date: Tue Sep 16 13:30:24 2003 New Revision: 1320 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt Log: made the blurb about Psyco as long as the one about Stackless :-) Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 16 13:30:24 2003 @@ -191,10 +191,13 @@ which have been traditionally depended on closely following CPython's code base. -Psyco should be re-implemented in pure Python, and maybe -partially in RPython. The intent is to be able to create -specialized code at run-time. All prior knowledge of the Psyco -project will be integrated into PyPy. +Psyco provides techniques to increase the performance of C +Python by generating specialized machine code at run-time. +Developing C Psyco has proved that more flexibility would be of +paramount importance to the project. All prior knowledge of the +Psyco project will thus be integrated into PyPy, as Python and +RPython code, enabling yet more efficient optimizations and +allowing new processor architectures to be targeted. Stackless Python has implemented high speed multitasking in a single thread for C Python in two different ways: From tismer at codespeak.net Tue Sep 16 14:04:49 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 14:04:49 +0200 (MEST) Subject: [pypy-svn] rev 1321 - pypy/trunk/doc/funding Message-ID: <20030916120449.31A645C304@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 14:04:48 2003 New Revision: 1321 Added: pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Removed: pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt Log: splitted Wp4 into two and reworked. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 16 14:04:48 2003 @@ -20,7 +20,7 @@ applications. -*XXX: AT the moment, I'm listing the working packages here. +*XXX: At the moment, I'm listing the working packages here. I think this is no good. The WPs are now moved out of here. The text here gets shortened, the text outside gets extended.* @@ -94,8 +94,9 @@ we could target (C, Pyrex, others?). -WP04_: _`Bootstrapping` PyPy -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +_`Bootstrapping` PyPy +------------------------------ + The goal is to put interpreter (WP01_, WP02_) and translator (WP03_) together. @@ -107,7 +108,9 @@ first attempt. Analysis and redesign will have to be repeated until the result is satisfactory. -**Part a): Specific Analysis and Redesign** + +WP04_: Specific Analysis and Redesign +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The global translation of PyPy is going to raise particular problems, other than more general RPython programs do. @@ -115,12 +118,14 @@ bootstrap process and the main target of the translator, we need to investigate and isolate the particular problems, and redesign specific parts of PyPy -to support translation, code generation and optimzation +to support translation, code generation and optimisation better. This will also include a re-iteration of the interface design between application level and interpreter level until we reach overall convergence. -**Part b): Low Level PyPy Runtime** + +WP05_: Low Level PyPy Runtime +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In order to give a working environment to the translated RPython program, we need to build the low-level-specific **run-time** @@ -145,18 +150,15 @@ executed directly, but goes though a code generator which produces the actual machine code. This layer is highly configurable. -The associated working packages -should be done in parallel,since the results are rather dependant -from each other. -WP05_: Several Object Implementations +WP06_: Several Object Implementations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ develop **several object implementations** for the same types, as explicitly allowed by the standard object space, and develop heuristics to switch between implementations during execution. (see `Annex SOI`_) Study the efficiency of different approaches. -WP06_: Translator Optimisations +WP07_: Translator Optimisations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It should be identified, which **optimizations** would benefit from support from the translator. These are the optimizations @@ -164,29 +166,25 @@ large-scale code rewrites. (see `Annex OPT`) This includes design-considerations including the decision whether to use reference counting together with garbage collection, -or to go for garbage collection, only. - -*XXX use a different WP for the next?* +or whether to go for garbage collection, only. For each issue, work on **several solutions** when no one is obviously -better than the other ones. The meta-programming underlying [[WP10]] +better than the other ones. The meta-programming underlying WP07_ -- namely the work on the translator instead of on the resulting code -- is what gives us the possibility of actually implementing several very different schemes. The outcome of this effort is of course not unique, but depends on the specific target of the optimization. There will be at least two efforts at the same time: -- optimization towards maximum performance fo an integrated PyPy system +- optimization towards maximum performance of an integrated PyPy system - optimization towards minimal code size for embedded systems. *add a lot more about this* -WP07_: Integration of Existing Technology +WP08_: Integration of Existing Technology ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -*XXX should this be split into two WPs?* - There are existing projects, notably Psyco and Stackless, which have been traditionally depended on closely following CPython's code base. @@ -208,8 +206,8 @@ between different machines. Rewrite each one as a meta-component that hooks into the -translator plus a dedicated run-time component. Further -develop these technologies based on the results gathered in WP04_, +translator plus a dedicated run-time component (WP05_). Further +develop these technologies based on the results gathered in WP07_, e.g. identify when these technologies would guide specific choices among the solutions developed in WP05_ and studying several solutions in WP06_. Modified: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt (original) +++ pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Tue Sep 16 14:04:48 2003 @@ -38,12 +38,13 @@ **Part b): Code Production** +- Compare different low-level environments that + we could target (C, Pyrex, others?). Select at least + one suitable target. - Produce low-level code out of the data gathered in Part a). - Again investigate how this is best done (AST-guided translation or reverse-engeneering of the low-level control flow gathered by the annotation object space). -- Compare different low-level environments that - we could target (C, Pyrex, others?). - Provide some measure for the efficiency of the generated code, in comparison to Standard Python. Modified: pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt (original) +++ pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt Tue Sep 16 14:04:48 2003 @@ -18,7 +18,7 @@ **Objectives** -Title: Bootstrapping PyPy +Title: Specific Analysis and Redesign - Putting interpreter (WP01_, WP02_) and translator (WP03_) together. - Enhancing the PyPy Design. @@ -36,20 +36,6 @@ - Re-iterate this until we agree to have reached an optimum solution. - -**Part b): Low Level PyPy Runtime** - -In order to give a working environment to the translated RPython -program, we need to build the low-level-specific **run-time** -components of PyPy. -Most notably are the object layout, the memory management, -and possibly threading support, as well as an -efficient multimethod dispatch. -The target language is not yet decided on. We might go different -paths at the same time. -If producing C code is a target, important -parts can be directly re-used from CPython. - **Deliverables** @@ -58,3 +44,4 @@ **Milestones and Expected Result** + Deleted: /pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp04_bootstrapping_pypy.txt Tue Sep 16 14:04:48 2003 +++ (empty file) @@ -1,60 +0,0 @@ -.. include:: crossreferences.asc - -Workpackage Description -======================== - -================================ ========================== -**Workpackage number** 04 --------------------------------- -------------------------- -**Start date or starting event** 0 -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 3 3 -================================= ======= ======= ======= ======= ======= ======= ======= ======= - - -**Objectives** - -Title: Bootstrapping PyPy - -- Putting interpreter (WP01_, WP02_) and translator (WP03_) together. -- Enhancing the PyPy Design. - - -**Description of work** - - -**Part a): Specific Analysis and Redesign** - -- Investigate the particular problems that the - global translation of PyPy is going to raise. -- Enhance and redesign parts of PyPy as needed - to give best support to the translation of RPython. -- Re-iterate this until we agree to have reached - an optimum solution. - - -**Part b): Low Level PyPy Runtime** - -In order to give a working environment to the translated RPython -program, we need to build the low-level-specific **run-time** -components of PyPy. -Most notably are the object layout, the memory management, -and possibly threading support, as well as an -efficient multimethod dispatch. -The target language is not yet decided on. We might go different -paths at the same time. -If producing C code is a target, important -parts can be directly re-used from CPython. - - - -**Deliverables** - - - -**Milestones and Expected Result** - Added: pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Tue Sep 16 14:04:48 2003 @@ -0,0 +1,52 @@ +.. include:: crossreferences.asc + +Workpackage Description +======================== + +================================ ========================== +**Workpackage number** 05 +-------------------------------- -------------------------- +**Start date or starting event** 0 +================================ ========================== + +================================= ======= ======= ======= ======= ======= ======= ======= ======= +**Participant id** X Y +--------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- +**Person-months per participant** 2 2 +================================= ======= ======= ======= ======= ======= ======= ======= ======= + + +**Objectives** + +Title: Low Level PyPy Runtime + +- Providing a Runtime Library for PyPy + + +**Description of work** + + +In order to give a working environment to the translated RPython +program, we need to build the low-level-specific **run-time** +components of PyPy. + +- Design the object layout and implement it. +- Decide about memory management and implement it. +- Consider threading support and implement it. Make sure + that the other components are thread-aware. +- Design and implement an efficient multimethod dispatch. + +The target language is not yet decided on. We might go different +paths at the same time. +If producing C code is a target, important +parts can be directly re-used from CPython. + + + +**Deliverables** + +DL??? Runtime Library for PyPy + +**Milestones and Expected Result** + + From tismer at codespeak.net Tue Sep 16 15:14:23 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 15:14:23 +0200 (MEST) Subject: [pypy-svn] rev 1322 - pypy/trunk/doc/funding Message-ID: <20030916131423.3D7825AAF6@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 15:14:22 2003 New Revision: 1322 Added: pypy/trunk/doc/funding/mkxref.py Modified: pypy/trunk/doc/funding/crossreferences.asc Log: added a little tool that creates the crossreferences automatically. It preserves extra info in the file. Well, refreshing, now back to the boring writing :-) Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Tue Sep 16 15:14:22 2003 @@ -1,7 +1,8 @@ -.. _WP01: B6.wp01_interpreter_completion.html -.. _WP02: B6.wp02_cpython_to_python.html -.. _WP03: B6.wp03_translation_of_rpython.html -.. _WP04: B6.wp04_bootstrapping_pypy.html -.. _WP05: B6.wp05_several_object_implementations.html -.. _WP06: B6.wp06_translator_optimisations.html -.. _WP07: B6.wp07_psyco_and_stackless.html +.. _WP01: B6.wp01_interpreter_completion.txt +.. _WP02: B6.wp02_cpython_to_python.txt +.. _WP03: B6.wp03_translation_of_rpython.txt +.. _WP04: B6.wp04_bootstrap_redesign.txt +.. _WP05: B6.wp05_bootstrap_runtime.txt +.. _WP06: +.. _WP07: +.. _WP08: Added: pypy/trunk/doc/funding/mkxref.py ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/mkxref.py Tue Sep 16 15:14:22 2003 @@ -0,0 +1,75 @@ +# mkxref.py +# +# Generate crossreferences.asc +# as a global include file for all text files here. +# + +import os, sys + +# find all work package files, like +# B6.wp02_cpython_to_python.txt + +START_STR = "B6.wp".lower() + +def get_wps_from_dir(): + res = {} + for fname in os.listdir("."): + flow = fname.lower() + pre, ext = os.path.splitext(flow) + if flow.startswith(START_STR) and ext == ".txt": + num = flow[len(START_STR):].split("_", 1)[0] + if len(num)==2 and num.isdigit(): + res["WP"+num] = fname + return res + +REF_FILE = "B6.1_plan_introduction.txt" + +def get_wps_from_file(): + # read the file and find all WPnn_ references + txt = file(REF_FILE).read() + res = {} + for piece in txt.split("WP"): + num = piece.split("_", 1)[0] + if len(num)==2 and num.isdigit(): + res["WP"+num] = "" + return res + +# read the existing xref file, find the WP section and replace it. + +XREF_FILE = "crossreferences.asc" + +def split_xref_text(): + """ split the xref file into three parts: + before, in, and after the workpackages + """ + try: + lines = file(XREF_FILE).readlines() + except IOError: + lines = [] + pre = [] + pat = "..\s+_WP\d\d\:" + for line in lines: + if re.match(pat, line): + break + pre.append(line) + post = lines[len(pre):] + mid = [] + for line in post[:]: + if re.match(pat, line): + mid.append(post.pop(0)) + else: + break + return pre, mid, post + +def build_xref_file(): + pre, old, post = split_xref_text() + wps = get_wps_from_file() + wps.update(get_wps_from_dir()) + wplist = wps.items() + wplist.sort() + mid = [ ".. _%s: %s\n" % (name, url) for name, url in wplist] + if mid != old: + file(XREF_FILE, "w").writelines( pre + mid + post) + +if __name__ == "__main__": + build_xref_file() \ No newline at end of file From arigo at codespeak.net Tue Sep 16 16:39:15 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 16 Sep 2003 16:39:15 +0200 (MEST) Subject: [pypy-svn] rev 1323 - pypy/trunk/doc/funding Message-ID: <20030916143915.3F6625C302@thoth.codespeak.net> Author: arigo Date: Tue Sep 16 16:39:14 2003 New Revision: 1323 Modified: pypy/trunk/doc/funding/B4.2_other_countries.txt Log: Switzerland question Modified: pypy/trunk/doc/funding/B4.2_other_countries.txt ============================================================================== --- pypy/trunk/doc/funding/B4.2_other_countries.txt (original) +++ pypy/trunk/doc/funding/B4.2_other_countries.txt Tue Sep 16 16:39:14 2003 @@ -9,7 +9,7 @@ ASK_STOCKHOLM -Samuele Pedroni is Swiss. But he says that he will move to do this +Samuele Pedroni is Swiss.[*] But he says that he will move to do this full time. What is the Status of Israel? Is a participant a physical person, or a company? Does it matter where members sleep at night? Could we just allocate funds to AB Strakt, say, and then @@ -19,3 +19,5 @@ would Tim have to move? If Samuele is moving, can we get the Marie Curie funds to pay for it? +[*] Armin Rigo is Swiss too. Switzerland and the EU have general +agreement for education and research programs. Do they cover this case? From tismer at codespeak.net Tue Sep 16 16:47:55 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 16:47:55 +0200 (MEST) Subject: [pypy-svn] rev 1324 - pypy/trunk/doc/funding Message-ID: <20030916144755.52F095C302@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 16:47:54 2003 New Revision: 1324 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/crossreferences.asc pypy/trunk/doc/funding/mkxref.py Log: getting forward. WP 6 is ready. Yet to do: give real estimates on months and dependencies. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 16 16:47:54 2003 @@ -141,6 +141,7 @@ _`High-performance` PyPy-Python ----------------------------------- + The goal is to optimize `Bootstrapping`_ in possibly various ways, building on its flexibility to go beyond CPython. The main lack of flexibility in CPython stems from the @@ -153,10 +154,16 @@ WP06_: Several Object Implementations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -develop **several object implementations** for the same types, as -explicitly allowed by the standard object space, and develop heuristics -to switch between implementations during execution. (see `Annex SOI`_) -Study the efficiency of different approaches. +Since the standard object space allows coexistance of different +implementations of the same type, we can develop +**several object implementations** . +We will develop heuristics to switch between diferent +implementations during execution. +The goal is to study the efficiency of different approaches, +with the possibility to change the default implementation +if favor of a different one, not known in CPython. +Some object layouts will further exist in parallel, if their +efficiency is highly application dependant.. WP07_: Translator Optimisations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -307,11 +314,6 @@ not yet worked out enough. Another hour...* -.. _Annex SOI: - -Annex SOI and others: *where do we put this?* - - Infrastructure ------------------ Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Tue Sep 16 16:47:54 2003 @@ -1,8 +1,9 @@ -.. _WP01: B6.wp01_interpreter_completion.txt -.. _WP02: B6.wp02_cpython_to_python.txt -.. _WP03: B6.wp03_translation_of_rpython.txt -.. _WP04: B6.wp04_bootstrap_redesign.txt -.. _WP05: B6.wp05_bootstrap_runtime.txt -.. _WP06: +.. _WP01: B6.wp01_interpreter_completion.html +.. _WP02: B6.wp02_cpython_to_python.html +.. _WP03: B6.wp03_translation_of_rpython.html +.. _WP04: B6.wp04_bootstrap_redesign.html +.. _WP05: B6.wp05_bootstrap_runtime.html +.. _WP06: B6.wp06_several_obj_impl.html .. _WP07: .. _WP08: +.. attention:: **2 files missing: WP07 WP08** Modified: pypy/trunk/doc/funding/mkxref.py ============================================================================== --- pypy/trunk/doc/funding/mkxref.py (original) +++ pypy/trunk/doc/funding/mkxref.py Tue Sep 16 16:47:54 2003 @@ -4,7 +4,7 @@ # as a global include file for all text files here. # -import os, sys +import os, sys, re # find all work package files, like # B6.wp02_cpython_to_python.txt @@ -19,7 +19,8 @@ if flow.startswith(START_STR) and ext == ".txt": num = flow[len(START_STR):].split("_", 1)[0] if len(num)==2 and num.isdigit(): - res["WP"+num] = fname + pre, ext = os.path.splitext(fname) + res["WP"+num] = pre + ".html" return res REF_FILE = "B6.1_plan_introduction.txt" @@ -47,7 +48,7 @@ except IOError: lines = [] pre = [] - pat = "..\s+_WP\d\d\:" + pat = "..\s+_WP\d\d\:|.. attention::" for line in lines: if re.match(pat, line): break @@ -68,6 +69,10 @@ wplist = wps.items() wplist.sort() mid = [ ".. _%s: %s\n" % (name, url) for name, url in wplist] + empties = [name for name, url in wplist if not url] + if empties: + mid.append(".. attention:: **%d files missing: %s**\n" % + (len(empties), " ".join(empties))) if mid != old: file(XREF_FILE, "w").writelines( pre + mid + post) From tismer at codespeak.net Tue Sep 16 17:14:06 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 17:14:06 +0200 (MEST) Subject: [pypy-svn] rev 1325 - pypy/trunk/doc/funding Message-ID: <20030916151406.255755C302@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 17:14:06 2003 New Revision: 1325 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/mkxref.py Log: WP7 is ready. Now on to WP8 which is fun (Psyco, Stackless) Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 16 17:14:06 2003 @@ -170,7 +170,7 @@ It should be identified, which **optimizations** would benefit from support from the translator. These are the optimizations not easily available to CPython because they would require -large-scale code rewrites. (see `Annex OPT`) +large-scale code rewrites. This includes design-considerations including the decision whether to use reference counting together with garbage collection, or whether to go for garbage collection, only. Modified: pypy/trunk/doc/funding/mkxref.py ============================================================================== --- pypy/trunk/doc/funding/mkxref.py (original) +++ pypy/trunk/doc/funding/mkxref.py Tue Sep 16 17:14:06 2003 @@ -71,8 +71,9 @@ mid = [ ".. _%s: %s\n" % (name, url) for name, url in wplist] empties = [name for name, url in wplist if not url] if empties: - mid.append(".. attention:: **%d files missing: %s**\n" % - (len(empties), " ".join(empties))) + n = len(empties) + mid.append(".. attention:: **%d file%s missing: %s**\n" % + (n, "s"[0:n>1], " ".join(empties))) if mid != old: file(XREF_FILE, "w").writelines( pre + mid + post) From tismer at codespeak.net Tue Sep 16 17:16:38 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 17:16:38 +0200 (MEST) Subject: [pypy-svn] rev 1326 - pypy/trunk/doc/funding Message-ID: <20030916151638.AA5065C302@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 17:16:38 2003 New Revision: 1326 Added: pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt Log: forgot to add the file... Added: pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt Tue Sep 16 17:16:38 2003 @@ -0,0 +1,62 @@ +.. include:: crossreferences.asc + +Workpackage Description +======================== + +================================ ========================== +**Workpackage number** 07 +-------------------------------- -------------------------- +**Start date or starting event** 0 +================================ ========================== + +================================= ======= ======= ======= ======= ======= ======= ======= ======= +**Participant id** X Y +--------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- +**Person-months per participant** 4 4 +================================= ======= ======= ======= ======= ======= ======= ======= ======= + + +**Objectives** + +Title: Translator Optimisations + +- Identification and Implementation of Optimisations + through modifications of the Translator. + +**Description of work** + +The following alternatives are mostly specific to a particular low-level run-time. + +* Implement and compare different object layout and memory management strategy + or strategies, e.g. reference counting vs. Boehm garbage collection vs. our own. + Includes speed vs. data size trade-offs. + +* Study approaches concerning code size vs. speed trade-offs (e.g. whether the + final interpreter should still include compact precompiled bytecode or be + completely translated into C). + +* Solve the complex issue of threading (global interpreter lock vs. + alternatives). + +* Enhance multimethod dispatching. + +* Implement schemes of pointer tagging, e.g. encoding an integer object as a pointer with a + special value instead of a real pointer to a data structure representing + the integer. + +* Run various performance tests on the above, check for an + overall improvement or application dependency. + +* Create reports and merge the results back into the optimization effort. + As of general interest, submit the reports as publication. + +Note that compatibility with legacy C extensions can be acheived by choosing, +for each of the following issues, the same one as CPython did. + + +**Deliverables** + +DL??? to be defined. Publication. + +**Milestones and Expected Result** + From tismer at codespeak.net Tue Sep 16 17:37:10 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 17:37:10 +0200 (MEST) Subject: [pypy-svn] rev 1328 - pypy/trunk/doc/funding Message-ID: <20030916153710.AA0F75AAF6@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 17:37:10 2003 New Revision: 1328 Modified: pypy/trunk/doc/funding/B4.2_other_countries.txt Log: turned foot note into a real foot note. Modified: pypy/trunk/doc/funding/B4.2_other_countries.txt ============================================================================== --- pypy/trunk/doc/funding/B4.2_other_countries.txt (original) +++ pypy/trunk/doc/funding/B4.2_other_countries.txt Tue Sep 16 17:37:10 2003 @@ -9,7 +9,7 @@ ASK_STOCKHOLM -Samuele Pedroni is Swiss.[*] But he says that he will move to do this +Samuele Pedroni is Swiss. [*]_ But he says that he will move to do this full time. What is the Status of Israel? Is a participant a physical person, or a company? Does it matter where members sleep at night? Could we just allocate funds to AB Strakt, say, and then @@ -19,5 +19,5 @@ would Tim have to move? If Samuele is moving, can we get the Marie Curie funds to pay for it? -[*] Armin Rigo is Swiss too. Switzerland and the EU have general -agreement for education and research programs. Do they cover this case? +.. [*] Armin Rigo is Swiss too. Switzerland and the EU have general + agreement for education and research programs. Do they cover this case? From tismer at codespeak.net Tue Sep 16 18:25:00 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 18:25:00 +0200 (MEST) Subject: [pypy-svn] rev 1329 - pypy/trunk/doc/funding Message-ID: <20030916162500.E66445A087@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 18:24:59 2003 New Revision: 1329 Added: pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt pypy/trunk/doc/funding/crossreferences.asc Log: Added and (hopefully) finished WP08. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 16 18:24:59 2003 @@ -175,8 +175,7 @@ to use reference counting together with garbage collection, or whether to go for garbage collection, only. -For each issue, work on **several solutions** when no one is obviously -better than the other ones. The meta-programming underlying WP07_ +The meta-programming underlying WP07_ -- namely the work on the translator instead of on the resulting code -- is what gives us the possibility of actually implementing several very different schemes. The outcome of this effort is of course not unique, @@ -187,7 +186,6 @@ - optimization towards minimal code size for embedded systems. -*add a lot more about this* WP08_: Integration of Existing Technology ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,12 +210,8 @@ Stackless Python and should enable PyPy for load-balancing between different machines. -Rewrite each one as a meta-component that hooks into the -translator plus a dedicated run-time component (WP05_). Further -develop these technologies based on the results gathered in WP07_, -e.g. identify when these technologies would guide specific -choices among the solutions developed in WP05_ and studying -several solutions in WP06_. +Both will be rewritten as a meta-component that hooks into the +translator plus a dedicated run-time component (WP05_). *XXX The following paragraph is not clear to me, Added: pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt Tue Sep 16 18:24:59 2003 @@ -0,0 +1,58 @@ +.. include:: crossreferences.asc + +Workpackage Description +======================== + +================================ ========================== +**Workpackage number** 06 +-------------------------------- -------------------------- +**Start date or starting event** 0 +================================ ========================== + +================================= ======= ======= ======= ======= ======= ======= ======= ======= +**Participant id** X Y +--------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- +**Person-months per participant** 2 2 +================================= ======= ======= ======= ======= ======= ======= ======= ======= + + +**Objectives** + +Title: Several Object Implementations + +- Providing several object implementations for + - general optimisation + - application specific alternatives + + +**Description of work** + +The following objects are mostly independent from any particular low-level run-time +environment. + +* Implement dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, + or with string-only or integer-only keys. Dictionaries with specific + support for "on-change" callbacks (useful for Psyco). + +* Implement strings as plain immutable memory buffers vs. immutable but more + complex data structures (see functional languages) vs. internally + mutable data structures (e.g. Psyco's concatenated strings) + +* Implement ints as machine words vs. two machine words vs. internal longs vs. + external bignum library (investigate if completely unifying ints and + longs is possible in the Python language at this stage). + +* Consider further objects and do multiple implementations of them. + +* Run various performance tests on the above, check for an + overall improvement or application dependency. + +* Create reports and merge the results back into the optimization effort. + As of general interest, submit the reports as publication. + +**Deliverables** + +DL??? to be defined. Publication. + +**Milestones and Expected Result** + Modified: pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt (original) +++ pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt Tue Sep 16 18:24:59 2003 @@ -46,6 +46,9 @@ * Run various performance tests on the above, check for an overall improvement or application dependency. + +* For each issue, work on **several solutions** when no one is obviously + better than the other ones. * Create reports and merge the results back into the optimization effort. As of general interest, submit the reports as publication. Added: pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt Tue Sep 16 18:24:59 2003 @@ -0,0 +1,66 @@ +.. include:: crossreferences.asc + +Workpackage Description +======================== + +================================ ========================== +**Workpackage number** 08 +-------------------------------- -------------------------- +**Start date or starting event** 0 +================================ ========================== + +================================= ======= ======= ======= ======= ======= ======= ======= ======= +**Participant id** AR CT X +--------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- +**Person-months per participant** 4 2 2 +================================= ======= ======= ======= ======= ======= ======= ======= ======= + + +**Objectives** + +Title: Integration of Existing Technology + +- Integration of Psyco's technology into PyPy. + - Optimization by Creating Specialized Machine Code at Run-time. + - Adressing new Processor Architectures +- Integration of Stackless's technology into PyPy. + - Enable Massive Parallelism in a Single Thread + - Allow Pickling of a running Program + + +**Description of work** + + +- Determine which parts of Psyco need to be written in RPython. + +- Design Psyco as a meta-component + that hooks into the translator (WP03_) + +- Implement Psyco partially in RPython, partially in Python, + favoring the latter. + +- Implement a dedicated run-time component (WP05_) for Psyco. + + + +- Determine which parts of Stackless need to be written in RPython. + +- Design Stackless as a meta-component + that hooks into the translator (WP03_) + +- Implement Stackless partially in RPython, partially in Python, + favoring the latter. + +- Implement a dedicated run-time component (WP05_) for Stackless. + +- Further develop these technologies based on the results gathered in WP07_, + e.g. identify when these technologies would guide specific choices among the + solutions developed in WP05_ and studying several solutions in WP06_. + + +**Deliverables** + +DL??? to be defined. Speed boost, architectures, green threads, pickling. + +**Milestones and Expected Result** + Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Tue Sep 16 18:24:59 2003 @@ -4,6 +4,5 @@ .. _WP04: B6.wp04_bootstrap_redesign.html .. _WP05: B6.wp05_bootstrap_runtime.html .. _WP06: B6.wp06_several_obj_impl.html -.. _WP07: -.. _WP08: -.. attention:: **2 files missing: WP07 WP08** +.. _WP07: B6.wp07_translator_optimisations.html +.. _WP08: B6.wp08_integrate_technology.html From tismer at codespeak.net Tue Sep 16 18:28:05 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 16 Sep 2003 18:28:05 +0200 (MEST) Subject: [pypy-svn] rev 1330 - pypy/trunk/doc/funding Message-ID: <20030916162805.94DE25A087@thoth.codespeak.net> Author: tismer Date: Tue Sep 16 18:28:04 2003 New Revision: 1330 Modified: pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt Log: needed blank lines to separate indentation levels Modified: pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt (original) +++ pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt Tue Sep 16 18:28:04 2003 @@ -21,9 +21,11 @@ Title: Integration of Existing Technology - Integration of Psyco's technology into PyPy. + - Optimization by Creating Specialized Machine Code at Run-time. - Adressing new Processor Architectures - Integration of Stackless's technology into PyPy. + - Enable Massive Parallelism in a Single Thread - Allow Pickling of a running Program From tismer at codespeak.net Wed Sep 17 11:38:07 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 17 Sep 2003 11:38:07 +0200 (MEST) Subject: [pypy-svn] rev 1331 - pypy/trunk/doc/funding Message-ID: <20030917093807.0813F5A114@thoth.codespeak.net> Author: tismer Date: Wed Sep 17 11:38:07 2003 New Revision: 1331 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt Log: moving on. Trying to enhance the introduction structure. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Wed Sep 17 11:38:07 2003 @@ -3,8 +3,17 @@ B 6.1 Workplan Introduction =============================== -The PyPy project has a number of objectives, which -can be categorized into two major groups: +The PyPy project has a number of topics, which +can be categorized into the following groups. +Each group is further broken down into working packages. + +- The PyPy Interpreter + +- The PyPy Compiler + +- Bootstrapping PyPy + +- High-performance PyPy-Python - development of PyPy itself, as a Python written in Python. This includes code generation for various targets, re-implementation @@ -20,14 +29,9 @@ applications. -*XXX: At the moment, I'm listing the working packages here. -I think this is no good. The WPs are now moved out of here. -The text here gets shortened, the text outside gets -extended.* +XXX re-organise: Major topics, WPs go beyond them. -Working Packages -=================== The PyPy Interpreter --------------------------- @@ -38,11 +42,11 @@ WP01_: Development and Completion of the PyPy Interpreter ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The PyPy interpreter itself should be developed and completed -as a regular Python program. This package includes all +as a regular Python/RPython program. This package includes all parts of CPython tht we don't want to move to WP02_. -Further investigate the unorthodox multimethod -concepts that the standard object space is based on, and how to hook in -the bytecode compiler. +Further investigation is needed concerning the +multimethod concepts of the standard object space, +and how to hook in the bytecode compiler. WP02_: Translation of CPython into regular Python @@ -139,10 +143,10 @@ parts can be directly re-used from CPython. -_`High-performance` PyPy-Python +High-performance PyPy-Python ----------------------------------- -The goal is to optimize `Bootstrapping`_ in possibly various ways, +The goal is to optimize Bootstrapping in possibly various ways, building on its flexibility to go beyond CPython. The main lack of flexibility in CPython stems from the fact that all structures are hard-coded in C, and there is @@ -213,9 +217,7 @@ Both will be rewritten as a meta-component that hooks into the translator plus a dedicated run-time component (WP05_). - -*XXX The following paragraph is not clear to me, -how to group it into packages. Armin?* +XXX turning the following into an application for embedded sstems. Low-level targets, tools and releases -------------------------------------------- @@ -234,7 +236,7 @@ (e.g. a pure Python one that should allow generic third-party code to hook into the PyPy interpreter source code without worrying about the translation process). Others depend on the low-level environment and on - the choices made for the issues of `High-performance`. + the choices made for the issues of High-performance. c) (_`APP`)combine different solutions for the different issues discussed in `High-performance`_. Gather statistics with real-work Python applications. Modified: pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt (original) +++ pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt Wed Sep 17 11:38:07 2003 @@ -26,11 +26,12 @@ **Description of work** -- Coordinate with WP02_ which parts of CPython should be implemented in RPython +- Coordinate with WP02_ which parts of CPython + should be implemented in RPython. - Implementation of all C modules and C objects - Further investigation of the unorthodox multimethod - concepts that the standard object space is based on, and how to hook in - the bytecode compiler. + concepts that the standard object space is based on, + and how to hook in the bytecode compiler. **Deliverables** Modified: pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt (original) +++ pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt Wed Sep 17 11:38:07 2003 @@ -24,10 +24,10 @@ - Optimization by Creating Specialized Machine Code at Run-time. - Adressing new Processor Architectures -- Integration of Stackless's technology into PyPy. +- Integration of Stackless's Technology into PyPy. - Enable Massive Parallelism in a Single Thread - - Allow Pickling of a running Program + - Allow Pickling of a Running Program **Description of work** From arigo at codespeak.net Wed Sep 17 13:38:07 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Sep 2003 13:38:07 +0200 (MEST) Subject: [pypy-svn] rev 1332 - in pypy/branch/builtinrefactor/pypy: appspace interpreter interpreter/test module module/test objspace objspace/std objspace/std/attic objspace/std/test Message-ID: <20030917113807.E6D325A114@thoth.codespeak.net> Author: arigo Date: Wed Sep 17 13:38:03 2003 New Revision: 1332 Added: pypy/branch/builtinrefactor/pypy/interpreter/debug.py pypy/branch/builtinrefactor/pypy/interpreter/module.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/ pypy/branch/builtinrefactor/pypy/objspace/std/attic/classobject_app.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/classobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/dictobject_app.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/dictobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/floatobject_app.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/floatobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/funcobject.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/funcobject.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/functype.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/functype.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/generatorobject.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/generatorobject.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/generatortype.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/generatortype.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/instmethobject.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/instmethobject.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/instmethtype.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/instmethtype.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/intobject_app.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/intobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/moduleobject.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/moduleobject.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/moduletype.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/moduletype.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/nullobject.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/nullobject.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/nulltype.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/nulltype.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/sliceobject_app.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/sliceobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/attic/stringobject_app.py (props changed) - copied unchanged from rev 1322, pypy/branch/builtinrefactor/pypy/objspace/std/stringobject_app.py Removed: pypy/branch/builtinrefactor/pypy/objspace/std/classobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/dictobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/floatobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/funcobject.py pypy/branch/builtinrefactor/pypy/objspace/std/functype.py pypy/branch/builtinrefactor/pypy/objspace/std/generatorobject.py pypy/branch/builtinrefactor/pypy/objspace/std/generatortype.py pypy/branch/builtinrefactor/pypy/objspace/std/instmethobject.py pypy/branch/builtinrefactor/pypy/objspace/std/instmethtype.py pypy/branch/builtinrefactor/pypy/objspace/std/intobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/moduleobject.py pypy/branch/builtinrefactor/pypy/objspace/std/moduletype.py pypy/branch/builtinrefactor/pypy/objspace/std/nullobject.py pypy/branch/builtinrefactor/pypy/objspace/std/nulltype.py pypy/branch/builtinrefactor/pypy/objspace/std/sliceobject_app.py pypy/branch/builtinrefactor/pypy/objspace/std/stringobject_app.py Modified: pypy/branch/builtinrefactor/pypy/appspace/types.py pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py pypy/branch/builtinrefactor/pypy/interpreter/error.py pypy/branch/builtinrefactor/pypy/interpreter/eval.py pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py pypy/branch/builtinrefactor/pypy/interpreter/function.py pypy/branch/builtinrefactor/pypy/interpreter/gateway.py pypy/branch/builtinrefactor/pypy/interpreter/generator.py pypy/branch/builtinrefactor/pypy/interpreter/main.py pypy/branch/builtinrefactor/pypy/interpreter/miscutils.py pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py pypy/branch/builtinrefactor/pypy/interpreter/py.py pypy/branch/builtinrefactor/pypy/interpreter/pycode.py pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py pypy/branch/builtinrefactor/pypy/interpreter/test/test_interpreter.py pypy/branch/builtinrefactor/pypy/interpreter/test/test_main.py pypy/branch/builtinrefactor/pypy/interpreter/test/test_objspace.py pypy/branch/builtinrefactor/pypy/interpreter/unittest_w.py pypy/branch/builtinrefactor/pypy/module/builtin.py pypy/branch/builtinrefactor/pypy/module/sysmodule.py pypy/branch/builtinrefactor/pypy/module/test/test_builtin.py pypy/branch/builtinrefactor/pypy/module/test/test_sysmodule.py pypy/branch/builtinrefactor/pypy/objspace/std/cpythonobject.py pypy/branch/builtinrefactor/pypy/objspace/std/dictobject.py pypy/branch/builtinrefactor/pypy/objspace/std/dicttype.py pypy/branch/builtinrefactor/pypy/objspace/std/floatobject.py pypy/branch/builtinrefactor/pypy/objspace/std/intobject.py pypy/branch/builtinrefactor/pypy/objspace/std/listobject.py pypy/branch/builtinrefactor/pypy/objspace/std/multimethod.py pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py pypy/branch/builtinrefactor/pypy/objspace/std/register_all.py pypy/branch/builtinrefactor/pypy/objspace/std/sliceobject.py pypy/branch/builtinrefactor/pypy/objspace/std/slicetype.py pypy/branch/builtinrefactor/pypy/objspace/std/stringobject.py pypy/branch/builtinrefactor/pypy/objspace/std/test/test_dictobject.py pypy/branch/builtinrefactor/pypy/objspace/std/test/test_instmethobject.py pypy/branch/builtinrefactor/pypy/objspace/std/test/test_sliceobject.py pypy/branch/builtinrefactor/pypy/objspace/std/tupleobject.py pypy/branch/builtinrefactor/pypy/objspace/std/typeobject.py pypy/branch/builtinrefactor/pypy/objspace/std/typetype.py pypy/branch/builtinrefactor/pypy/objspace/trivial.py Log: Big "final" check-in in this branch. Individual comments about changes will be made during the merge. Modified: pypy/branch/builtinrefactor/pypy/appspace/types.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/appspace/types.py (original) +++ pypy/branch/builtinrefactor/pypy/appspace/types.py Wed Sep 17 13:38:03 2003 @@ -4,7 +4,7 @@ Types that are part of optional modules (e.g. array) are not listed. """ -from __future__ import generators +#from __future__ import generators import sys @@ -101,4 +101,4 @@ #DictProxyType = type(TypeType.__dict__) -del sys, _f, _C, _x, generators # Not for export +del sys, _f, _C, _x#, generators # Not for export Modified: pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py Wed Sep 17 13:38:03 2003 @@ -53,8 +53,7 @@ self.sys._setmodule(self.w_sys) # XXX get rid of this. - def get_builtin_module(self, w_name): - name = self.unwrap(w_name) + def get_builtin_module(self, name): if name == '__builtin__': return self.w_builtin elif name == 'sys': @@ -301,7 +300,5 @@ # newstring([w_1, w_2,...]) -> w_string from ascii numbers (bytes) # newdict([(w_key,w_value),...]) -> w_dict # newslice(w_start,w_stop,w_end) -> w_slice (w_end may be a real None) -# newfunction(...) -> w_function -# newmodule(w_name) -> w_module # next(w_iter) -> w_value or raise NoValue # Added: pypy/branch/builtinrefactor/pypy/interpreter/debug.py ============================================================================== --- (empty file) +++ pypy/branch/builtinrefactor/pypy/interpreter/debug.py Wed Sep 17 13:38:03 2003 @@ -0,0 +1,11 @@ +""" +PyPy-oriented interface to pdb. +""" + +import pdb, sys + +def fire(operationerr): + if not operationerr.debug_tbs: + return + tb = operationerr.debug_tbs[-1] + pdb.post_mortem(tb) Modified: pypy/branch/builtinrefactor/pypy/interpreter/error.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/error.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/error.py Wed Sep 17 13:38:03 2003 @@ -1,5 +1,7 @@ import sys +AUTO_DEBUG = 1 + class PyPyError(Exception): "Raise this when you encounter an exceptional situation in PyPy itself." @@ -111,6 +113,9 @@ print >> file, exc_typename else: print >> file, exc_typename+':', exc_value + if AUTO_DEBUG: + import debug + debug.fire(self) # Utilities Modified: pypy/branch/builtinrefactor/pypy/interpreter/eval.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/eval.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/eval.py Wed Sep 17 13:38:03 2003 @@ -30,9 +30,9 @@ and possibly more locals.""" argnames, varargname, kwargname = self.signature() if varargname is not None: - argnames.append(argname) + argnames = argnames + [varargname] if kwargname is not None: - argnames.append(kwargname) + argnames = argnames + [kwargname] return argnames @@ -43,13 +43,14 @@ """A frame is an environment supporting the execution of a code object. Abstract base class.""" - def __init__(self, space, code, w_globals, numlocals=0): + def __init__(self, space, code, w_globals=None, numlocals=-1): self.space = space self.code = code # Code instance self.w_globals = w_globals # wrapped dict of globals self.w_locals = None # wrapped dict of locals - # flat list of wrapped locals - self.fastlocals_w = [UNDEFINED]*numlocals + if numlocals < 0: # compute the minimal size based on arguments + numlocals = len(code.getvarnames()) + self.fastlocals_w = [UNDEFINED]*numlocals # flat list of wrapped locals def run(self): "Run the frame." @@ -83,7 +84,7 @@ """Initialize the fast locals from a list of values, where the order is according to self.code.signature().""" if len(scope_w) > len(self.fastlocals_w): - raise ValueError, "too many fastlocals" + raise ValueError, "new fastscope is longer than the allocated area" self.fastlocals_w[:len(scope_w)] = scope_w def fast2locals(self): Modified: pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py Wed Sep 17 13:38:03 2003 @@ -4,34 +4,37 @@ """ -from pypy.interpreter.gateway import DictProxy -from pypy.interpreter.miscutils import InitializedClass -from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter import gateway +from pypy.interpreter.miscutils import InitializedClass, RwDictProxy +from pypy.interpreter.module import Module -class ExtModule(Wrappable): +class ExtModule(Module): """An empty extension module. Non-empty extension modules are made by subclassing ExtModule.""" def __init__(self, space): - self.space = space - self.w_dict = self._appdict.makedict(space, self) + Module.__init__(self, space, space.wrap(self.__name__)) + + # to build the dictionary of the module we get all the objects + # accessible as 'self.xxx'. Methods are bound. + d = {} + for cls in self.__class__.__mro__: + for name, value in cls.__dict__.iteritems(): + # ignore names in '_xyz' + if not name.startswith('_') or name.endswith('_'): + if isinstance(value, gateway.Gateway): + name = value.name + value = value.__get__(self) # get a Method + else: + if hasattr(value, '__get__'): + continue # ignore CPython functions + if name not in d: + d[name] = value + for key, value in d.iteritems(): + space.setitem(self.w_dict, space.wrap(key), space.wrap(value)) __metaclass__ = InitializedClass def __initclass__(cls): - # make sure that the class has its own appdict - if '_appdict' not in cls.__dict__: - cls._appdict = DictProxy(implicitspace=True, - implicitself=True) - # automatically publish all functions from this class - cls._appdict.exportall(cls.__dict__) - # automatically import all app_ functions - cls._appdict.importall(cls.__dict__, cls) - - def __getattr__(self, attr): - # XXX temporary, for use by objspace.trivial and - # objspace.std.cpythonobject - try: - return self.__class__._appdict.content[attr] - except KeyError: - raise AttributeError, attr + gateway.exportall(RwDictProxy(cls)) # xxx() -> app_xxx() + gateway.importall(RwDictProxy(cls)) # app_xxx() -> xxx() Modified: pypy/branch/builtinrefactor/pypy/interpreter/function.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/function.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/function.py Wed Sep 17 13:38:03 2003 @@ -13,15 +13,13 @@ an object space, a dictionary of globals, default arguments, and an arbitrary 'closure' passed to the code object.""" - def __init__(self, space, code, w_globals, w_defs=None, closure=None): + def __init__(self, space, code, w_globals=None, defs_w=[], closure=None): self.space = space self.func_code = code # Code instance self.w_globals = w_globals # the globals dictionary self.closure = closure # normally, list of Cell instances or None - if w_defs is None: - self.defs_w = [] - else: - self.defs_w = space.unpackiterable(w_defs) # list of w_default's + self.defs_w = defs_w # list of w_default's + self.__name__ = self.func_code.co_name # XXX def call(self, w_args, w_kwds=None): scope_w = self.parse_args(w_args, w_kwds) @@ -29,6 +27,7 @@ self.closure) frame.setfastscope(scope_w) return frame.run() + pypy_call = call def parse_args(self, w_args, w_kwds=None): """ parse args and kwargs to initialize the frame. @@ -46,9 +45,10 @@ # We try to give error messages following CPython's, which are # very informative. # - if w_kwds is None: + if w_kwds is None or not space.is_true(w_kwds): w_kwargs = space.newdict([]) else: + # space.is_true() above avoids infinite recursion copy<->parse_args w_kwargs = space.call_method(w_kwds, "copy") co_argcount = len(argnames) # expected formal arguments, without */** @@ -131,13 +131,13 @@ msg2, plural, nargs) - raise OperationError(self.space.w_TypeError, msg) + raise OperationError(self.space.w_TypeError, self.space.wrap(msg)) def raise_argerr_multiple_values(self, argname): msg = "%s() got multiple values for keyword argument %s" % ( self.func_code.co_name, argname) - raise OperationError(self.space.w_TypeError, msg) + raise OperationError(self.space.w_TypeError, self.space.wrap(msg)) def raise_argerr_unknown_kwds(self, w_kwds): nkwds = self.space.unwrap(self.space.len(w_kwds)) @@ -151,22 +151,65 @@ msg = "%s() got %d unexpected keyword arguments" % ( self.func_code.co_name, nkwds) - raise OperationError(self.space.w_TypeError, msg) + raise OperationError(self.space.w_TypeError, self.space.wrap(msg)) + + def __get__(self, obj, cls=None): + wrap = self.space.wrap + if obj is not None: + if cls is None: + cls = obj.__class__ + return Method(self.space, wrap(self), wrap(obj), wrap(cls)) + else: + return Method(self.space, wrap(self), None, wrap(cls)) + + def pypy_get(self, w_obj, w_cls): + wrap = self.space.wrap + if not self.space.is_true(self.space.is_(w_obj, self.space.w_None)): + if self.space.is_true(self.space.is_(w_cls, self.space.w_None)): + w_cls = self.space.type(w_obj) + return wrap(Method(self.space, wrap(self), w_obj, w_cls)) + else: + return wrap(Method(self.space, wrap(self), None, w_cls)) + def __call__(self, *args_w, **kwds_w): + wrap = self.space.wrap + w_args = self.space.newtuple(args_w) + w_kwds = self.space.newdict([(wrap(key), w_value) + for key, w_value in kwds_w.items()]) + return self.call(w_args, w_kwds) + + +class Method(object): + """A method is a function bound to a specific instance or class.""" + + def __init__(self, space, w_function, w_instance, w_class): + self.space = space + self.w_function = w_function + self.w_instance = w_instance # or None + self.w_class = w_class - def __get__(self, inst, cls=None): - # for TrivialObjSpace only !!! - # use the mecanisms of gateway.py otherwise - import sys, new - assert 'pypy.objspace.trivial' in sys.modules, ( - "don't try to __get__() Function instances out of classes") - self.__name__ = self.func_code.co_name - return new.instancemethod(self, inst, cls) - - def __call__(self, *args, **kwds): - # for TrivialObjSpace only !!! - # use the mecanisms of gateway.py otherwise - import sys, new - assert 'pypy.objspace.trivial' in sys.modules, ( - "don't try to __call__() Function instances directly") - return self.call(args, kwds) + def call(self, w_args, w_kwds=None): + args_w = self.space.unpacktuple(w_args) + if self.w_instance is not None: + # bound method + args_w = [self.w_instance] + args_w + w_args = self.space.newtuple(args_w) + else: + # unbound method + if (len(args_w) >= 1 and self.space.is_true( + self.space.isinstance(args_w[0], self.w_class))): + pass # ok + else: + msg = ("unbound method must be called with " + "instance as first argument") # XXX fix error msg + raise OperationError(self.space.w_TypeError, + self.space.wrap(msg)) + return self.space.call(self.w_function, w_args, w_kwds) + pypy_call = call + + def __call__(self, *args_w, **kwds_w): + wrap = self.space.wrap + w_args = self.space.newtuple(args_w) + w_kwds = self.space.newdict([(wrap(key), w_value) + for key, w_value in kwds_w.items()]) + return self.call(w_args, w_kwds) Modified: pypy/branch/builtinrefactor/pypy/interpreter/gateway.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/gateway.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/gateway.py Wed Sep 17 13:38:03 2003 @@ -2,21 +2,19 @@ Gateway between app-level and interpreter-level: * BuiltinCode (call interp-level code from app-level) +* Gateway (a space-independent gateway to a Code object) * app2interp (embed an app-level function into an interp-level callable) * interp2app (publish an interp-level object to be visible from app-level) -* publishall (mass-call interp2app on a whole list of objects) +* exportall (mass-call interp2app on a whole dict of objects) +* importall (mass-call app2interp on a whole dict of objects) """ -# -# XXX warning, this module is a bit scary in the number of classes that -# all play a similar role but in slightly different contexts -# - import types +from weakref import WeakKeyDictionary from pypy.interpreter import eval, pycode from pypy.interpreter.baseobjspace import Wrappable, ObjSpace -from pypy.interpreter.function import Function +from pypy.interpreter.function import Function, Method class BuiltinCode(eval.Code): @@ -25,47 +23,23 @@ # When a BuiltinCode is stored in a Function object, # you get the functionality of CPython's built-in function type. - def __init__(self, func, **argflags): + def __init__(self, func): # 'implfunc' is the interpreter-level function. - # See below for 'argflags'. # Note that this uses a lot of (construction-time) introspection. eval.Code.__init__(self, func.__name__) self.func = func - self.argflags = argflags # extract the signature from the (CPython-level) code object tmp = pycode.PyCode(None) tmp._from_code(func.func_code) self.sig = tmp.signature() - if isinstance(func, types.MethodType) and func.im_self is not None: - argnames, varargname, kwargname = self.sig - argnames = argnames[1:] # implicit hidden self argument - self.sig = argnames, varargname, kwargname - self.nargs = len(self.getvarnames()) - - def bind_code(self, instance): - """Create another version of this code object that calls 'func' - as a method, with implicit first argument 'instance'.""" - return BuiltinCode(self.func.__get__(instance, instance.__class__), - **self.argflags) + self.ismethod = False def create_frame(self, space, w_globals, closure=None): - return BuiltinFrame(space, self, w_globals, numlocals=self.nargs) + return BuiltinFrame(space, self, w_globals) def signature(self): return self.sig -# An application-level function always implicitely expects wrapped arguments, -# but not an interpreter-level function not. The extra keywords given to the -# constructor of BuiltinCode describes what kind of arguments 'func' expects. -# -# Default signature: -# def func(space, w_arg1, w_arg2...) <- plain functions -# def func(self, space, w_arg1, w_arg2...) <- methods -# -# Flags: (XXX need more) -# implicitspace=True method with no 'space' arg. We use 'self.space' -# implicitself=True the app-level doesn't see any 'self' argument - class BuiltinFrame(eval.Frame): "Frame emulation for BuiltinCode." @@ -75,8 +49,10 @@ def run(self): argarray = self.fastlocals_w - if not self.code.argflags.get('implicitspace'): - argarray = [space] + argarray + if self.code.ismethod: + argarray = [self.space.unwrap(argarray[0])] + argarray[1:] + else: + argarray = [self.space] + argarray return call_with_prepared_arguments(self.space, self.code.func, argarray) @@ -103,198 +79,140 @@ class Gateway(object): - # General-purpose utility for the interpreter-level to create callables - # that transparently invoke code objects (and thus possibly interpreted - # app-level code). - - # 'argflags' controls how the Gateway instance should decode its - # arguments. It only influences calls made from interpreter-level. - # It has the same format as BuiltinCode.argflags. + """General-purpose utility for the interpreter-level to create callables + that transparently invoke code objects (and thus possibly interpreted + app-level code).""" + + # This is similar to a Function object, but not bound to a particular + # object space. During the call, the object space is either given + # explicitely as the first argument (for plain function), or is read + # from 'self.space' for methods. - def __init__(self, code, staticglobals, staticdefs=[], **argflags): + def __init__(self, name, code, staticglobals=None, staticdefs=[]): + self.name = name self.code = code - self.staticglobals = staticglobals # a DictProxy instance + self.staticglobals = staticglobals self.staticdefs = staticdefs - self.argflags = argflags - - def make_function(self, space, bind_instance=None, w_globals=None): - if w_globals is None: - #w_globals = space.wrap(self.staticglobals) - w_globals = self.staticglobals.makedict(space, bind_instance) - defs_w = [space.wrap(def_value) for def_value in self.staticdefs] - code = self.code - if self.argflags.get('implicitself') and isinstance(code, BuiltinCode): - assert bind_instance is not None, ("built-in function can only " - "be used as a method") - code = code.bind_code(bind_instance) - return Function(space, code, w_globals, defs_w) + self.functioncache = WeakKeyDictionary() # map {space: Function} def __wrap__(self, space): # to wrap a Gateway, we first make a real Function object out of it # and the result is a wrapped version of this Function. - return space.wrap(self.make_function(space)) + return space.wrap(self.get_function(space)) - def __call__(self, space, *args, **kwds): - wrap = space.wrap - w_args = [wrap(arg) for arg in args] - w_kwds = space.newdict([(wrap(key), wrap(value)) - for key, value in kwds.items()]) - fn = self.make_function(space) - return fn.call(w_args, w_kwds) + def __call__(self, space, *args_w, **kwds_w): + # to call the Gateway as a non-method, 'space' must be explicitely + # supplied. We build the Function object and call it. + fn = self.get_function(space) + return fn(*args_w, **kwds_w) def __get__(self, obj, cls=None): + # to get the Gateway as a method out of an instance, we build a + # Function and get it. if obj is None: - return self - else: - return BoundGateway(self, obj) - - -class BoundGateway(object): - - def __init__(self, gateway, obj): - self.gateway = gateway - self.obj = obj - - def __call__(self, *args, **kwds): - if self.gateway.argflags.get('implicitspace'): - # we read 'self.space' from the object we are bound to - space = self.obj.space + return self # Gateways as unbound methods not implemented else: - space = args[0] # explicit 'space' as a first argument - args = args[1:] - if not isinstance(space, ObjSpace): - raise TypeError, "'space' expected as first argument" - wrap = space.wrap - if self.gateway.argflags.get('implicitself'): - pass # app-space gets no 'self' argument - else: - args = (wrap(self.obj),) + args # insert 'w_self' - w_args = [wrap(arg) for arg in args] - w_kwds = space.newdict([(wrap(key), wrap(value)) - for key, value in kwds.items()]) - fn = self.gateway.make_function(space, self.obj) - return fn.call(w_args, w_kwds) - - -class DictProxy(Wrappable): - # This class exposes at app-level a read-only dict-like interface. - # The items in the DictProxy are not wrapped (they are independent - # of any object space, and are just interpreter-level objects) until - # app-level code reads them. - - # Instances of DictProxy play the role of the 'globals' for app-level - # helpers. This is why app2interp and interp2app are methods of - # DictProxy. Calling them on the same DictProxy for several functions - # gives all these functions the same 'globals', allowing them to see - # and call each others. - - # XXX a detail has been (temporarily?) changed because too many - # places (notably in pyopcode.py) assume that the globals should - # satisfy 'instance(globals, dict)'. Now wrapping a DictProxy - # gives you a real dict. - - def __init__(self, basedict=None, **defaultargflags): - if basedict is None: - self.content = {} - else: - self.content = basedict.content.copy() - self.defaultargflags = defaultargflags - - #def __getitem__(self, key): - # return self.content[key] - # - #def __iter__(self): - # return iter(self.content) - def __wrap__(self, space): - return self.makedict(space) - - def app2interp(self, app, app_name=None, **argflags): - """Build a Gateway that calls 'app' at app-level and insert it - into the DictProxy.""" - # app must be a function whose name starts with 'app_' - if not isinstance(app, types.FunctionType): - raise TypeError, "function expected, got %r instead" % app - if app_name is None: - if not app.func_name.startswith('app_'): - raise ValueError, ("function name must start with 'app_'; " - "%r does not" % app.func_name) - app_name = app.func_name[4:] - argflags1 = self.defaultargflags.copy() - argflags1.update(argflags) - code = pycode.PyCode(None) - code._from_code(app.func_code) - staticdefs = list(app.func_defaults or ()) - gateway = Gateway(code, self, staticdefs, **argflags1) - self.content[app_name] = gateway - return gateway - - def interp2app(self, f, **argflags): - """Insert an interp-level function into the DictProxy to make - it callable from app-level.""" - # f must be a function whose name does NOT starts with 'app_' - if not isinstance(f, types.FunctionType): - raise TypeError, "function expected, got %r instead" % f - assert not f.func_name.startswith('app_'), ( - "function name %r suspiciously starts with 'app_'" % f.func_name) - argflags1 = self.defaultargflags.copy() - argflags1.update(argflags) - builtincode = BuiltinCode(f, **argflags1) - staticdefs = list(f.func_defaults or ()) - gateway = Gateway(builtincode, self, staticdefs, **argflags1) - self.content[f.func_name] = gateway - - def exportname(self, name, obj, optional=0): - """Publish an object of the given name by inserting it into the - DictProxy. See implementation for the known types of 'obj'.""" - if name.startswith('app_'): - publicname = name[4:] - optional = 0 - else: - publicname = name - if isinstance(obj, types.FunctionType): - assert name == obj.func_name - if name == publicname: - # an interpreter-level function - self.interp2app(obj) + # the object space is implicitely fetched out of the instance + fn = self.get_function(obj.space) + self.code.ismethod = True # that's a hack all right + return fn.__get__(obj, cls) + + def get_function(self, space): + try: + return self.functioncache[space] + except KeyError: + # the construction is supposed to be done only once in advance, + # but must be done lazily when needed only, because + # 1) it depends on the object space + # 2) the w_globals must not be built before the underlying + # staticglobals is completely initialized, because + # w_globals must be built only once for all the Gateway + # instances of staticglobals + if self.staticglobals is None: + w_globals = None else: - # an app-level function - self.app2interp(obj) - elif not optional: - # assume a simple, easily wrappable object - self.content[publicname] = obj - # else skip the object if we cannot recognize it - - def exportall(self, d): - """Publish every object from a dict.""" - for name, obj in d.items(): - # ignore names in '_xyz' - if not name.startswith('_') or name.endswith('_'): - self.exportname(name, obj, optional=1) - - def importall(self, d, cls=None): - """Import all app_-level functions as Gateways into a dict. - Also import literals whose name starts with 'app_'.""" - for name, obj in d.items(): - if name.startswith('app_') and name[4:] not in d: - if isinstance(obj, types.FunctionType): - # an app-level function - assert name == obj.func_name - obj = self.app2interp(obj) - else: - pass # assume a simple, easily wrappable object - if cls is None: - d[name[4:]] = obj + # is there another Gateway in staticglobals for which we + # already have a w_globals for this space ? + for value in self.staticglobals.itervalues(): + if isinstance(value, Gateway): + if space in value.functioncache: + # yes, we share its w_globals + fn = value.functioncache[space] + w_globals = fn.w_globals + break else: - setattr(cls, name[4:], obj) - - def makedict(self, space, bind_instance=None): - """Turn the proxy into a normal dict. - Gateways to interpreter-level functions that were defined - with 'implicitself' are bound to 'bind_instance', so that - calling them from app-space will add this extra hidden argument.""" - w_dict = space.newdict([]) - for key, value in self.content.items(): - if isinstance(value, Gateway): - value = value.make_function(space, bind_instance, w_dict) - space.setitem(w_dict, space.wrap(key), space.wrap(value)) - return w_dict + # no, we build all Gateways in the staticglobals now. + w_globals = build_dict(self.staticglobals, space) + return self.build_function(space, w_globals) + + def build_function(self, space, w_globals): + if space in self.functioncache: + fn = self.functioncache[space] + else: + defs_w = [space.wrap(def_value) for def_value in self.staticdefs] + fn = Function(space, self.code, w_globals, defs_w) + self.functioncache[space] = fn + return fn + + +def app2interp(app, app_name=None): + """Build a Gateway that calls 'app' at app-level.""" + # app must be a function whose name starts with 'app_'. + if not isinstance(app, types.FunctionType): + if isinstance(app, Gateway): + return app + raise TypeError, "function expected, got %r instead" % app + if app_name is None: + if not app.func_name.startswith('app_'): + raise ValueError, ("function name must start with 'app_'; " + "%r does not" % app.func_name) + app_name = app.func_name[4:] + code = pycode.PyCode(None) + code._from_code(app.func_code) + staticglobals = app.func_globals + staticdefs = list(app.func_defaults or ()) + return Gateway(app_name, code, staticglobals, staticdefs) + +def interp2app(f, app_name=None): + """Build a Gateway that calls 'f' at interp-level.""" + # f must be a function whose name does NOT starts with 'app_' + if not isinstance(f, types.FunctionType): + if isinstance(f, Gateway): + return f + raise TypeError, "function expected, got %r instead" % f + if app_name is None: + if f.func_name.startswith('app_'): + raise ValueError, ("function name %r suspiciously starts " + "with 'app_'" % f.func_name) + app_name = f.func_name + builtincode = BuiltinCode(f) + staticdefs = list(f.func_defaults or ()) + return Gateway(app_name, builtincode, None, staticdefs) + + +def exportall(d): + """Publish every function from a dict.""" + for name, obj in d.items(): + # ignore names in '_xyz' + if not name.startswith('_') or name.endswith('_'): + if isinstance(obj, types.FunctionType): + if 'app_'+name not in d: + d['app_'+name] = interp2app(obj, name) + +def importall(d): + """Import all app_-level functions as Gateways into a dict.""" + for name, obj in d.items(): + if name.startswith('app_') and name[4:] not in d: + if isinstance(obj, types.FunctionType): + d[name[4:]] = app2interp(obj, name[4:]) + +def build_dict(d, space): + """Search all Gateways and put them into a wrapped dictionary.""" + w_globals = space.newdict([]) + for value in d.itervalues(): + if isinstance(value, Gateway): + fn = value.build_function(space, w_globals) + w_name = space.wrap(value.name) + w_object = space.wrap(fn) + space.setitem(w_globals, w_name, w_object) + return w_globals Modified: pypy/branch/builtinrefactor/pypy/interpreter/generator.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/generator.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/generator.py Wed Sep 17 13:38:03 2003 @@ -2,6 +2,7 @@ from pypy.interpreter.baseobjspace import NoValue from pypy.interpreter.eval import Frame from pypy.interpreter.pyframe import ControlFlowException, ExitFrame +from pypy.interpreter import function, gateway # # Generator support. Note that GeneratorFrame is not a subclass of PyFrame. @@ -18,7 +19,7 @@ def run(self): "Build a generator-iterator." self.exhausted = False - return GeneratorIterator(self) + return self.space.wrap(GeneratorIterator(self)) ### extra opcodes ### @@ -39,10 +40,14 @@ "An iterator created by a generator." def __init__(self, frame): + self.space = frame.space self.frame = frame self.running = False - def nextvalue(self): + def pypy_iter(self): + return self.space.wrap(self) + + def pypy_next(self): # raise NoValue when exhausted if self.running: space = self.frame.space @@ -56,19 +61,31 @@ finally: self.running = False - - # XXX trick for trivialobjspace - # XXX make these __iter__() and next() app-visible - - def __iter__(self): - return self - def next(self): - # XXX trivialobjspace only !! try: - return self.nextvalue() + return self.pypy_next() except NoValue: - raise StopIteration + raise OperationError(self.space.w_StopIteration, + self.space.w_None) + app_next = gateway.interp2app(next) + + def pypy_getattr(self, w_attr): + # XXX boilerplate that should disappear at some point + attr = self.space.unwrap(w_attr) + if attr == 'next': + return self.space.wrap(self.app_next) + raise OperationError(self.space.w_AttributeError, w_attr) + + # XXX the following is for TrivialObjSpace only, when iteration is + # done by C code (e.g. when calling 'list(g())'). + def __iter__(self): + class hack: + def next(h): + try: + return self.pypy_next() + except NoValue: + raise StopIteration + return hack() # # the specific ControlFlowExceptions used by generators Modified: pypy/branch/builtinrefactor/pypy/interpreter/main.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/main.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/main.py Wed Sep 17 13:38:03 2003 @@ -1,6 +1,6 @@ import autopath from pypy.tool import option -from pypy.interpreter import executioncontext, baseobjspace, gateway +from pypy.interpreter import executioncontext, baseobjspace, gateway, module import sys, os def _run_eval_string(source, filename, space, eval): @@ -21,8 +21,8 @@ ec = executioncontext.ExecutionContext(space) - w_mainmodule = space.newmodule(space.wrap("__main__")) - w_globals = space.getattr(w_mainmodule, space.wrap("__dict__")) + mainmodule = module.Module(space, space.wrap("__main__")) + w_globals = mainmodule.w_dict except baseobjspace.OperationError, operationerr: operationerr.record_interpreter_traceback() Modified: pypy/branch/builtinrefactor/pypy/interpreter/miscutils.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/miscutils.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/miscutils.py Wed Sep 17 13:38:03 2003 @@ -51,6 +51,26 @@ self.__initclass__() +class RwDictProxy(object): + """A dict-like class standing for 'cls.__dict__', to work around + the fact that the latter is a read-only proxy for new-style classes.""" + + def __init__(self, cls): + self.cls = cls + + def __getitem__(self, attr): + return self.cls.__dict__[attr] + + def __setitem__(self, attr, value): + setattr(self.cls, attr, value) + + def __contains__(self, value): + return value in self.cls.__dict__ + + def items(self): + return self.cls.__dict__.items() + + class ThreadLocals: """Thread-local storage.""" Added: pypy/branch/builtinrefactor/pypy/interpreter/module.py ============================================================================== --- (empty file) +++ pypy/branch/builtinrefactor/pypy/interpreter/module.py Wed Sep 17 13:38:03 2003 @@ -0,0 +1,30 @@ +""" +Module objects. +""" + +from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter.error import OperationError + + +class Module(Wrappable): + """A module.""" + + def __init__(self, space, w_name): + self.space = space + self.w_dict = space.newdict([(space.wrap('__name__'), w_name)]) + self.w_name = w_name + + def pypy_getattr(self, w_attr): + space = self.space + if space.is_true(space.eq(w_attr, space.wrap('__dict__'))): + return self.w_dict + try: + return space.getitem(self.w_dict, w_attr) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + # XXX fix error message + raise OperationError(space.w_AttributeError, w_attr) + + def pypy_setattr(self, w_attr, w_value): + self.space.setitem(self.w_dict, w_attr, w_value) Modified: pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/nestedscope.py Wed Sep 17 13:38:03 2003 @@ -1,6 +1,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.eval import UNDEFINED from pypy.interpreter.pyopcode import PyInterpFrame +from pypy.interpreter import function, pycode class Cell(object): @@ -157,12 +158,12 @@ def MAKE_CLOSURE(f, numdefaults): w_codeobj = f.valuestack.pop() codeobj = f.space.unwrap(w_codeobj) + assert isinstance(codeobj, pycode.PyCode) nfreevars = len(codeobj.co_freevars) - freevars = [f.valuestack.pop() for i in range(nfreevars)] + freevars = [f.space.unwrap(f.valuestack.pop()) for i in range(nfreevars)] freevars.reverse() defaultarguments = [f.valuestack.pop() for i in range(numdefaults)] defaultarguments.reverse() - w_defaultarguments = f.space.newtuple(defaultarguments) - w_func = f.space.newfunction(f.space.unwrap(w_codeobj), - f.w_globals, w_defaultarguments, freevars) - f.valuestack.push(w_func) + fn = function.Function(f.space, codeobj, f.w_globals, + defaultarguments, freevars) + f.valuestack.push(f.space.wrap(fn)) Modified: pypy/branch/builtinrefactor/pypy/interpreter/py.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/py.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/py.py Wed Sep 17 13:38:03 2003 @@ -3,7 +3,7 @@ import autopath from pypy.tool import option from pypy.tool.optik import make_option -from pypy.interpreter import main, interactive, baseobjspace +from pypy.interpreter import main, interactive, baseobjspace, error import sys class Options(option.Options): @@ -35,12 +35,12 @@ if Options.command: try: main.run_string(Options.command[0], '', space) - except baseobjspace.PyPyError, pypyerr: + except error.PyPyError, pypyerr: pypyerr.operationerr.print_detailed_traceback(pypyerr.space) elif args: try: main.run_file(args[0], space) - except baseobjspace.PyPyError, pypyerr: + except error.PyPyError, pypyerr: pypyerr.operationerr.print_detailed_traceback(pypyerr.space) else: go_interactive = 1 Modified: pypy/branch/builtinrefactor/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/pycode.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/pycode.py Wed Sep 17 13:38:03 2003 @@ -72,7 +72,7 @@ def signature(self): "([list-of-arg-names], vararg-name-or-None, kwarg-name-or-None)." argcount = self.co_argcount - argnames = self.co_varnames[:argcount] + argnames = list(self.co_varnames[:argcount]) if self.co_flags & CO_VARARGS: varargname = self.co_varnames[argcount] argcount += 1 Modified: pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/pyframe.py Wed Sep 17 13:38:03 2003 @@ -1,10 +1,9 @@ """ PyFrame class implementation with the interpreter main loop. """ -from pypy.interpreter import eval, baseobjspace +from pypy.interpreter import eval, baseobjspace, gateway from pypy.interpreter.miscutils import Stack from pypy.interpreter.error import OperationError -from pypy.interpreter.gateway import DictProxy class PyFrame(eval.Frame): @@ -97,6 +96,17 @@ break self.exceptionstack.pop() + ### public attributes ### + + def pypy_getattr(self, w_attr): + # XXX surely not the Right Way to do this + attr = self.space.unwrap(w_attr) + if attr == 'f_locals': return self.w_locals + if attr == 'f_globals': return self.w_globals + if attr == 'f_builtins': return self.w_builtins + if attr == 'f_code': return self.space.wrap(self.code) + raise OperationError(self.space.w_AttributeError, w_attr) + ### Frame Blocks ### @@ -182,7 +192,7 @@ else: raise Exception, "?!" # XXX return etype, evalue -normalize_exception = DictProxy().app2interp(app_normalize_exception) +normalize_exception = gateway.app2interp(app_normalize_exception) class FinallyBlock(FrameBlock): Modified: pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/pyopcode.py Wed Sep 17 13:38:03 2003 @@ -6,9 +6,8 @@ from pypy.interpreter.baseobjspace import OperationError, NoValue from pypy.interpreter.eval import UNDEFINED -from pypy.interpreter import baseobjspace, pyframe +from pypy.interpreter import baseobjspace, pyframe, gateway, function from pypy.interpreter.miscutils import InitializedClass -from pypy.interpreter.gateway import DictProxy class unaryoperation: @@ -34,7 +33,6 @@ class PyInterpFrame(pyframe.PyFrame): """A PyFrame that knows about interpretation of standard Python opcodes minus the ones related to nested scopes.""" - appdict = DictProxy(implicitspace=True) ### opcode dispatch ### @@ -195,20 +193,20 @@ f.valuestack.push(w_result) def SLICE_0(f): - slice(f, None, None) + f.slice(None, None) def SLICE_1(f): w_start = f.valuestack.pop() - slice(f, w_start, None) + f.slice(w_start, None) def SLICE_2(f): w_end = f.valuestack.pop() - slice(f, None, w_end) + f.slice(None, w_end) def SLICE_3(f): w_end = f.valuestack.pop() w_start = f.valuestack.pop() - slice(f, w_start, w_end) + f.slice(w_start, w_end) def storeslice(f, w_start, w_end): w_slice = f.space.newslice(w_start, w_end, None) @@ -217,20 +215,20 @@ f.space.setitem(w_obj, w_slice, w_newvalue) def STORE_SLICE_0(f): - storeslice(f, None, None) + f.storeslice(None, None) def STORE_SLICE_1(f): w_start = f.valuestack.pop() - storeslice(f, w_start, None) + f.storeslice(w_start, None) def STORE_SLICE_2(f): w_end = f.valuestack.pop() - storeslice(f, None, w_end) + f.storeslice(None, w_end) def STORE_SLICE_3(f): w_end = f.valuestack.pop() w_start = f.valuestack.pop() - storeslice(f, w_start, w_end) + f.storeslice(w_start, w_end) def deleteslice(f, w_start, w_end): w_slice = f.space.newslice(w_start, w_end, None) @@ -238,20 +236,20 @@ f.space.delitem(w_obj, w_slice) def DELETE_SLICE_0(f): - deleteslice(f, None, None) + f.deleteslice(None, None) def DELETE_SLICE_1(f): w_start = f.valuestack.pop() - deleteslice(f, w_start, None) + f.deleteslice(w_start, None) def DELETE_SLICE_2(f): w_end = f.valuestack.pop() - deleteslice(f, None, w_end) + f.deleteslice(None, w_end) def DELETE_SLICE_3(f): w_end = f.valuestack.pop() w_start = f.valuestack.pop() - deleteslice(f, w_start, w_end) + f.deleteslice(w_start, w_end) def STORE_SUBSCR(f): "obj[subscr] = newvalue" @@ -339,15 +337,15 @@ locals = prog[2] prog = prog[0] if globals is None: - globals = f.w_globals # XXX should be f.f_globals + globals = f.f_globals if locals is None: - locals = f.w_locals # XXX should be f.f_locals + locals = f.f_locals if locals is None: locals = globals if not isinstance(globals, dict): raise TypeError("exec: arg 2 must be a dictionary or None") elif not globals.has_key('__builtins__'): - globals['__builtins__'] = f.w_builtins # XXX should be f.f_builtins + globals['__builtins__'] = f.f_builtins if not isinstance(locals, dict): raise TypeError("exec: arg 3 must be a dictionary or None") # XXX - HACK to check for code object @@ -592,9 +590,9 @@ def IMPORT_STAR(f): w_module = f.valuestack.pop() - w_locals = f.getfastscope() + w_locals = f.getdictscope() import_all_from(f.space, w_module, w_locals) - f.setfastscope(w_locals) + f.setdictscope(w_locals) def IMPORT_FROM(f, nameindex): name = f.getname(nameindex) @@ -703,12 +701,11 @@ def MAKE_FUNCTION(f, numdefaults): w_codeobj = f.valuestack.pop() + codeobj = f.space.unwrap(w_codeobj) defaultarguments = [f.valuestack.pop() for i in range(numdefaults)] defaultarguments.reverse() - w_defaultarguments = f.space.newtuple(defaultarguments) - w_func = f.space.newfunction(f.space.unwrap(w_codeobj), - f.w_globals, w_defaultarguments) - f.valuestack.push(w_func) + fn = function.Function(f.space, codeobj, f.w_globals, defaultarguments) + f.valuestack.push(f.space.wrap(fn)) def BUILD_SLICE(f, numargs): if numargs == 3: @@ -759,7 +756,7 @@ cls.dispatch_table = dispatch_table - appdict.importall(locals()) + gateway.importall(locals()) # app_xxx() -> xxx() ### helpers written at the application-level ### @@ -885,5 +882,4 @@ raise ImportError("cannot import name '%s'" % name) -appdict = DictProxy() -appdict.importall(globals()) # app_xxx() -> xxx() +gateway.importall(globals()) # app_xxx() -> xxx() Modified: pypy/branch/builtinrefactor/pypy/interpreter/test/test_interpreter.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/test/test_interpreter.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/test/test_interpreter.py Wed Sep 17 13:38:03 2003 @@ -7,7 +7,8 @@ def codetest(self, source, functionname, args): """Compile and run the given code string, and then call its function named by 'functionname' with arguments 'args'.""" - from pypy.interpreter import baseobjspace, executioncontext, pyframe, gateway + from pypy.interpreter import baseobjspace, executioncontext + from pypy.interpreter import pyframe, gateway, module space = self.space compile = space.builtin.compile @@ -16,8 +17,8 @@ ec = executioncontext.ExecutionContext(space) - w_tempmodule = space.newmodule(w("__temp__")) - w_glob = space.getattr(w_tempmodule, w("__dict__")) + tempmodule = module.Module(space, w("__temp__")) + w_glob = tempmodule.w_dict space.setitem(w_glob, w("__builtins__"), space.w_builtins) code = space.unwrap(w_code) @@ -260,6 +261,12 @@ yield 1 g = f() self.assertEquals(list(g), [1]) + + def test_generator4(self): + def f(): + yield 1 + g = f() + self.assertEquals([x for x in g], [1]) def test_generator_restart(self): def g(): Modified: pypy/branch/builtinrefactor/pypy/interpreter/test/test_main.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/test/test_main.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/test/test_main.py Wed Sep 17 13:38:03 2003 @@ -21,7 +21,7 @@ def checkoutput(expected_output,f,*args): space = test.objspace() - w_sys = space.get_builtin_module(space.wrap("sys")) + w_sys = space.get_builtin_module("sys") w_oldout = space.getattr(w_sys, space.wrap("stdout")) capture.reset() space.setattr(w_sys, space.wrap("stdout"), space.wrap(capture)) Modified: pypy/branch/builtinrefactor/pypy/interpreter/test/test_objspace.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/test/test_objspace.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/test/test_objspace.py Wed Sep 17 13:38:03 2003 @@ -58,23 +58,5 @@ self.failUnless(self.space.is_true(w_true)) self.failIf(self.space.is_true(w_false)) - def test_newmodule(self): - # a shoddy test: - w_mod = self.space.newmodule(self.space.wrap("mod")) - - def test_newfunction(self): - # this tests FAR more than newfunction, but that's probably - # unavoidable - def f(x): - return x - - from pypy.interpreter.pycode import PyCode - cpycode = PyCode()._from_code(f.func_code) - w_globals = self.space.newdict([]) - w_defs = self.space.newtuple([]) - w_f = self.space.newfunction(cpycode, w_globals, w_defs) - self.assertEqual_w(self.space.call_function(w_f, self.space.wrap(1)), - self.space.wrap(1)) - if __name__ == '__main__': test.main() Modified: pypy/branch/builtinrefactor/pypy/interpreter/unittest_w.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/unittest_w.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/unittest_w.py Wed Sep 17 13:38:03 2003 @@ -2,23 +2,22 @@ import sys, os import unittest -from pypy.interpreter.gateway import DictProxy +from pypy.interpreter import gateway def make_testcase_class(space, tc_w): # XXX this is all a bit insane (but it works) - # space-independent part: collect the test methods into - # a DictProxy. - d = DictProxy(implicitspace=True) + # collect the test methods into a dictionary + w = space.wrap + w_dict = space.newdict([]) for name in dir(AppTestCase): if ( name.startswith('assert') or name.startswith('fail') and name != 'failureException'): - d.app2interp(getattr(tc_w, name).im_func, name) + fn = gateway.app2interp(getattr(tc_w, name).im_func, name) + space.setitem(w_dict, w(name), w(fn)) # space-dependent part: make an object-space-level dictionary # and use it to build the class. - w = space.wrap - w_dict = d.makedict(space) space.setitem(w_dict, w('failureException'), space.w_AssertionError) w_tc = space.call_function(space.w_type, w('TestCase'), @@ -44,10 +43,8 @@ setattr(space, w_tc_attr, w_tc) f = self.testMethod.im_func - gateway = DictProxy(implicitspace=True).app2interp(f, f.func_name) - w_f = space.wrap(gateway) - # w_f = wrap_func(space, self.testMethod.im_func) - space.call_function(w_f, w_tc) + gway = gateway.app2interp(f, f.func_name) + gway(space, w_tc) class IntTestCase(unittest.TestCase): Modified: pypy/branch/builtinrefactor/pypy/module/builtin.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/module/builtin.py (original) +++ pypy/branch/builtinrefactor/pypy/module/builtin.py Wed Sep 17 13:38:03 2003 @@ -1,5 +1,6 @@ from __future__ import generators from pypy.interpreter import executioncontext +from pypy.interpreter.module import Module from pypy.interpreter.extmodule import ExtModule from pypy.interpreter.error import OperationError @@ -13,7 +14,10 @@ """ Template for PyPy's '__builtin__' module. """ - app___name__ = '__builtin__' + __name__ = '__builtin__' + + open = cpy_builtin.open + file = cpy_builtin.file def _actframe(self, index=-1): return self.space.getexecutioncontext().framestack.items[index] @@ -33,7 +37,7 @@ except OperationError,e: if not e.match(space, space.w_KeyError): raise - w_mod = space.get_builtin_module(w_modulename) + w_mod = space.get_builtin_module(space.unwrap(w_modulename)) if w_mod is not None: space.setitem(space.sys.w_modules, w_modulename, w_mod) return w_mod @@ -42,11 +46,10 @@ for path in space.sys.path: f = os.path.join(path, space.unwrap(w_modulename) + '.py') if os.path.exists(f): - w_mod = space.newmodule(w_modulename) + w_mod = space.wrap(Module(space, w_modulename)) space.setitem(space.sys.w_modules, w_modulename, w_mod) space.setattr(w_mod, w('__file__'), w(f)) w_dict = space.getattr(w_mod, w('__dict__')) - import sys; print >> sys.stderr, self.__class__.__dict__['execfile'] self.execfile(w(f), w_dict, w_dict) return w_mod @@ -77,7 +80,7 @@ from pypy.interpreter.pycode import PyCode return space.wrap(PyCode()._from_code(c)) - def app_execfile(filename, glob=None, loc=None): + def app_execfile(self, filename, glob=None, loc=None): if glob is None: glob = globals() if loc is None: @@ -141,11 +144,11 @@ # app-level functions - def app_apply(function, args, kwds={}): + def app_apply(self, function, args, kwds={}): """call a function (or other callable object) and return its result""" return function(*args, **kwds) - def app_map(function, *collections): + def app_map(self, function, *collections): """does 3 separate things, hence this enormous docstring. 1. if function is None, return a list of tuples, each with one item from each collection. If the collections have different @@ -191,7 +194,7 @@ return res idx = idx + 1 - def app_filter(function, collection): + def app_filter(self, function, collection): """construct a list of those elements of collection for which function is True. If function is None, then return the items in the sequence which are True.""" @@ -208,7 +211,7 @@ else: return res - def app_zip(*collections): + def app_zip(self, *collections): """return a list of tuples, where the nth tuple contains every nth item of each collection. If the collections have different lengths, zip returns a list as long as the shortest collection, @@ -229,7 +232,7 @@ idx = idx + 1 return res - def app_reduce(function, l, *initialt): + def app_reduce(self, function, l, *initialt): """ Apply function of two arguments cumulatively to the items of sequence, from left to right, so as to reduce the sequence to a single value. Optionally begin with an initial value.""" @@ -251,7 +254,7 @@ break return initial - def app_isinstance(obj, klass_or_tuple): + def app_isinstance(self, obj, klass_or_tuple): objcls = obj.__class__ if issubclass(klass_or_tuple.__class__, tuple): for klass in klass_or_tuple: @@ -264,7 +267,7 @@ except TypeError: raise TypeError, "isinstance() arg 2 must be a class or type" - def app_range(x, y=None, step=1): + def app_range(self, x, y=None, step=1): """ returns a list of integers in arithmetic position from start (defaults to zero) to stop - 1 by step (defaults to 1). Use a negative step to get a list in decending order.""" @@ -303,7 +306,7 @@ # min and max could be one function if we had operator.__gt__ and # operator.__lt__ Perhaps later when we have operator. - def app_min(*arr): + def app_min(self, *arr): """return the smallest number in a list""" if not arr: @@ -323,7 +326,7 @@ min = i return min - def app_max(*arr): + def app_max(self, *arr): """return the largest number in a list""" if not arr: @@ -344,7 +347,7 @@ return max - def app_cmp(x, y): + def app_cmp(self, x, y): """return 0 when x == y, -1 when x < y and 1 when x > y """ if x < y: return -1 @@ -353,7 +356,7 @@ else: return 1 - def app_vars(*obj): + def app_vars(self, *obj): """return a dictionary of all the attributes currently bound in obj. If called with no argument, return the variables bound in local scope.""" @@ -367,7 +370,7 @@ except AttributeError: raise TypeError, "vars() argument must have __dict__ attribute" - def app_hasattr(ob, attr): + def app_hasattr(self, ob, attr): try: getattr(ob, attr) return True @@ -375,7 +378,7 @@ return False - def app_xrange(start, stop=None, step=1): + def app_xrange(self, start, stop=None, step=1): class xrange: def __init__(self, start, stop=None, step=1): if stop is None: Modified: pypy/branch/builtinrefactor/pypy/module/sysmodule.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/module/sysmodule.py (original) +++ pypy/branch/builtinrefactor/pypy/module/sysmodule.py Wed Sep 17 13:38:03 2003 @@ -7,10 +7,10 @@ class Sys(ExtModule): """ Template for PyPy's 'sys' module. - Currently we only provide 'path', 'modules', 'stdout' and 'displayhook' + Currently we only provide a handful of attributes. """ - app___name__ = 'sys' + __name__ = 'sys' def __init__(self, space): opd = os.path.dirname @@ -22,6 +22,7 @@ stdout = cpy_sys.stdout stderr = cpy_sys.stderr + maxint = cpy_sys.maxint def _setmodule(self, w_module): """ put a module into the modules list """ @@ -31,7 +32,7 @@ def displayhook(self, w_x): space = self.space w = space.wrap - if w_x != space.w_None: + if not space.is_true(space.is_(w_x, space.w_None)): try: print space.unwrap(self.space.repr(w_x)) except OperationError: Modified: pypy/branch/builtinrefactor/pypy/module/test/test_builtin.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/module/test/test_builtin.py (original) +++ pypy/branch/builtinrefactor/pypy/module/test/test_builtin.py Wed Sep 17 13:38:03 2003 @@ -7,10 +7,10 @@ self.space = test.objspace() def test_import(self): - import types d = {} - m = __import__('quopri', d, d, []) - self.assertEquals(type(m), types.ModuleType) + m = __import__('types', d, d, []) + self.assertEquals(m.IntType, type(123)) + self.assertEquals(m.__name__, "types") def test_chr(self): self.assertEquals(chr(65), 'A') Modified: pypy/branch/builtinrefactor/pypy/module/test/test_sysmodule.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/module/test/test_sysmodule.py (original) +++ pypy/branch/builtinrefactor/pypy/module/test/test_sysmodule.py Wed Sep 17 13:38:03 2003 @@ -4,7 +4,7 @@ class SysTests(test.TestCase): def setUp(self): self.space = test.objspace() - self.sys_w = self.space.get_builtin_module(self.space.wrap("sys")) + self.sys_w = self.space.get_builtin_module("sys") def tearDown(self): pass Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/classobject_app.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/classobject_app.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,16 +0,0 @@ - - -class ClassType(object): - __slots__ = ['....'] - - def __init__(self, name, bases, dict): - self.__name__ = name - self.__bases__ = bases - self.__dict__ = dict - - def __getattr__(self, attr): - try: - return self.__dict__[attr] - except KeyError: - raise AttributeError, "Class %s has no attribute %s" % ( - self.__name__, attr) Modified: pypy/branch/builtinrefactor/pypy/objspace/std/cpythonobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/cpythonobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/cpythonobject.py Wed Sep 17 13:38:03 2003 @@ -37,7 +37,9 @@ # real-to-wrapped exceptions def wrap_exception(space): exc, value, tb = sys.exc_info() - raise OperationError(space.wrap(exc), space.wrap(value)) + if exc is OperationError: + raise exc, value, tb # just re-raise it + raise OperationError, OperationError(space.wrap(exc), space.wrap(value)), tb # in-place operators def inplace_pow(x1, x2): @@ -81,11 +83,17 @@ x1 ^= x2 return x1 +def getter(o, i, c): + if hasattr(o, '__get__'): + return o.__get__(i, c) + else: + return o + # regular part of the interface (minus 'next' and 'call') MethodImplementation = { 'id': id, 'type': type, -# 'issubtype': issubclass, +# 'issubtype': see below, 'repr': repr, 'str': str, 'len': len, @@ -93,10 +101,16 @@ 'getattr': getattr, 'setattr': setattr, 'delattr': delattr, +# 'getitem': see below, +# 'setitem': see below, +# 'delitem': see below, 'pos': operator.pos, 'neg': operator.neg, 'not_': operator.not_, 'abs': operator.abs, + 'hex': hex, + 'oct': oct, + 'ord': ord, 'invert': operator.invert, 'add': operator.add, 'sub': operator.sub, @@ -112,6 +126,8 @@ 'and_': operator.and_, 'or_': operator.or_, 'xor': operator.xor, + 'int': int, + 'float': float, 'inplace_add': inplace_add, 'inplace_sub': inplace_sub, 'inplace_mul': inplace_mul, @@ -133,22 +149,29 @@ 'ge': operator.ge, 'contains': operator.contains, 'iter': iter, + 'get': getter, +# 'set': setter, +# 'delete': deleter, } for _name, _symbol, _arity, _specialnames in ObjSpace.MethodTable: f = MethodImplementation.get(_name) if f: if _arity == 1: - def cpython_f(space, w_1, f=f): + def cpython_f(space, w_1, f=f, pypymethod='pypy_'+_name): x = space.unwrap(w_1) + if hasattr(x, pypymethod): + return getattr(x, pypymethod)() try: y = f(x) except: wrap_exception(space) return space.wrap(y) elif _arity == 2: - def cpython_f(space, w_1, w_2, f=f): + def cpython_f(space, w_1, w_2, f=f, pypymethod='pypy_'+_name): x1 = space.unwrap(w_1) + if hasattr(x1, pypymethod): + return getattr(x1, pypymethod)(w_2) x2 = space.unwrap(w_2) try: y = f(x1, x2) @@ -156,8 +179,10 @@ wrap_exception(space) return space.wrap(y) elif _arity == 3: - def cpython_f(space, w_1, w_2, w_3, f=f): + def cpython_f(space, w_1, w_2, w_3, f=f, pypymethod='pypy_'+_name): x1 = space.unwrap(w_1) + if hasattr(x1, pypymethod): + return getattr(x1, pypymethod)(w_2, w_3) x2 = space.unwrap(w_2) x3 = space.unwrap(w_3) try: @@ -234,6 +259,8 @@ def next__CPython(space, w_obj): obj = space.unwrap(w_obj) + if hasattr(obj, 'pypy_next'): + return obj.pypy_next() try: result = obj.next() except StopIteration: @@ -245,6 +272,8 @@ def call__CPython_ANY_ANY(space, w_obj, w_arguments, w_keywords): # XXX temporary hack similar to objspace.trivial.call() callable = space.unwrap(w_obj) + if hasattr(callable, 'pypy_call'): + return callable.pypy_call(w_arguments, w_keywords) args = space.unwrap(w_arguments) keywords = space.unwrap(w_keywords) try: Modified: pypy/branch/builtinrefactor/pypy/objspace/std/dictobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/dictobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/dictobject.py Wed Sep 17 13:38:03 2003 @@ -6,11 +6,8 @@ """ from pypy.objspace.std.objspace import * -from dicttype import W_DictType, _no_object +from dicttype import W_DictType from stringobject import W_StringObject -from pypy.interpreter.extmodule import make_builtin_func - -applicationfile = StdObjSpace.AppFile(__name__) class _NoValueInCell: pass @@ -106,7 +103,7 @@ else: raise OperationError(space.w_TypeError, space.wrap("dict() takes at most 1 argument")) - dict_update__Dict_Dict(space, w_dict, w_kwds) + space.call_method(w_dict, 'update', w_kwds) def getitem__Dict_ANY(space, w_dict, w_lookup): data = w_dict.non_empties() @@ -144,9 +141,11 @@ return iterobject.W_SeqIterObject(space, w_keys) def eq__Dict_Dict(space, w_left, w_right): - if len(w_left.data) != len(w_right.data): + dataleft = w_left.non_empties() + dataright = w_right.non_empties() + if len(dataleft) != len(dataright): return space.w_False - for w_key, cell in w_left.non_empties(): + for w_key, cell in dataleft: try: w_rightval = space.getitem(w_right, w_key) except OperationError: @@ -157,13 +156,15 @@ def lt__Dict_Dict(space, w_left, w_right): # Different sizes, no problem - if len(w_left.data) < len(w_right.data): + dataleft = w_left.non_empties() + dataright = w_right.non_empties() + if len(dataleft) < len(dataright): return space.w_True - if len(w_left.data) > len(w_right.data): + if len(dataleft) > len(dataright): return space.w_False # Same size - for w_key, cell in w_left.non_empties(): + for w_key, cell in dataleft: # This is incorrect, but we need to decide what comparisons on # dictionaries of equal size actually means # The Python language specification is silent on the subject @@ -176,28 +177,6 @@ # The dictionaries are equal. This is correct. return space.w_False - -def gt__Dict_Dict(space, w_left, w_right): - # Different sizes, no problem - if len(w_left.data) > len(w_right.data): - return space.w_True - if len(w_left.data) < len(w_right.data): - return space.w_False - - # Same size - for w_key, cell in w_left.non_empties(): - # This is incorrect, but we need to decide what comparisons on - # dictionaries of equal size actually means - # The Python language specification is silent on the subject - try: - w_rightval = space.getitem(w_right, w_key) - except OperationError: - return space.w_True - if space.is_true(space.gt(cell.w_value, w_rightval)): - return space.w_True - # The dictionaries are equal. This is correct. - return space.w_False - def dict_copy__Dict(space, w_self): return W_DictObject(space, [(w_key,cell.get()) for w_key,cell in @@ -229,13 +208,6 @@ def dict_clear__Dict(space, w_self): w_self.data = [] -def dict_update__Dict_Dict(space, w_self, w_other): - w_self.space.gethelper(applicationfile).call("dict_update", [w_self, w_other]) - -def dict_popitem__Dict(space, w_self): - w_item = w_self.space.gethelper(applicationfile).call("dict_popitem", [w_self]) - return w_item - def dict_get__Dict_ANY_ANY(space, w_self, w_lookup, w_default): data = w_self.non_empties() for w_key, cell in data: @@ -243,28 +215,4 @@ return cell.get() return w_default -def dict_setdefault__Dict_ANY_ANY(space, w_self, w_key, w_default): - w_value = w_self.space.gethelper(applicationfile).call("dict_setdefault", [w_self, w_key, w_default]) - return w_value - -def dict_pop__Dict_ANY_ANY(space, w_self, w_key, w_default): - default = space.unwrap(w_default) - if default is _no_object: - w_value = w_self.space.gethelper(applicationfile).call("dict_pop_no_default", [w_self, w_key]) - else: - w_value = w_self.space.gethelper(applicationfile).call("dict_pop_with_default", [w_self, w_key, w_default]) - return w_value - -def dict_iteritems__Dict(space, w_self): - w_item = w_self.space.gethelper(applicationfile).call("dict_iteritems", [w_self]) - return w_item - -def dict_iterkeys__Dict(space, w_self): - w_item = w_self.space.gethelper(applicationfile).call("dict_iterkeys", [w_self]) - return w_item - -def dict_itervalues__Dict(space, w_self): - w_item = w_self.space.gethelper(applicationfile).call("dict_itervalues", [w_self]) - return w_item - register_all(vars(), W_DictType) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/dictobject_app.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/dictobject_app.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,45 +0,0 @@ - - -def dict_update(d,o): - for k in o.keys(): - d[k] = o[k] - -def dict_popitem(d): - k = d.keys()[0] - v = d[k] - del d[k] - return k, v - -def dict_get(d, k, v=None): - if d.has_key(k): - return d[k] - return v - -def dict_setdefault(d, k, v): - if d.has_key(k): - return d[k] - d[k] = v - return v - -def dict_pop_no_default(d, k): - v = d[k] - del d[k] - return v - -def dict_pop_with_default(d, k, v): - try: - v = d[k] - del d[k] - except KeyError: - pass - return v - -def dict_iteritems(d): - return iter(d.items()) - -def dict_iterkeys(d): - return iter(d.keys()) - -def dict_itervalues(d): - return iter(d.values()) - Modified: pypy/branch/builtinrefactor/pypy/objspace/std/dicttype.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/dicttype.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/dicttype.py Wed Sep 17 13:38:03 2003 @@ -3,10 +3,9 @@ """ from pypy.objspace.std.objspace import * +from pypy.interpreter import gateway from typeobject import W_TypeObject -class _no_object: pass - class W_DictType(W_TypeObject): typename = 'dict' @@ -18,7 +17,7 @@ dict_has_key = MultiMethod('has_key', 2) dict_clear = MultiMethod('clear', 1) dict_get = MultiMethod('get', 3, defaults=(None,)) - dict_pop = MultiMethod('pop', 3, defaults=(_no_object,)) + dict_pop = MultiMethod('pop', 2, varargs=True) dict_popitem = MultiMethod('popitem', 1) dict_setdefault = MultiMethod('setdefault', 3) dict_update = MultiMethod('update', 2) @@ -32,4 +31,53 @@ def type_new__DictType_DictType_ANY_ANY(space, w_basetype, w_dicttype, w_args, w_kwds): return space.newdict([]), True -register_all(vars()) + +# default application-level implementations for some operations + +def app_dict_update__ANY_ANY(d, o): + for k in o.keys(): + d[k] = o[k] + +def app_dict_popitem__ANY(d): + k = d.keys()[0] + v = d[k] + del d[k] + return k, v + +def app_dict_get__ANY_ANY_ANY(d, k, v=None): + if d.has_key(k): + return d[k] + return v + +def app_dict_setdefault__ANY_ANY_ANY(d, k, v): + if d.has_key(k): + return d[k] + d[k] = v + return v + +def app_dict_pop__ANY_ANY(d, k, default): + if len(default) > 1: + raise TypeError, "pop expected at most 2 arguments, got %d" % ( + 1 + len(default)) + try: + v = d[k] + del d[k] + except KeyError, e: + if default: + return default[0] + else: + raise e + return v + +def app_dict_iteritems__ANY(d): + return iter(d.items()) + +def app_dict_iterkeys__ANY(d): + return iter(d.keys()) + +def app_dict_itervalues__ANY(d): + return iter(d.values()) + + +gateway.importall(globals()) +register_all(vars(), W_DictType) Modified: pypy/branch/builtinrefactor/pypy/objspace/std/floatobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/floatobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/floatobject.py Wed Sep 17 13:38:03 2003 @@ -10,8 +10,6 @@ import math from intobject import W_IntObject -applicationfile = StdObjSpace.AppFile(__name__) - class W_FloatObject(W_Object): """This is a reimplementation of the CPython "PyFloatObject" it is assumed that the constructor takes a real Python float as @@ -188,7 +186,7 @@ return W_FloatObject(space, w_float.floatval) def abs__Float(space, w_float): - return W_FloatObject(space, fabs(w_float.floatval)) + return W_FloatObject(space, abs(w_float.floatval)) def is_true__Float(space, w_float): return w_float.floatval != 0.0 Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/floatobject_app.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/floatobject_app.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,10 +0,0 @@ -from math import * - -def float_fmod(v,w): - return fmod(v,w) - -def float_floor(v): - return floor(v) - -def float_fabs(v): - return fabs(v) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/funcobject.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/funcobject.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,59 +0,0 @@ -from __future__ import nested_scopes -from pypy.objspace.std.objspace import * -from functype import W_FuncType -import pypy.interpreter.pyframe -from pypy.objspace.std.instmethobject import W_InstMethObject - - -class W_FuncObject(W_Object): - statictype = W_FuncType - - def __init__(w_self, space, code, w_globals, w_defaultarguments, w_closure): - W_Object.__init__(w_self, space) - w_self.code = code - w_self.w_globals = w_globals - w_self.w_defaultarguments = w_defaultarguments - w_self.w_closure = w_closure - - -registerimplementation(W_FuncObject) - - -def unwrap__Func(space, w_function): - # XXX this is probably a temporary hack - def proxy_function(*args, **kw): - w_arguments = space.wrap(args) - w_keywords = space.wrap(kw) - w_result = call__Func_ANY_ANY(space, w_function, w_arguments, w_keywords) - return space.unwrap(w_result) - # XXX no closure implemented - return proxy_function - -CO_GENERATOR = 0x0020 - -def call__Func_ANY_ANY(space, w_function, w_arguments, w_keywords): - somecode = w_function.code - w_globals = w_function.w_globals - w_locals = somecode.build_arguments(space, w_arguments, w_keywords, - w_defaults = w_function.w_defaultarguments, - w_closure = w_function.w_closure) - if somecode.co_flags & CO_GENERATOR: - from generatorobject import W_GeneratorObject - from pypy.interpreter import pyframe - frame = pyframe.PyFrame() - frame.initialize(space, somecode, w_globals, w_locals) - w_ret = W_GeneratorObject(space, frame) - else: - w_ret = somecode.eval_code(space, w_globals, w_locals) - return w_ret - -def get__Func_ANY_ANY(space, w_function, w_instance, w_cls): - return W_InstMethObject(space, w_function, w_instance, w_cls) - -def getattr__Func_ANY(space, w_function, w_attr): - if space.is_true(space.eq(w_attr, space.wrap('func_code'))): - return space.wrap(w_function.code) - else: - raise FailedToImplement - -register_all(vars()) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/functype.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/functype.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,9 +0,0 @@ -from pypy.objspace.std.objspace import * -from typeobject import W_TypeObject - - -class W_FuncType(W_TypeObject): - - typename = 'FunctionType' - -registerimplementation(W_FuncType) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/generatorobject.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/generatorobject.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,31 +0,0 @@ -from pypy.objspace.std.objspace import * -from generatortype import W_GeneratorType - -class W_GeneratorObject(W_Object): - statictype = W_GeneratorType - - def __init__(self, space, frame): - self.frame = frame - self.running = 0 - -registerimplementation(W_GeneratorObject) - -def next__Generator(space, w_gen): - if w_gen.running: - raise OperationError(space.w_ValueError, - space.wrap("generator already executing")) - ec = space.getexecutioncontext() - - w_gen.running = 1 - try: - w_ret = ec.eval_frame(w_gen.frame) - finally: - w_gen.running = 0 - - return w_ret - - -def iter__Generator(space, w_gen): - return w_gen - -register_all(vars()) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/generatortype.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/generatortype.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,9 +0,0 @@ -from pypy.objspace.std.objspace import * -from typeobject import W_TypeObject - - -class W_GeneratorType(W_TypeObject): - - typename = 'GeneratorType' - -registerimplementation(W_GeneratorType) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/instmethobject.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/instmethobject.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,53 +0,0 @@ -""" -Reviewed 03-06-21 -This object should implement both bound and unbound methods. -Currently, the bound methods work: -__call__ tested, OK. - -Unbound methods do not work yet (test is broken) at least in -application space. Changing this module to make them work -would be easy: test in call__InstMeth_ANY_ANY if w_instmeth -is None; if so, insert nothing in the arguments, but rather -perform a typetest on the first argument. However, this would -not be testable until getattr on a typeobject will return an -unbound-method, which, so far, it doesn't yet. -""" - -from __future__ import nested_scopes -from pypy.objspace.std.objspace import * -from instmethtype import W_InstMethType - - -class W_InstMethObject(W_Object): - statictype = W_InstMethType - - def __init__(w_self, space, w_im_func, w_im_self, w_im_class): - W_Object.__init__(w_self, space) - w_self.w_im_self = w_im_self - w_self.w_im_func = w_im_func - w_self.w_im_class = w_im_class - - -registerimplementation(W_InstMethObject) - -def call__InstMeth_ANY_ANY(space, w_instmeth, w_args, w_keywords): - if w_instmeth.w_im_self == space.w_Null: - if space.is_true(space.eq(space.len(w_args), space.wrap(0))): - raise OperationError(space.w_TypeError, space.wrap('bleargh')) - w_self = space.getitem(w_args, space.wrap(0)) - w_selftype = space.type(w_self) - w_issubtype = space.issubtype(w_selftype, w_instmeth.w_im_class) - if not space.is_true(w_issubtype): - raise OperationError(space.w_TypeError, - space.wrap("unbound method %s() must be " - "called with %s instance as first " - "argument (got %s instance instead)")) - else: - w_args = space.add(space.newtuple([w_instmeth.w_im_self]), - w_args) - w_ret = space.call(w_instmeth.w_im_func, w_args, w_keywords) - return w_ret - -# XXX do __get__ for instance methods - -register_all(vars()) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/instmethtype.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/instmethtype.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,13 +0,0 @@ -""" -Reviewed 03-06-21 -""" - -from pypy.objspace.std.objspace import * -from typeobject import W_TypeObject - - -class W_InstMethType(W_TypeObject): - - typename = 'MethodType' - -registerimplementation(W_InstMethType) Modified: pypy/branch/builtinrefactor/pypy/objspace/std/intobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/intobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/intobject.py Wed Sep 17 13:38:03 2003 @@ -3,8 +3,6 @@ from noneobject import W_NoneObject from restricted_int import r_int, LONG_BIT -applicationfile = StdObjSpace.AppFile(__name__) - """ The implementation of integers is a bit difficult, since integers are currently undergoing the change to turn Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/intobject_app.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/intobject_app.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1 +0,0 @@ -#empty Modified: pypy/branch/builtinrefactor/pypy/objspace/std/listobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/listobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/listobject.py Wed Sep 17 13:38:03 2003 @@ -2,7 +2,8 @@ from listtype import W_ListType from intobject import W_IntObject from sliceobject import W_SliceObject -from pypy.interpreter.extmodule import make_builtin_func +import slicetype +from pypy.interpreter import gateway from restricted_int import r_int, r_uint @@ -73,11 +74,8 @@ def getitem__List_Slice(space, w_list, w_slice): items = w_list.ob_item - w_length = space.wrap(w_list.ob_size) - w_start, w_stop, w_step, w_slicelength = w_slice.indices(w_length) - start = space.unwrap(w_start) - step = space.unwrap(w_step) - slicelength = space.unwrap(w_slicelength) + length = w_list.ob_size + start, stop, step, slicelength = slicetype.indices4(space, w_slice, length) assert slicelength >= 0 w_res = W_ListObject(space, []) _list_resize(w_res, slicelength) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/moduleobject.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/moduleobject.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,62 +0,0 @@ -""" -state of W_ModuleObject - -- old style registration - -getattr ok,tested -setattr ok,tested -delattr ok,tested -""" -from pypy.objspace.std.objspace import * -from moduletype import W_ModuleType -from dictobject import W_DictObject - - -class W_ModuleObject(W_Object): - statictype = W_ModuleType - - def __init__(w_self, space, w_name): - W_Object.__init__(w_self, space) - w_key_name = space.wrap('__name__') - w_key_doc = space.wrap('__doc__') - items = [(w_key_name, w_name), - (w_key_doc, space.w_None)] - w_self.w_dict = W_DictObject(w_self.space, items) - -registerimplementation(W_ModuleObject) - -def getattr_module_any(space, w_module, w_attr): - if space.is_true(space.eq(w_attr, space.wrap('__dict__'))): - return w_module.w_dict - else: - try: - return space.getitem(w_module.w_dict, w_attr) - except OperationError, e: - if e.match(space, space.w_KeyError): - raise FailedToImplement(space.w_AttributeError) - else: - raise - -def setattr_module_any_any(space, w_module, w_attr, w_value): - if space.is_true(space.eq(w_attr, space.wrap('__dict__'))): - raise OperationError(space.w_TypeError, - space.wrap("readonly attribute")) - else: - space.setitem(w_module.w_dict, w_attr, w_value) - -def delattr_module_any(space, w_module, w_attr): - if space.is_true(space.eq(w_attr, space.wrap('__dict__'))): - raise OperationError(space.w_TypeError, - space.wrap("readonly attribute")) - else: - try: - space.delitem(w_module.w_dict, w_attr) - except OperationError, e: - if e.match(space, space.w_KeyError): - raise FailedToImplement(space.w_AttributeError) - else: - raise - -StdObjSpace.getattr.register(getattr_module_any, W_ModuleObject, W_ANY) -StdObjSpace.setattr.register(setattr_module_any_any, W_ModuleObject, W_ANY,W_ANY) -StdObjSpace.delattr.register(delattr_module_any, W_ModuleObject, W_ANY) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/moduletype.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/moduletype.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,9 +0,0 @@ -from pypy.objspace.std.objspace import * -from typeobject import W_TypeObject - - -class W_ModuleType(W_TypeObject): - - typename = 'module' - -registerimplementation(W_ModuleType) Modified: pypy/branch/builtinrefactor/pypy/objspace/std/multimethod.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/multimethod.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/multimethod.py Wed Sep 17 13:38:03 2003 @@ -203,7 +203,7 @@ class BoundMultiMethod: - #ASSERT_BASE_TYPE = None + ASSERT_BASE_TYPE = None def __init__(self, space, multimethod): self.space = space @@ -237,11 +237,14 @@ arity = self.multimethod.arity extraargs = args[arity:] -## if self.ASSERT_BASE_TYPE: -## for a in args[:arity]: -## assert issubclass(a.dispatchclass, self.ASSERT_BASE_TYPE), ( -## "multimethod '%s' call with non wrapped argument: %r" % -## (self.multimethod.operatorsymbol, a)) + if self.ASSERT_BASE_TYPE: + for a in args[:arity]: + assert hasattr(a, 'dispatchclass'), ( + "multimethod '%s' call with non wrapped argument: %r" % + (self.multimethod.operatorsymbol, a)) + assert issubclass(a.dispatchclass, self.ASSERT_BASE_TYPE), ( + "multimethod '%s' call with non wrapped argument: %r" % + (self.multimethod.operatorsymbol, a)) # look for an exact match first firstfailure = None Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/nullobject.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/nullobject.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,24 +0,0 @@ -""" - Null Object implementation - - ok and tested -""" - -from pypy.objspace.std.objspace import * -from nulltype import W_NullType - -class W_NullObject(W_Object): - statictype = W_NullType -registerimplementation(W_NullObject) - -def unwrap__Null(space, w_null): - return Null - -def is_true__Null(space, w_null): - return False - -def repr__Null(space, w_null): - return space.wrap('Null') - -register_all(vars()) - Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/nulltype.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/nulltype.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,9 +0,0 @@ -from pypy.objspace.std.objspace import * -from typeobject import W_TypeObject - - -class W_NullType(W_TypeObject): - - typename = 'NullType' - -registerimplementation(W_NullType) Modified: pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py Wed Sep 17 13:38:03 2003 @@ -1,10 +1,3 @@ -""" -without intending to test objectspace, we discovered -some unforeseen wrapping of PyByteCode. -XXX further analysis needed. -""" - -import pypy.interpreter.appfile from register_all import register_all from pypy.interpreter.baseobjspace import * from multimethod import * @@ -23,7 +16,7 @@ W_ANY = W_Object # synonyms for use in .register() -#BoundMultiMethod.ASSERT_BASE_TYPE = W_Object +BoundMultiMethod.ASSERT_BASE_TYPE = W_Object MultiMethod.BASE_TYPE_OBJECT = W_AbstractTypeObject # delegation priorities @@ -48,11 +41,6 @@ PACKAGE_PATH = 'objspace.std' - class AppFile(pypy.interpreter.appfile.AppFile): - pass - AppFile.LOCAL_PATH = [PACKAGE_PATH] - - def standard_types(self): class result: "Import here the types you want to have appear in __builtin__." @@ -72,17 +60,12 @@ def clone_exception_hierarchy(self): from usertype import W_UserType - from funcobject import W_FuncObject - from pypy.interpreter.pycode import PyByteCode + from pypy.interpreter import gateway w = self.wrap - def __init__(self, *args): + def app___init__(self, *args): self.args = args - code = PyByteCode() - code._from_code(__init__.func_code) - w_init = W_FuncObject(self, code, - self.newdict([]), self.newtuple([]), None) -## w_init = w(__init__) # should this work? --mwh - def __str__(self): + w_init = w(gateway.app2interp(app___init__)) + def app___str__(self): l = len(self.args) if l == 0: return '' @@ -90,10 +73,7 @@ return str(self.args[0]) else: return str(self.args) - code = PyByteCode() - code._from_code(__str__.func_code) - w_str = W_FuncObject(self, code, - self.newdict([]), self.newtuple([]), None) + w_str = w(gateway.app2interp(app___str__)) import exceptions # to create types, we should call the standard type object; @@ -146,13 +126,11 @@ def initialize(self): from noneobject import W_NoneObject - from nullobject import W_NullObject from boolobject import W_BoolObject from cpythonobject import W_CPythonObject # singletons self.w_None = W_NoneObject(self) - self.w_Null = W_NullObject(self) self.w_False = W_BoolObject(self, False) self.w_True = W_BoolObject(self, True) self.w_NotImplemented = self.wrap(NotImplemented) # XXX do me @@ -176,7 +154,6 @@ for_builtins.update(self.clone_exception_hierarchy()) self.make_builtins() - self.make_sys() # insert stuff into the newly-made builtins for key, w_value in for_builtins.items(): @@ -221,8 +198,9 @@ wrappeditems = [self.wrap(item) for item in x] import listobject return listobject.W_ListObject(self, wrappeditems) + if hasattr(x, '__wrap__'): + return x.__wrap__(self) # print "wrapping %r (%s)" % (x, type(x)) - # XXX unexpected wrapping of PyByteCode import cpythonobject return cpythonobject.W_CPythonObject(self, x) @@ -251,15 +229,6 @@ import sliceobject return sliceobject.W_SliceObject(self, w_start, w_end, w_step) - def newfunction(self, code, w_globals, w_defaultarguments, w_closure=None): - import funcobject - return funcobject.W_FuncObject(self, code, w_globals, - w_defaultarguments, w_closure) - - def newmodule(self, w_name): - import moduleobject - return moduleobject.W_ModuleObject(self, w_name) - def newstring(self, chars_w): try: chars = [chr(self.unwrap(w_c)) for w_c in chars_w] Modified: pypy/branch/builtinrefactor/pypy/objspace/std/register_all.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/register_all.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/register_all.py Wed Sep 17 13:38:03 2003 @@ -19,7 +19,7 @@ namespaces.insert(0, alt_ns) for name, obj in module_dict.items(): - if name.find('__')<1: + if name.find('__')<1 or name.startswith('app_'): continue funcname, sig = name.split('__') l=[] Modified: pypy/branch/builtinrefactor/pypy/objspace/std/sliceobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/sliceobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/sliceobject.py Wed Sep 17 13:38:03 2003 @@ -6,13 +6,8 @@ """ from pypy.objspace.std.objspace import * -from pypy.interpreter.appfile import AppFile -from pypy.interpreter.extmodule import make_builtin_func -from pypy.objspace.std.instmethobject import W_InstMethObject from slicetype import W_SliceType -appfile = AppFile(__name__, ["objspace.std"]) - class W_SliceObject(W_Object): statictype = W_SliceType @@ -22,15 +17,6 @@ w_self.w_start = w_start w_self.w_stop = w_stop w_self.w_step = w_step - def indices(w_self, w_length): - # this is used internally, analogous to CPython's PySlice_GetIndicesEx - w_ret = w_self.space.gethelper(appfile).call("sliceindices", [w_self, w_length]) - w_start, w_stop, w_step, w_slicelength = w_self.space.unpackiterable(w_ret, 4) - return w_start, w_stop, w_step, w_slicelength - def indices2(w_self, w_length): - # this is used to implement the user-visible method 'indices' of slice objects - return w_self.space.newtuple(w_self.indices(w_length)[:-1]) - registerimplementation(W_SliceObject) @@ -51,10 +37,6 @@ return space.w_None else: return w_slice.w_step - if space.is_true(space.eq(w_attr, space.wrap('indices'))): - w_builtinfn = make_builtin_func(space, W_SliceObject.indices2) - return W_InstMethObject(space, w_builtinfn, w_slice, w_slice.statictype) - raise FailedToImplement(space.w_AttributeError) register_all(vars()) Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/sliceobject_app.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/sliceobject_app.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,50 +0,0 @@ - -def sliceindices(slice, length): - step = slice.step - if step is None: - step = 1 - elif step == 0: - raise ValueError, "slice step cannot be zero" - if step < 0: - defstart = length - 1 - defstop = -1 - else: - defstart = 0 - defstop = length - - start = slice.start - if start is None: - start = defstart - else: - if start < 0: - start += length - if start < 0: - if step < 0: - start = -1 - else: - start = 0 - elif start >= length: - if step < 0: - start = length - 1 - else: - start = length - - stop = slice.stop - if stop is None: - stop = defstop - else: - if stop < 0: - stop += length - if stop < 0: - stop = -1 - elif stop > length: - stop = length - - if (step < 0 and stop >= start) or (step > 0 and start >= stop): - slicelength = 0 - elif step < 0: - slicelength = (stop-start+1)//step + 1 - else: - slicelength = (stop-start-1)//step + 1 - - return start, stop, step, slicelength Modified: pypy/branch/builtinrefactor/pypy/objspace/std/slicetype.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/slicetype.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/slicetype.py Wed Sep 17 13:38:03 2003 @@ -1,4 +1,5 @@ from pypy.objspace.std.objspace import * +from pypy.interpreter import gateway from typeobject import W_TypeObject @@ -6,6 +7,9 @@ typename = 'slice' + slice_indices = MultiMethod('indices', 2) + + registerimplementation(W_SliceType) @@ -31,4 +35,78 @@ space.wrap("slice() takes at least 1 argument")) return space.newslice(start, stop, step), True -register_all(vars()) + +# default application-level implementations for some operations + +def app_slice_indices3(slice, length): + # this is used internally, analogous to CPython's PySlice_GetIndicesEx + step = slice.step + if step is None: + step = 1 + elif step == 0: + raise ValueError, "slice step cannot be zero" + if step < 0: + defstart = length - 1 + defstop = -1 + else: + defstart = 0 + defstop = length + + start = slice.start + if start is None: + start = defstart + else: + if start < 0: + start += length + if start < 0: + if step < 0: + start = -1 + else: + start = 0 + elif start >= length: + if step < 0: + start = length - 1 + else: + start = length + + stop = slice.stop + if stop is None: + stop = defstop + else: + if stop < 0: + stop += length + if stop < 0: + stop = -1 + elif stop > length: + stop = length + + return start, stop, step +slice_indices__ANY_ANY = slice_indices3 = gateway.app2interp(app_slice_indices3) + +def app_slice_indices4(slice, length): + start, stop, step = slice_indices3(slice, length) + + if (step < 0 and stop >= start) or (step > 0 and start >= stop): + slicelength = 0 + elif step < 0: + slicelength = (stop-start+1)//step + 1 + else: + slicelength = (stop-start-1)//step + 1 + + return start, stop, step, slicelength +slice_indices4 = gateway.app2interp(app_slice_indices4) + +# utility functions +def indices3(space, w_slice, length): + w_result = slice_indices3(space, w_slice, space.wrap(length)) + w_1, w_2, w_3 = space.unpacktuple(w_result, 3) + return space.unwrap(w_1), space.unwrap(w_2), space.unwrap(w_3) + +def indices4(space, w_slice, length): + w_result = slice_indices4(space, w_slice, space.wrap(length)) + w_1, w_2, w_3, w_4 = space.unpacktuple(w_result, 4) + return (space.unwrap(w_1), space.unwrap(w_2), + space.unwrap(w_3), space.unwrap(w_4)) + + +register_all(vars(), W_SliceType) Modified: pypy/branch/builtinrefactor/pypy/objspace/std/stringobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/stringobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/stringobject.py Wed Sep 17 13:38:03 2003 @@ -75,12 +75,14 @@ from stringtype import W_StringType from intobject import W_IntObject from sliceobject import W_SliceObject +import slicetype from listobject import W_ListObject -from instmethobject import W_InstMethObject from noneobject import W_NoneObject from tupleobject import W_TupleObject -applicationfile = StdObjSpace.AppFile(__name__) +# XXX consider reimplementing _value to be a list of characters +# instead of a plain string + class W_StringObject(W_Object): statictype = W_StringType @@ -826,21 +828,14 @@ return W_StringObject(space, str[ival]) def getitem__String_Slice(space, w_str, w_slice): - return space.gethelper(applicationfile).call( - "getitem_string_slice", [w_str, w_slice]) + # XXX this is really too slow for slices with no step argument w = space.wrap - u = space.unwrap - w_start, w_stop, w_step, w_sl = w_slice.indices(w(len(u(w_str._value)))) - start = u(w_start) - stop = u(w_stop) - step = u(w_step) - sl = u(w_sl) - r = [None] * sl - for i in range(sl): - r[i] = space.getitem(w_str, w(start + i*step)) + length = len(w_str._value) + start, stop, step, sl = slicetype.indices4(space, w_slice, length) + r = [space.getitem(w_str, w(start + i*step)) for i in range(sl)] w_r = space.newlist(r) w_empty = space.newstring([]) - return str_join(space, w_empty, w_r) + return str_join__String_ANY(space, w_empty, w_r) def mul__String_Int(space, w_str, w_mul): u = space.unwrap Deleted: /pypy/branch/builtinrefactor/pypy/objspace/std/stringobject_app.py ============================================================================== --- /pypy/branch/builtinrefactor/pypy/objspace/std/stringobject_app.py Wed Sep 17 13:38:03 2003 +++ (empty file) @@ -1,6 +0,0 @@ -def getitem_string_slice(str, sliceob): - r = [] - for i in range(*sliceob.indices(len(str))): - r.append(str[i]) - return ''.join(r) - Modified: pypy/branch/builtinrefactor/pypy/objspace/std/test/test_dictobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/test/test_dictobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/test/test_dictobject.py Wed Sep 17 13:38:03 2003 @@ -61,6 +61,35 @@ cell = space.unwrap(d.cell(space,wk2)) self.assertEqual_w(cell.get(),space.wrap(2)) + def test_empty_cell(self): + space = self.space + d = W_DictObject(space, + [(space.wrap('colour'), space.wrap(0)), + (space.wrap('of'), space.wrap(2)), + (space.wrap('magic'), space.wrap(1))]) + w_cell = d.cell(space, space.wrap('of')) + d2 = W_DictObject(space, + [(space.wrap('colour'), space.wrap(0)), + (space.wrap('magic'), space.wrap(1))]) + self.assertNotEqual_w(d, d2) + space.delitem(d, space.wrap('of')) + self.assertEqual_w(d, d2) + + def test_empty_cell2(self): + space = self.space + d = W_DictObject(space, + [(space.wrap('colour'), space.wrap(0)), + (space.wrap('of'), space.wrap(2)), + (space.wrap('magic'), space.wrap(1))]) + w_cell = d.cell(space, space.wrap('of')) + d2 = W_DictObject(space, + [(space.wrap('colour'), space.wrap(0)), + (space.wrap('magic'), space.wrap(1))]) + self.assertNotEqual_w(d, d2) + cell = space.unwrap(w_cell) + cell.make_empty() + self.assertEqual_w(d, d2) + def test_wrap_dict(self): self.assert_(isinstance(self.space.wrap({}), W_DictObject)) Modified: pypy/branch/builtinrefactor/pypy/objspace/std/test/test_instmethobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/test/test_instmethobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/test/test_instmethobject.py Wed Sep 17 13:38:03 2003 @@ -1,22 +1,8 @@ import autopath from pypy.tool import test -class TestInstMethObject(test.TestCase): - def setUp(self): - self.space = test.objspace('std') - - def test_unbound(self): - from pypy.objspace.std.instmethobject import W_InstMethObject - space = self.space - w_list = space.newlist([]) - w_boundmeth = space.getattr(w_list, space.wrap('__len__')) - w_unboundmeth = W_InstMethObject(space, - w_boundmeth.w_im_func, - space.w_Null, - w_boundmeth.w_im_class) - self.assertEqual_w(space.call_function(w_unboundmeth, w_list), - space.wrap(0)) - +# NB. instmethobject.py has been removed, +# but the following tests still make sense class TestInstMethObjectApp(test.AppTestCase): def setUp(self): Modified: pypy/branch/builtinrefactor/pypy/objspace/std/test/test_sliceobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/test/test_sliceobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/test/test_sliceobject.py Wed Sep 17 13:38:03 2003 @@ -10,29 +10,30 @@ pass def equal_indices(self, got, expected): - got = self.space.unwrap(got) self.assertEqual(len(got), len(expected)) for g, e in zip(got, expected): self.assertEqual(g, e) def test_indices(self): + from pypy.objspace.std import slicetype space = self.space w = space.wrap w_None = space.w_None w_slice = space.newslice(w_None, w_None, w_None) - self.equal_indices(w_slice.indices2(w(6)), (0, 6, 1)) + self.equal_indices(slicetype.indices3(space, w_slice, 6), (0, 6, 1)) w_slice = space.newslice(w(0), w(6), w(1)) - self.equal_indices(w_slice.indices2(w(6)), (0, 6, 1)) + self.equal_indices(slicetype.indices3(space, w_slice, 6), (0, 6, 1)) w_slice = space.newslice(w_None, w_None, w(-1)) - self.equal_indices(w_slice.indices2(w(6)), (5, -1, -1)) + self.equal_indices(slicetype.indices3(space, w_slice, 6), (5, -1, -1)) def test_indices_fail(self): + from pypy.objspace.std import slicetype space = self.space w = space.wrap w_None = space.w_None w_slice = space.newslice(w_None, w_None, w(0)) self.assertRaises_w(space.w_ValueError, - w_slice.indices, w(10)) + slicetype.indices3, space, w_slice, 10) class Test_SliceObject(test.AppTestCase): def setUp(self): @@ -49,5 +50,13 @@ self.failUnless(cmp_slice(slice(23), slice(None, 23, None))) self.failUnless(cmp_slice(slice(23, 45), slice(23, 45, None))) + def test_indices(self): + self.assertEqual(slice(4,11,2).indices(28), (4, 11, 2)) + self.assertEqual(slice(4,11,2).indices(8), (4, 8, 2)) + self.assertEqual(slice(4,11,2).indices(2), (2, 2, 2)) + self.assertEqual(slice(11,4,-2).indices(28), (11, 4, -2)) + self.assertEqual(slice(11,4,-2).indices(8), (7, 4, -2)) + self.assertEqual(slice(11,4,-2).indices(2), (1, 2, -2)) + if __name__ == '__main__': test.main() Modified: pypy/branch/builtinrefactor/pypy/objspace/std/tupleobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/tupleobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/tupleobject.py Wed Sep 17 13:38:03 2003 @@ -2,6 +2,7 @@ from tupletype import W_TupleType from intobject import W_IntObject from sliceobject import W_SliceObject +import slicetype class W_TupleObject(W_Object): @@ -39,11 +40,8 @@ def getitem__Tuple_Slice(space, w_tuple, w_slice): items = w_tuple.wrappeditems - w_length = space.wrap(len(items)) - w_start, w_stop, w_step, w_slicelength = w_slice.indices(w_length) - start = space.unwrap(w_start) - step = space.unwrap(w_step) - slicelength = space.unwrap(w_slicelength) + length = len(items) + start, stop, step, slicelength = slicetype.indices4(space, w_slice, length) assert slicelength >= 0 subitems = [None] * slicelength for i in range(slicelength): Modified: pypy/branch/builtinrefactor/pypy/objspace/std/typeobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/typeobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/typeobject.py Wed Sep 17 13:38:03 2003 @@ -1,4 +1,4 @@ -from pypy.interpreter import pycode +from pypy.interpreter import eval, function from pypy.objspace.std.objspace import * @@ -58,7 +58,8 @@ raise KeyError # pass on the KeyError if code.slice().is_empty(): raise KeyError - return space.newfunction(code, space.w_None, code.getdefaults(space)) + fn = function.Function(space, code, defs_w=code.getdefaults(space)) + return space.wrap(fn) import typetype, objecttype @@ -86,7 +87,7 @@ for multimethod in (hack_out_multimethods(typeclass) + hack_out_multimethods(spaceclass)): for i in range(len(multimethod.specialnames)): - # each PyMultimethodCode embeds a multimethod + # each MultimethodCode embeds a multimethod name = multimethod.specialnames[i] if name in multimethods: # conflict between e.g. __lt__ and @@ -94,102 +95,96 @@ code = multimethods[name] if code.bound_position < i: continue - pycodeclass = multimethod.extras.get('pycodeclass') - if pycodeclass is None: + mmframeclass = multimethod.extras.get('mmframeclass') + if mmframeclass is None: if len(multimethod.specialnames) > 1: - pycodeclass = SpecialMultimethodCode + mmframeclass = SpecialMmFrame else: - pycodeclass = PyMultimethodCode - code = pycodeclass(multimethod, typeclass, i) + mmframeclass = MmFrame + code = MultimethodCode(multimethod, mmframeclass, typeclass, i) multimethods[name] = code # add some more multimethods with a special interface - code = NextMultimethodCode(spaceclass.next, typeclass) + code = MultimethodCode(spaceclass.next, NextMmFrame, typeclass) multimethods['next'] = code - code = NonZeroMultimethodCode(spaceclass.is_true, typeclass) + code = MultimethodCode(spaceclass.is_true, NonZeroMmFrame, typeclass) multimethods['__nonzero__'] = code return multimethods -class PyMultimethodCode(pycode.PyBaseCode): - - def __init__(self, multimethod, typeclass, bound_position=0): - pycode.PyBaseCode.__init__(self) - argnames = ['x%d'%(i+1) for i in range(multimethod.arity)] - argnames.insert(0, argnames.pop(bound_position)) - self.co_name = multimethod.operatorsymbol - self.co_varnames = tuple(argnames) - self.co_argcount = multimethod.arity - self.co_flags = 0 - if multimethod.extras.get('varargs', False): - self.co_flags |= pycode.CO_VARARGS - if multimethod.extras.get('keywords', False): - self.co_flags |= pycode.CO_VARKEYWORDS +class MultimethodCode(eval.Code): + """A code object that invokes a multimethod.""" + + def __init__(self, multimethod, framecls, typeclass, bound_position=0): + eval.Code.__init__(self, multimethod.operatorsymbol) self.basemultimethod = multimethod self.typeclass = typeclass self.bound_position = bound_position + self.framecls = framecls + argnames = ['x%d'%(i+1) for i in range(multimethod.arity)] + argnames.insert(0, argnames.pop(self.bound_position)) + varargname = kwargname = None + if multimethod.extras.get('varargs', False): + varargname = 'args' + if multimethod.extras.get('keywords', False): + kwargname = 'keywords' + self.sig = argnames, varargname, kwargname + + def signature(self): + return self.sig def getdefaults(self, space): - return space.wrap(self.basemultimethod.extras.get('defaults', ())) + return [space.wrap(x) + for x in self.basemultimethod.extras.get('defaults', ())] def slice(self): return self.basemultimethod.slice(self.typeclass, self.bound_position) - def prepare_args(self, space, w_globals, w_locals): - multimethod = self.slice() - dispatchargs = [] - for i in range(multimethod.arity): - w_arg = space.getitem(w_locals, space.wrap('x%d'%(i+1))) - dispatchargs.append(w_arg) - dispatchargs = tuple(dispatchargs) - return multimethod.get(space), dispatchargs + def create_frame(self, space, w_globals, closure=None): + return self.framecls(space, self) - def do_call(self, space, w_globals, w_locals): +class MmFrame(eval.Frame): + def run(self): "Call the multimethod, raising a TypeError if not implemented." - mm, args = self.prepare_args(space, w_globals, w_locals) - return mm(*args) - - def eval_code(self, space, w_globals, w_locals): - "Call the multimethods, or raise a TypeError." - w_result = self.do_call(space, w_globals, w_locals) + mm = self.code.slice().get(self.space) + args = self.fastlocals_w + w_result = mm(*args) # we accept a real None from operations with no return value if w_result is None: - w_result = space.w_None + w_result = self.space.w_None return w_result -class SpecialMultimethodCode(PyMultimethodCode): - - def do_call(self, space, w_globals, w_locals): +class SpecialMmFrame(eval.Frame): + def run(self): "Call the multimethods, possibly returning a NotImplemented." - mm, args = self.prepare_args(space, w_globals, w_locals) + mm = self.code.slice().get(self.space) + args = self.fastlocals_w try: return mm.perform_call(args) except FailedToImplement, e: if e.args: raise OperationError(*e.args) else: - return space.w_NotImplemented - -class NextMultimethodCode(PyMultimethodCode): + return self.space.w_NotImplemented - def eval_code(self, space, w_globals, w_locals): +class NextMmFrame(eval.Frame): + def run(self): "Call the next() multimethod." + mm = self.code.slice().get(self.space) + args = self.fastlocals_w try: - return self.do_call(space, w_globals, w_locals) + return mm(*args) except NoValue: - raise OperationError(space.w_StopIteration, space.w_None) + raise OperationError(self.space.w_StopIteration, + self.space.w_None) -class NonZeroMultimethodCode(PyMultimethodCode): - - def eval_code(self, space, w_globals, w_locals): +class NonZeroMmFrame(eval.Frame): + def run(self): "Call the is_true() multimethods." - result = self.do_call(space, w_globals, w_locals) - return space.newbool(result) - -class NewMultimethodCode(PyMultimethodCode): + mm = self.code.slice().get(self.space) + args = self.fastlocals_w + result = mm(*args) + return self.space.newbool(result) - def eval_code(self, space, w_globals, w_locals): - "Call the __new__() method of typetype.py." - w_result, callinit = self.do_call(space, w_globals, w_locals) - return w_result +# see also class NewMmFrame in typetype.py def call__Type_ANY_ANY(space, w_type, w_args, w_kwds): @@ -205,7 +200,7 @@ return space.newbool(w_type2 in w_type1.getmro()) def repr__Type(space, w_obj): - return space.wrap("" % w_obj.typename) + return space.wrap("" % w_obj.typename) # XXX remove 'pypy' def getattr__Type_ANY(space, w_type, w_attr): # XXX mwh doubts this is the Right Way to do this... @@ -217,7 +212,7 @@ desc = w_type.lookup(w_attr) except KeyError: raise FailedToImplement #OperationError(space.w_AttributeError,w_attr) - return space.get(desc, space.w_Null, w_type) + return space.get(desc, space.w_None, w_type) register_all(vars()) Modified: pypy/branch/builtinrefactor/pypy/objspace/std/typetype.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/typetype.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/typetype.py Wed Sep 17 13:38:03 2003 @@ -1,11 +1,16 @@ from pypy.objspace.std.objspace import * from pypy.objspace.std.register_all import register_all +from pypy.interpreter import eval from typeobject import W_TypeObject -def NewMultimethodCode_builder(*args): - from typeobject import NewMultimethodCode # sadly necessary late import hack - return NewMultimethodCode(*args) +class NewMmFrame(eval.Frame): + def run(self): + "Call the __new__() method of typetype.py." + mm = self.code.slice().get(self.space) + args = self.fastlocals_w + w_result, callinit = mm(*args) + return w_result class W_TypeType(W_TypeObject): @@ -23,7 +28,7 @@ # Attention, this internally returns a tuple (w_result, flag), # where 'flag' specifies whether we would like __init__() to be called. type_new = MultiMethod('__new__', 4, varargs=True, keywords=True, - pycodeclass=NewMultimethodCode_builder) + mmframeclass=NewMmFrame) registerimplementation(W_TypeType) Modified: pypy/branch/builtinrefactor/pypy/objspace/trivial.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/trivial.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/trivial.py Wed Sep 17 13:38:03 2003 @@ -177,9 +177,11 @@ def _auto(name, sourcefn, classlocals): s = """ -def %(name)s(self, *args): +def %(name)s(self, x, *args): + if hasattr(x, 'pypy_%(name)s'): + return x.pypy_%(name)s(*args) try: - value = %(sourcefn)s(*args) + value = %(sourcefn)s(x, *args) except: self.reraise() return self.wrap(value) @@ -188,7 +190,6 @@ # from the built-ins _auto('issubtype', 'issubclass', locals()) - _auto('newmodule', 'new.module', locals()) _auto('newtuple', 'tuple', locals()) _auto('newlist', 'list', locals()) _auto('newdict', 'dict', locals()) @@ -262,7 +263,7 @@ if stop is None: stop = sys.maxint return start, stop return None - + def getitem(self, w_obj, w_index): obj = self.unwrap(w_obj) index = self.unwrap(w_index) @@ -302,25 +303,13 @@ # misc def next(self, w): + if hasattr(w, 'pypy_next'): + return w.pypy_next() try: return self.wrap(w.next()) except StopIteration: raise NoValue - def newfunction(self, code, globals, defs, closure=None): - #from pypy.interpreter.gateway import Function - #return Function(self, code, globals, defaultarguments, closure) - - #assert hasattr(code, 'co_name') - #assert hasattr(code, 'build_arguments') - #assert hasattr(code, 'eval_code') - - # XXX the newxxx() interface should disappear at some point - # XXX because the interpreter can perfectly do the following - # XXX itself without bothering the object space - from pypy.interpreter.function import Function - return self.wrap(Function(self, code, globals, defs, closure)) - def newstring(self, asciilist): try: return ''.join([chr(ascii) for ascii in asciilist]) @@ -344,6 +333,8 @@ assert not isinstance(callable, gateway.Gateway), ( "trivial object space is detecting an object that has not " "been wrapped") + if hasattr(callable, 'pypy_call'): + return callable.pypy_call(args, kwds) try: return self.wrap(callable(*args, **(kwds or {}))) except OperationError: From tismer at codespeak.net Wed Sep 17 14:10:12 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 17 Sep 2003 14:10:12 +0200 (MEST) Subject: [pypy-svn] rev 1333 - pypy/trunk/doc/funding Message-ID: <20030917121012.0AB955A114@thoth.codespeak.net> Author: tismer Date: Wed Sep 17 14:10:12 2003 New Revision: 1333 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Log: refining stuff. RUnning out of time for the train... Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Wed Sep 17 14:10:12 2003 @@ -3,22 +3,22 @@ B 6.1 Workplan Introduction =============================== -The PyPy project has a number of topics, which +The PyPy project addresses a number of topics, which can be categorized into the following groups. Each group is further broken down into working packages. -- The PyPy Interpreter - -- The PyPy Compiler - -- Bootstrapping PyPy - -- High-performance PyPy-Python - - development of PyPy itself, as a Python written in Python. This includes code generation for various targets, re-implementation of all builtin Python objects and some extension modules, and the development of several object spaces, testing and documentation. + + - The PyPy Interpreter + + - The PyPy Compiler + + - Bootstrapping PyPy + + - High-performance PyPy-Python - application level code which shows the results of PyPy as a usability study. Generating down-sized code for embedded systems, load balancing @@ -28,11 +28,25 @@ academical exercise but very apropriate for industrial strength applications. - - -XXX re-organise: Major topics, WPs go beyond them. - - + - Load Balancing in a Multi-Processor environment + + - Specialized Code for Embedded Devices + + - Numerical Applications + +- Infra structure tasks carried out throughout the whole project + + - Coordination and Management + + - Project Documentation + + - Maintenance of Tools + + - Quality Control + + - Synchronisation with Standard Python + + The PyPy Interpreter --------------------------- @@ -239,7 +253,7 @@ the choices made for the issues of High-performance. c) (_`APP`)combine different solutions for the different issues discussed in - `High-performance`_. Gather statistics with real-work Python applications. + High-performance. Gather statistics with real-work Python applications. Compare the results. This is where the flexibility of the whole project is vital. Typically, very different trade-offs need to be made on different environments. @@ -259,17 +273,17 @@ Application Level Targets ----------------------------- -[[WPxxx]] Load Balancing in a Multi-Processor environment +Load Balancing in a Multi-Processor environment ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *XXX to be finished, collecting snippets* -[[WPxxx]] Specialized Code for Embedded Devices +Specialized Code for Embedded Devices ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *XXX to be finished, collecting snippets* -[[WPxxx]] Numerical Applications +Numerical Applications ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ PyPy is extremely flexible, since it supports static and dynamic code generation by pluggable processor @@ -307,7 +321,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *XXX this is what I'm working on at the moment -- the text is -not yet worked out enough. Another hour...* +not yet worked out enough. This will be two WPs.* Modified: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt (original) +++ pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Wed Sep 17 14:10:12 2003 @@ -6,7 +6,7 @@ ================================ ========================== **Workpackage number** 03 -------------------------------- -------------------------- -**Start date or starting event** 0 +**Start date or starting event** end of WP02_/WP03_ ================================ ========================== ================================= ======= ======= ======= ======= ======= ======= ======= ======= Modified: pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt (original) +++ pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt Wed Sep 17 14:10:12 2003 @@ -6,7 +6,7 @@ ================================ ========================== **Workpackage number** 04 -------------------------------- -------------------------- -**Start date or starting event** 0 +**Start date or starting event** WP03_ ================================ ========================== ================================= ======= ======= ======= ======= ======= ======= ======= ======= Modified: pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt (original) +++ pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Wed Sep 17 14:10:12 2003 @@ -6,7 +6,7 @@ ================================ ========================== **Workpackage number** 05 -------------------------------- -------------------------- -**Start date or starting event** 0 +**Start date or starting event** WP03_ ================================ ========================== ================================= ======= ======= ======= ======= ======= ======= ======= ======= From tismer at codespeak.net Wed Sep 17 14:12:03 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 17 Sep 2003 14:12:03 +0200 (MEST) Subject: [pypy-svn] rev 1334 - pypy/trunk/doc/funding Message-ID: <20030917121203.930CC5A145@thoth.codespeak.net> Author: tismer Date: Wed Sep 17 14:12:02 2003 New Revision: 1334 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt Log: spelling corrections Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Wed Sep 17 14:12:02 2003 @@ -93,7 +93,7 @@ The goal is to be able to translate arbitrary RPython source code (e.g. the one produced in WP01_ into low-level code (C, Pyrex, Java, others). This includes making a stand-alone, not-PyPy-related tool for general -optimization of arbitrary but suitably restricted Python application or +optimisation of arbitrary but suitably restricted Python application or parts thereof. **Part a): Code Analysis** @@ -185,8 +185,8 @@ WP07_: Translator Optimisations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -It should be identified, which **optimizations** would benefit -from support from the translator. These are the optimizations +It should be identified, which **optimisations** would benefit +from support from the translator. These are the optimisations not easily available to CPython because they would require large-scale code rewrites. This includes design-considerations including the decision whether @@ -197,12 +197,12 @@ -- namely the work on the translator instead of on the resulting code -- is what gives us the possibility of actually implementing several very different schemes. The outcome of this effort is of course not unique, -but depends on the specific target of the optimization. +but depends on the specific target of the optimisation. There will be at least two efforts at the same time: -- optimization towards maximum performance of an integrated PyPy system +- optimisation towards maximum performance of an integrated PyPy system -- optimization towards minimal code size for embedded systems. +- optimisation towards minimal code size for embedded systems. WP08_: Integration of Existing Technology @@ -217,7 +217,7 @@ Developing C Psyco has proved that more flexibility would be of paramount importance to the project. All prior knowledge of the Psyco project will thus be integrated into PyPy, as Python and -RPython code, enabling yet more efficient optimizations and +RPython code, enabling yet more efficient optimisations and allowing new processor architectures to be targeted. Stackless Python has implemented high speed multitasking in @@ -243,7 +243,7 @@ high-performance Python interpreter. For each of these, focus will be given to: -a) develop the translation process, run-time and those optimizations +a) develop the translation process, run-time and those optimisations that depend on low-level details. b) design interfaces for extension modules. Some can be very general From tismer at codespeak.net Wed Sep 17 14:14:02 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 17 Sep 2003 14:14:02 +0200 (MEST) Subject: [pypy-svn] rev 1335 - pypy/trunk/doc/funding Message-ID: <20030917121402.4AA8D5A145@thoth.codespeak.net> Author: tismer Date: Wed Sep 17 14:14:01 2003 New Revision: 1335 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt Log: spelling Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Wed Sep 17 14:14:01 2003 @@ -57,7 +57,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The PyPy interpreter itself should be developed and completed as a regular Python/RPython program. This package includes all -parts of CPython tht we don't want to move to WP02_. +parts of CPython that we don't want to move to WP02_. Further investigation is needed concerning the multimethod concepts of the standard object space, and how to hook in the bytecode compiler. From arigo at codespeak.net Wed Sep 17 14:50:07 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Sep 2003 14:50:07 +0200 (MEST) Subject: [pypy-svn] rev 1337 - pypy/trunk/src/pypy/tool Message-ID: <20030917125007.1455A5A114@thoth.codespeak.net> Author: arigo Date: Wed Sep 17 14:50:06 2003 New Revision: 1337 Modified: pypy/trunk/src/pypy/tool/fixeol Log: A version of fixeol that finds *.txt files as well, and can correct existing files on your working copy that have inconsistent line endings Modified: pypy/trunk/src/pypy/tool/fixeol ============================================================================== --- pypy/trunk/src/pypy/tool/fixeol (original) +++ pypy/trunk/src/pypy/tool/fixeol Wed Sep 17 14:50:06 2003 @@ -1,12 +1,56 @@ #! /usr/bin/env python -import os +import sys, os + +forbidden = range(0,32) +forbidden.remove(9) # tab +forbidden.remove(10) # lf +forbidden.remove(12) # ^L +forbidden.remove(13) # cr + +def looksbinary(data, forbidden = [chr(i) for i in forbidden]): + for c in forbidden: + if c in data: + return True + return False + +# hack to get the platform's native end-of-line format +f = open('@fixeol at tmp.txt', 'w') +print >> f +f.close() +f = open('@fixeol at tmp.txt', 'rb') +native_eol = f.read() +f.close() +os.unlink('@fixeol at tmp.txt') + +def binary2text(filename, native_eol = native_eol): + # convert to the platform's native end-of-line format if needed + f = open(filename, 'rb') + data = f.read() + f.close() + if looksbinary(data): + return False + original = data + data = data.replace('\r\n', '\n') + data = data.replace('\n', native_eol) + data = data.replace('\r', native_eol) + if data != original: + f = open(filename, 'wb') + f.write(data) + f.close() + return True + def fixpyfiles(ignored, dirname, fnames): numpyfiles = 0 for fname in fnames: - if fname.endswith('.py'): - # change end-of-line style of each .py file to 'native' + if fname.endswith('.py') or fname.endswith('.txt'): + # safety check to nail binary files fname = os.path.join(dirname, fname) + if not binary2text(fname): + print >> sys.stderr, "*** warning, looks like a binary file:", + print >> sys.stderr, fname + continue + # change end-of-line style of each .py and .txt file to 'native' os.system('svn propset svn:eol-style native %s' % fname) numpyfiles += 1 if numpyfiles: @@ -21,5 +65,7 @@ g.close() os.system('svn propset svn:ignore -F svn-ignore.tmp %s' % dirname) os.unlink('svn-ignore.tmp') + if '.svn' in fnames: + fnames.remove('.svn') os.path.walk(os.curdir, fixpyfiles, None) From hpk at codespeak.net Wed Sep 17 15:36:47 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 17 Sep 2003 15:36:47 +0200 (MEST) Subject: [pypy-svn] rev 1338 - in pypy/branch/builtinrefactor/pypy: interpreter module Message-ID: <20030917133647.BD4EE5A114@thoth.codespeak.net> Author: hpk Date: Wed Sep 17 15:36:47 2003 New Revision: 1338 Modified: pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py pypy/branch/builtinrefactor/pypy/module/builtin.py Log: - fixed two places to run with python-2.2 M pypy/interpreter/extmodule.py - fixed the introspection loop to work with python-2.2 - minimized it a bit M pypy/interpreter/baseobjspace.py - early initialize w_builtins to a dummy value in order to allow frame creation (which needs space.w_builtins) during sys/builtin init. M pypy/module/builtin.py - allow generators for the 'compile' builtin by default to allow tests working on python-2.2 Modified: pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/baseobjspace.py Wed Sep 17 15:36:47 2003 @@ -25,10 +25,14 @@ self.initialize() def make_builtins(self): + # initializing builtins may require creating a frame which in + # turn already accesses space.w_builtins, provide a dummy one ... + self.w_builtins = self.newdict([]) + assert not hasattr(self, 'builtin') if not hasattr(self, 'sys'): self.make_sys() - + from pypy.module import builtin self.builtin = builtin.__builtin__(self) self.w_builtin = self.wrap(self.builtin) Modified: pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py (original) +++ pypy/branch/builtinrefactor/pypy/interpreter/extmodule.py Wed Sep 17 15:36:47 2003 @@ -18,21 +18,19 @@ # to build the dictionary of the module we get all the objects # accessible as 'self.xxx'. Methods are bound. - d = {} for cls in self.__class__.__mro__: - for name, value in cls.__dict__.iteritems(): + for name in cls.__dict__: # ignore names in '_xyz' if not name.startswith('_') or name.endswith('_'): + value = cls.__dict__[name] if isinstance(value, gateway.Gateway): name = value.name value = value.__get__(self) # get a Method - else: - if hasattr(value, '__get__'): - continue # ignore CPython functions - if name not in d: - d[name] = value - for key, value in d.iteritems(): - space.setitem(self.w_dict, space.wrap(key), space.wrap(value)) + elif hasattr(value, '__get__'): + continue # ignore CPython functions + + space.call_method(self.w_dict, 'setdefault', + space.wrap(name), space.wrap(value)) __metaclass__ = InitializedClass def __initclass__(cls): Modified: pypy/branch/builtinrefactor/pypy/module/builtin.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/module/builtin.py (original) +++ pypy/branch/builtinrefactor/pypy/module/builtin.py Wed Sep 17 15:36:47 2003 @@ -76,7 +76,9 @@ dont_inherit = 0 #print (str, filename, startstr, supplied_flags, dont_inherit) - c = cpy_builtin.compile(str, filename, startstr, supplied_flags, dont_inherit) + # XXX we additionally allow GENERATORS because compiling some builtins + # requires it. doesn't feel quite right to do that here. + c = cpy_builtin.compile(str, filename, startstr, supplied_flags|4096, dont_inherit) from pypy.interpreter.pycode import PyCode return space.wrap(PyCode()._from_code(c)) From mwh at codespeak.net Wed Sep 17 16:15:31 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 16:15:31 +0200 (MEST) Subject: [pypy-svn] rev 1339 - pypy/trunk/doc/funding Message-ID: <20030917141531.96C9A5A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 16:15:30 2003 New Revision: 1339 Modified: pypy/trunk/doc/funding/B2.0.0_background.txt Log: consistently wrap lines at 70 chars. Modified: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_background.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Wed Sep 17 16:15:30 2003 @@ -8,41 +8,41 @@ Language (VHLL). Its development started in 1990 at CWI, Centrum voor Wiskunde en Informatica, the National Research Institute for Mathematics and Computer Science in the Netherlands. Its principal -author is Guido van Rossum, a Dutch citizen currently living in -the United States. It is an Free/Open-Source language. The most -recent version of the language is Python 2.3, released under -the Python Software Foundation License, which is approved by both -the Open Source Initiative, and the Free Software Foundation. +author is Guido van Rossum, a Dutch citizen currently living in the +United States. It is an Free/Open-Source language. The most recent +version of the language is Python 2.3, released under the Python +Software Foundation License, which is approved by both the Open Source +Initiative, and the Free Software Foundation. FIXME: Include this as a reference. http://www.opensource.org/licenses/PythonSoftFoundation.php -Python has tens, perhaps hundreds of thousands of active developers +Python has tens, perhaps hundreds of thousands of active developers worldwide, which makes it one of the top ten most popular programming -languages in the world. [FIXME Footnote -- see section XXX and Appendex -XXX for support for this assertion]. Some other languages, specifically -C, C++, Java, Perl, and Visual Basic, have even larger user bases but they -are either proprietary or rather low-level languages. On the other hand, -the languages which most excite the Computer Science Research community --- Self, Lisp, Haskell, Limbo, ML, and so on, are nowhere on this list --- yet they're the targets of most European academic research and -innovation. Thus European economic competitiveness suffers. The -innovative research lives in academia, trapped in languages that are -rarely used for commercial development. +languages in the world. [FIXME Footnote -- see section XXX and +Appendex XXX for support for this assertion]. Some other languages, +specifically C, C++, Java, Perl, and Visual Basic, have even larger +user bases but they are either proprietary or rather low-level +languages. On the other hand, the languages which most excite the +Computer Science Research community -- Self, Lisp, Haskell, Limbo, ML, +and so on -- are nowhere on this list, yet they're the targets of most +European academic research and innovation. Thus European economic +competitiveness suffers. The innovative research lives in academia, +trapped in languages that are rarely used for commercial development. Of those more popular languages, two, Java and Visual Basic are proprietary. Sun Microsystems owns Java, and Microsoft owns Visual -Basic. Any company which writes its software in Java or Visual Basic is -at the mercy of these large American companies. And this is a real, and -not theoretical threat. In 2002, Microsoft announced that it would no -longer be supporting Visual Basic 6.0 after the year 2005. All Visual -Basic Developers have been told to convert their code to run under -Microsoft's new .NET framework. In 2001 Microsoft immediately stopped -supporting its Visual J++ language, meant to be a direct competitor with -Java, after settling a lawsuit with Sun Microsystems. Microsoft is -making these decisions because they make business sense for Microsoft, -regardless of the effects on businesses who develop software using -Microsoft proprietary software. +Basic. Any company which writes its software in Java or Visual Basic +is at the mercy of these large American companies. And this is a +real, and not theoretical threat. In 2002, Microsoft announced that +it would no longer be supporting Visual Basic 6.0 after the year 2005. +All Visual Basic Developers have been told to convert their code to +run under Microsoft's new .NET framework. In 2001 Microsoft +immediately stopped supporting its Visual J++ language, meant to be a +direct competitor with Java, after settling a lawsuit with Sun +Microsystems. Microsoft is making these decisions because they make +business sense for Microsoft, regardless of the effects on businesses +who develop software using Microsoft proprietary software. European SMEs are moving to Free/Open Source platforms @@ -55,8 +55,8 @@ (www.python-in-business.org), at EuroPython, the European Python Community Conference (www.europython.org). -FIXME! include the bylaws of the PBF, its Board of Directors and -the like as an Appendex. +FIXME! include the bylaws of the PBF, its Board of Directors and the +like as an Appendex. Advancing the Python platform ----------------------------- @@ -69,19 +69,20 @@ applications designed for handhelds, and that the interpreter itself ran too slowly. -The developers of embedded systems intend to run on tiny machines would -like a language with a 'smaller footprint'. They would like to strip out -everything which they do not need from the language and run with the -bare-bones minimum. This is hard to do in any language, and Python was -not implemented with this goal in mind. - -Another goal that was not paramount in Python's design and implementation -was execution speed. Python is a dynamically-typed, late-binding, -interpreted language. While this proved to provide extremely productive -development environments, execution speed sometimes is not fast enough. -Today, Optimisation of high-level languages must be done at run-time, and -is notoriously more difficult to optimise than statically typed, early-binding -compiled languages such as C or C++. +The developers of embedded systems intend to run on tiny machines +would like a language with a 'smaller footprint'. They would like to +strip out everything which they do not need from the language and run +with the bare-bones minimum. This is hard to do in any language, and +Python was not implemented with this goal in mind. + +Another goal that was not paramount in Python's design and +implementation was execution speed. Python is a dynamically-typed, +late-binding, interpreted language. While this proved to provide +extremely productive development environments, execution speed +sometimes is not fast enough. Today, Optimisation of high-level +languages must be done at run-time, and is notoriously more difficult +to optimise than statically typed, early-binding compiled languages +such as C or C++. Now a number of people and factors played together to start what is now one of the most promising high-level-language projects. @@ -90,44 +91,45 @@ -------------------------- Independently some researchers who worked with Python were pondering -writing an implementation of Python in Python itself. This group included -Armin Rigo, author of Psyco http://psyco.sourceforge.net/introduction.html, -a specialising Just-In-Time compiler for Python. He is intimately familiar -with both Python internals and advanced research in compilers and runtime -systems, and saw a Python implementation in Python itself as a chance to -put the two fields together. It is useful to quote from his webpage which -states the goals of his Psyco in full: +writing an implementation of Python in Python itself. This group +included Armin Rigo, author of Psyco +http://psyco.sourceforge.net/introduction.html, a specialising +Just-In-Time compiler for Python. He is intimately familiar with both +Python internals and advanced research in compilers and runtime +systems, and saw a Python implementation in Python itself as a chance +to put the two fields together. It is useful to quote from his webpage +which states the goals of his Psyco in full: My goal in programming Psyco is to contribute to reduce the following wide gap between academic computer science and industrial programming tools. - While the former develops a number of programming languages with - very cool semantics and features, the latter stick with - low-level languages principally for performance reasons, on the - ground that the higher the level of a language, the slower it - is. Althought clearly justified in practice, this belief is - theoretically false, and even completely inverted --- for large, - evolving systems like a whole operating system and its + While the former develops a number of programming languages + with very cool semantics and features, the latter stick with + low-level languages principally for performance reasons, on + the ground that the higher the level of a language, the slower + it is. Althought clearly justified in practice, this belief + is theoretically false, and even completely inverted --- for + large, evolving systems like a whole operating system and its applications, high-level programming can deliver much higher performances. - The new class of languages called 'dynamic scripting languages', - of which Python is an example, is semantically close to - long-studied languages like Lisp. The constraints behind their - designs are however different: some high-level languages can be - relatively well statically compiled, we can do some type - inference, and so on, whereas with Python it is much harder --- - the design goal was different. We now have powerful enough - machines to stick with interpretation for a number of - applications. This, of course, contributes to the common belief - that high-level languages are terribly slow. + The new class of languages called 'dynamic scripting + languages', of which Python is an example, is semantically + close to long-studied languages like Lisp. The constraints + behind their designs are however different: some high-level + languages can be relatively well statically compiled, we can + do some type inference, and so on, whereas with Python it is + much harder --- the design goal was different. We now have + powerful enough machines to stick with interpretation for a + number of applications. This, of course, contributes to the + common belief that high-level languages are terribly slow. Psyco is both an academic and an industrial project. It is an - academic experiment testing some new techniques in the field of - on-line specialization. It develops an industrially useful - performance benefit for Python. And first of all it is a modest - step towards: + academic experiment testing some new techniques in the field + of on-line specialization. It develops an industrially useful + performance benefit for Python. And first of all it is a + modest step towards: High-level languages are faster than low-level ones! @@ -148,16 +150,16 @@ Some mailing list discussions ----------------------------- -On the German Python mailing list an independent discussion evolved -where developers and academics were pondering about the possibility -of developing a 'minimal' Python implemented in Python itself. Most -noticably Christian Tismer author of an industrial-use extension -('Stackless') of the Python language noted in a postscriptum of one -of his mails that having a minimized version of Python could provide -a new base to advance the language. Nevertheless, his extensions -to the language proved to be useful for companies who needed a way -to have millions of active objects and he had a branch of C-Python -to make this possible. +On the German Python mailing list an independent discussion evolved +where developers and academics were pondering about the possibility of +developing a 'minimal' Python implemented in Python itself. Most +noticably Christian Tismer author of an industrial-use extension +('Stackless') of the Python language noted in a postscriptum of one of +his mails that having a minimized version of Python could provide a +new base to advance the language. Nevertheless, his extensions to the +language proved to be useful for companies who needed a way to have +millions of active objects and he had a branch of C-Python to make +this possible. Some organizational experience ------------------------------ @@ -169,7 +171,8 @@ to-be successor of the successful Zope web-platform) he realized that agile 'sprints' in combination with the rapid development language Python provide an extremetly productive way of communicating about and -coding complex projects where traditional, slow-moving methods often fail. +coding complex projects where traditional, slow-moving methods often +fail. At a Sprint a group of people assemble together to write code and practice Agile software methodological techniques, such as @@ -179,13 +182,13 @@ ASK_STOCKHOLM DO we need to discuss Sprints more? Holger Krekel, seeing the opportunity to launch the PyPy project with -Armin Rigo and Christian Tismer offered to organize the first -one-week meeting, the 'Sprint towards a minimal Python'. Soon many -interested developers joined and intense academical and practical -planning ensused. Just a few weeks later the Sprint took place at +Armin Rigo and Christian Tismer offered to organize the first one-week +meeting, the 'Sprint towards a minimal Python'. Soon many interested +developers joined and intense academical and practical planning +ensused. Just a few weeks later the Sprint took place at 'Trillke-Gut', a castle-like building in Germany, bringing together some key developers, among them Michael Hudson, the release manager of -version 2.2.1 of Python. +version 2.2.1 of Python. Here is the mail that started this now rapidly evolving project http://starship.python.net/pipermail/python-de/2003q1/002925.html @@ -207,12 +210,12 @@ ----------------------------------------------------- This combination of research, applicability and industrial experience -was precisely what the Python Business Forum members had been -looking for. Academics, developers and practioners conversant in the -latest in language and platform architecture who were interested in -producing the Python interpreter which they needed. Laura Creighton and -Jacob Hall?n from the PBF attended the sprint and began participating -in the project. +was precisely what the Python Business Forum members had been looking +for. Academics, developers and practioners conversant in the latest +in language and platform architecture who were interested in producing +the Python interpreter which they needed. Laura Creighton and Jacob +Hall?n from the PBF attended the sprint and began participating in the +project. The group decided soon to test their proof of concept by developing a working prototype and to test their ability to work together. They @@ -265,16 +268,16 @@ < While some people suggested to apply to DARPA for funding we realized that most of us were rooted in Europe and it would make more sense to -look for european possibilities. +look for european possibilities. I don't think that this is wise. Why remind them that we could get -somebody else to fund it, maybe? Besides, everybody on the list -of people they are getting are in Europe, not just most ... +somebody else to fund it, maybe? Besides, everybody on the list of +people they are getting are in Europe, not just most ... > On June 17th, the 2nd Call of the Information Society Technologies [IST] Priority went out. Included in it was IST-2002-2.3.2.3 - Open development platforms for software and services. -We believe that what we intend to do is a perfect fit for the -goals of this call. +We believe that what we intend to do is a perfect fit for the goals of +this call. From mwh at codespeak.net Wed Sep 17 16:21:29 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 16:21:29 +0200 (MEST) Subject: [pypy-svn] rev 1340 - pypy/trunk/doc/funding Message-ID: <20030917142129.C245D5A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 16:21:29 2003 New Revision: 1340 Modified: pypy/trunk/doc/funding/B2.0.0_background.txt Log: speeling Modified: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_background.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Wed Sep 17 16:21:29 2003 @@ -20,7 +20,7 @@ Python has tens, perhaps hundreds of thousands of active developers worldwide, which makes it one of the top ten most popular programming languages in the world. [FIXME Footnote -- see section XXX and -Appendex XXX for support for this assertion]. Some other languages, +Appendix XXX for support for this assertion]. Some other languages, specifically C, C++, Java, Perl, and Visual Basic, have even larger user bases but they are either proprietary or rather low-level languages. On the other hand, the languages which most excite the @@ -56,7 +56,7 @@ Community Conference (www.europython.org). FIXME! include the bylaws of the PBF, its Board of Directors and the -like as an Appendex. +like as an Appendix. Advancing the Python platform ----------------------------- @@ -108,8 +108,8 @@ with very cool semantics and features, the latter stick with low-level languages principally for performance reasons, on the ground that the higher the level of a language, the slower - it is. Althought clearly justified in practice, this belief - is theoretically false, and even completely inverted --- for + it is. Although clearly justified in practise, this belief is + theoretically false, and even completely inverted --- for large, evolving systems like a whole operating system and its applications, high-level programming can deliver much higher performances. @@ -153,8 +153,8 @@ On the German Python mailing list an independent discussion evolved where developers and academics were pondering about the possibility of developing a 'minimal' Python implemented in Python itself. Most -noticably Christian Tismer author of an industrial-use extension -('Stackless') of the Python language noted in a postscriptum of one of +noticeably Christian Tismer author of an industrial-use extension +('Stackless') of the Python language noted in a postscript to one of his mails that having a minimized version of Python could provide a new base to advance the language. Nevertheless, his extensions to the language proved to be useful for companies who needed a way to have @@ -170,14 +170,14 @@ participating in two 'coding sprints' of the Zope3 web-platform (the to-be successor of the successful Zope web-platform) he realized that agile 'sprints' in combination with the rapid development language -Python provide an extremetly productive way of communicating about and +Python provide an extremely productive way of communicating about and coding complex projects where traditional, slow-moving methods often fail. At a Sprint a group of people assemble together to write code and practice Agile software methodological techniques, such as Pair-Programming. Not only is this a lot of fun, but it is a way to -transmit knowledge and enthusiasm thoughout the community. +transmit knowledge and enthusiasm throughout the community. ASK_STOCKHOLM DO we need to discuss Sprints more? @@ -185,7 +185,7 @@ Armin Rigo and Christian Tismer offered to organize the first one-week meeting, the 'Sprint towards a minimal Python'. Soon many interested developers joined and intense academical and practical planning -ensused. Just a few weeks later the Sprint took place at +ensued. Just a few weeks later the Sprint took place at 'Trillke-Gut', a castle-like building in Germany, bringing together some key developers, among them Michael Hudson, the release manager of version 2.2.1 of Python. @@ -211,7 +211,7 @@ This combination of research, applicability and industrial experience was precisely what the Python Business Forum members had been looking -for. Academics, developers and practioners conversant in the latest +for. Academics, developers and practitioners conversant in the latest in language and platform architecture who were interested in producing the Python interpreter which they needed. Laura Creighton and Jacob Hall?n from the PBF attended the sprint and began participating in the @@ -226,7 +226,7 @@ Laura Creighton organized the next Sprint at AB Strakt in Gothenburg, Sweden. At this Sprint, Samuele Pedroni, the lead developer of Jython (a tight industrial-use integration of Java and Python), joined the -project not only because such a project was relavant to his own +project not only because such a project was relevant to his own computer science research interests, but also to see if PyPy would help with his own java-integration project. Prototyping went quickly, with the work of all these experienced people, and by the end of the From mwh at codespeak.net Wed Sep 17 16:38:27 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 16:38:27 +0200 (MEST) Subject: [pypy-svn] rev 1341 - pypy/trunk/doc/funding Message-ID: <20030917143827.5A2585A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 16:38:26 2003 New Revision: 1341 Modified: pypy/trunk/doc/funding/B2.0.0_background.txt Log: A little light proof-reading: improve consistency, grammar and make the attitude to capitals a little less German... Modified: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_background.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Wed Sep 17 16:38:26 2003 @@ -9,7 +9,7 @@ Wiskunde en Informatica, the National Research Institute for Mathematics and Computer Science in the Netherlands. Its principal author is Guido van Rossum, a Dutch citizen currently living in the -United States. It is an Free/Open-Source language. The most recent +United States. It is an Free/Open Source language. The most recent version of the language is Python 2.3, released under the Python Software Foundation License, which is approved by both the Open Source Initiative, and the Free Software Foundation. @@ -25,17 +25,18 @@ user bases but they are either proprietary or rather low-level languages. On the other hand, the languages which most excite the Computer Science Research community -- Self, Lisp, Haskell, Limbo, ML, -and so on -- are nowhere on this list, yet they're the targets of most -European academic research and innovation. Thus European economic -competitiveness suffers. The innovative research lives in academia, -trapped in languages that are rarely used for commercial development. +and so on -- are nowhere on this list, yet they are the targets of +most European academic research and innovation. Thus European +economic competitiveness suffers. The innovative research lives in +academia, trapped in languages that are rarely used for commercial +development. Of those more popular languages, two, Java and Visual Basic are proprietary. Sun Microsystems owns Java, and Microsoft owns Visual Basic. Any company which writes its software in Java or Visual Basic is at the mercy of these large American companies. And this is a -real, and not theoretical threat. In 2002, Microsoft announced that -it would no longer be supporting Visual Basic 6.0 after the year 2005. +real, not theoretical, threat. In 2002, Microsoft announced that it +would no longer be supporting Visual Basic 6.0 after the year 2005. All Visual Basic Developers have been told to convert their code to run under Microsoft's new .NET framework. In 2001 Microsoft immediately stopped supporting its Visual J++ language, meant to be a @@ -79,7 +80,7 @@ implementation was execution speed. Python is a dynamically-typed, late-binding, interpreted language. While this proved to provide extremely productive development environments, execution speed -sometimes is not fast enough. Today, Optimisation of high-level +sometimes is not fast enough. Today, optimisation of high-level languages must be done at run-time, and is notoriously more difficult to optimise than statically typed, early-binding compiled languages such as C or C++. @@ -158,8 +159,8 @@ his mails that having a minimized version of Python could provide a new base to advance the language. Nevertheless, his extensions to the language proved to be useful for companies who needed a way to have -millions of active objects and he had a branch of C-Python to make -this possible. +millions of active objects and he had a branch of CPython to make this +possible. Some organizational experience ------------------------------ @@ -202,7 +203,7 @@ Sprint and helped set up a state-of-the-art open-source development environment. With his 12-year experience of setting up and leading a SME-company which is one of the worldwide leaders in -Print-Preprocessing technology he helped organise the development and +print-preprocessing technology he helped organise the development and net-connectivity for the various web services needed by the PyPy developers. @@ -222,7 +223,7 @@ would meet for Sprints in their own private time, and work on the prototype. -After the first Sprint at TrillkeGut in Hildesheim, Jacob Hall?n and +After the first Sprint at Trillke-Gut in Hildesheim, Jacob Hall?n and Laura Creighton organized the next Sprint at AB Strakt in Gothenburg, Sweden. At this Sprint, Samuele Pedroni, the lead developer of Jython (a tight industrial-use integration of Java and Python), joined the @@ -238,11 +239,11 @@ The third sprint was organized by interested developers in Belgium at -the University in Louvain-La-Neuve and held June 20-24. We invited -Guido van Rossum, the inventor of Python to attend. He not only -attended the Belgium sprint but announced a few days later at the +the University in Louvain-La-Neuve, Belgium and held June 20-24. We +invited Guido van Rossum, the inventor of Python to attend. He not +only attended the Belgium sprint but announced a few days later at the EuroPython conference that PyPy had a high priority on his list of -'dreams he hoped would come true' and he enjoyed Sprinting with us a +'dreams he hoped would come true' and he enjoyed sprinting with us a lot. By the end of the third one-week sprint at the University in @@ -259,16 +260,16 @@ No work had been done on actually optimising the code so it ran around 30,000 times slower than the existing CPython-implementation but this -was to be expected from the start. Nevertheless, for a proof of -concept, approximately four weeks work total for a group of about a -dozen people, it was clearly a success. +was expected from the start. Nevertheless, for a proof of concept, +approximately four weeks work total for a group of about a dozen +people, it was clearly a success. It was time to look for funding. < While some people suggested to apply to DARPA for funding we realized that most of us were rooted in Europe and it would make more sense to -look for european possibilities. +look for European possibilities. I don't think that this is wise. Why remind them that we could get somebody else to fund it, maybe? Besides, everybody on the list of From mwh at codespeak.net Wed Sep 17 16:44:41 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 16:44:41 +0200 (MEST) Subject: [pypy-svn] rev 1342 - pypy/trunk/doc/funding Message-ID: <20030917144441.9CD545A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 16:44:41 2003 New Revision: 1342 Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt Log: Wrap lines at 70 columns. Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_relevance.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_relevance.txt Wed Sep 17 16:44:41 2003 @@ -2,22 +2,23 @@ Describe the manner in which the proposed project's goals address the scientific, technical, wider societal and policy objectives of the IST -Priority in the areas concerned. +Priority in the areas concerned. (Recommended length ? three pages) ------------------------------------------- -PyPy is a precise match for IST-2002-2.3.2.3 - Open development platforms -for software and services. Its stated objective is as follows: +PyPy is a precise match for IST-2002-2.3.2.3 - Open development +platforms for software and services. Its stated objective is as +follows: To build open development and run-time environments for software and services providing the next generation of methodologies, - interoperable middleware and tools to support developers - through all - phases of the software life-cycle, from requirements analysis until - deployment and maintenance - in the production of networked and - distributed software systems and services, embedded software and - value-added user services. This will enable the development of future - software engineering methods and tools. + interoperable middleware and tools to support developers - through + all phases of the software life-cycle, from requirements analysis + until deployment and maintenance - in the production of networked + and distributed software systems and services, embedded software + and value-added user services. This will enable the development of + future software engineering methods and tools. The PyPy project aims to build an flexible and fast implementation of the Open Source programming language Python written in itself. Python @@ -31,90 +32,92 @@ Priority is to be given to projects in which 'strong industrial users join forces with software and service - suppliers in building common platforms with support of academic - research partners'. + suppliers in building common platforms with support of + academic research partners'. This Project is a collaboration between Academic Researchers, and -Software and Service providers to produce an outcome desired by -all industrial users. ASK_STOCKHOLM 'what's an Industrial User?' -Is AB Strakt one? Large companies, who are not software providers, -such as Bang and Bang & Olufsen, the Danish maker of high-end audio -equipment, http://www.bang-olufsen.com/ have expressed interest in -using PyPy once it is developed. +Software and Service providers to produce an outcome desired by all +industrial users. ASK_STOCKHOLM 'what's an Industrial User?' Is AB +Strakt one? Large companies, who are not software providers, such as +Bang and Bang & Olufsen, the Danish maker of high-end audio equipment, +http://www.bang-olufsen.com/ have expressed interest in using PyPy +once it is developed. Moreover, our project is relevant to every focus objective. Focus is on: a) High level methods and concepts (esp. at requirements and - architectural level) for system design, development and - integration, addressing non-functional aspects, complexity, + architectural level) for system design, development and + integration, addressing non-functional aspects, complexity, autonomy and composability. -PyPy will advance the 'state-of-the-art' in specializing compilers, with -a new architecture for Object Libraries that provide for radical -improvements in (the management of) complexity, autonomy (self-hosting), -and composability (seamless modularity). +PyPy will advance the 'state-of-the-art' in specializing compilers, +with a new architecture for Object Libraries that provide for radical +improvements in (the management of) complexity, autonomy +(self-hosting), and composability (seamless modularity). ASK_STOCKHOLM DO I HAVE TO EXPLAIN THIS MORE? - b) Open and modular development environments, enabling - flexibility and extensibility with new or sector-specific - tools (e.g. intelligent distributed decision support), - supporting different adaptable development processes and - methodologies and ensuring consistency and traceability + b) Open and modular development environments, enabling + flexibility and extensibility with new or sector-specific + tools (e.g. intelligent distributed decision support), + supporting different adaptable development processes and + methodologies and ensuring consistency and traceability across the development lifecycle. Flexibility is one of the Primary goals of PyPy. Our new architecture will be the basis of a host of more flexible and extensible ways to -create software products. PyPy will, of course, be released as +create software products. PyPy will, of course, be released as Free/Open software, under the MIT license. FOOTNOTE: Stick MIT License in here. -ASK_STOCKHOLM -- Am I correct? They want the thing we made to be -Open and Modular? Or are they asking that our own way of working uses -Open and Modular techniques? We do this as well .... - - c) Light/agile methodologies and adaptive workflow providing - a dynamic and adaptive environment, suitable for - co-operative and distributed development. +ASK_STOCKHOLM -- Am I correct? They want the thing we made to be Open +and Modular? Or are they asking that our own way of working uses Open +and Modular techniques? We do this as well .... + + c) Light/agile methodologies and adaptive workflow + providing a dynamic and adaptive environment, suitable + for co-operative and distributed development. Each member of the group is committed to Agile Methodolgies. Some of us have experience with Crystal and others with XP, [FIXME add URLS and see if we have any more trained people on the team]two of the more popular ones. Moreover, since we have been active for many years in the Open Source community, we are already used to co-operating with -others in a distributed development process. +others in a distributed development process. ASK_STOCKHOLM -- is this what they want, or more like this? AB Strakt is a young Swedish software company whose main product, CAPS -is an adaptive workflow framework. A faster Python would seamlessly improve -this product, and those applications which are based upon it. +is an adaptive workflow framework. A faster Python would seamlessly +improve this product, and those applications which are based upon it. -[FIXME: if we want this bit, we want to refere to the AB Strakt Appendex, that - says that we are a cool company, and that we do workflow and procurement, - and that we are responsible -- blah blah blah. I'll boast about - Strakt later, or see if I can steal some boasts from marketing.] +[FIXME: if we want this bit, we want to refere to the AB Strakt + Appendex, that says that we are a cool company, and that we do + workflow and procurement, and that we are responsible -- blah blah + blah. I'll boast about Strakt later, or see if I can steal some + boasts from marketing.] d) Open platforms, middleware and languages supporting standards for interoperability, composability and - integration. (incl. e.g. P2P, GRID, autonomy, agents, - dynamic adaptability and evolvability, context awareness, - customer profiles). Open source middleware layers can - facilitate rapid and broad adoption. - -PyPy is an Open Language. ASK_STOCKHOLM. So they have just asked us to -make them a languaqe? Or they just want us to use a language to make -what we are making? - - In addition, related foundational research, to be implemented by - Specific Targeted Research Projects and Coordination Actions, should - focus on fundamental design concepts, systematisation of domain - specifications, concurrency, distribution and timing, formal and - quantitative analysis and testing tools, and future database and - information system concepts. + integration. (incl. e.g. P2P, GRID, autonomy, agents, + dynamic adaptability and evolvability, context + awareness, customer profiles). Open source middleware + layers can facilitate rapid and broad adoption. + +PyPy is an Open Language. ASK_STOCKHOLM. So they have just asked us +to make them a languaqe? Or they just want us to use a language to +make what we are making? + + In addition, related foundational research, to be implemented + by Specific Targeted Research Projects and Coordination + Actions, should focus on fundamental design concepts, + systematisation of domain specifications, concurrency, + distribution and timing, formal and quantitative analysis and + testing tools, and future database and information system + concepts. ASK_STOCKHOLM PyPy is Applied Research. When they say 'foundational research' do they want 'foundational applied research', in which case @@ -125,16 +128,13 @@ 'it is nice to learn stuff'? Work should where appropriate, enhance and complement work - implemented under EUREKA/ITEA and in software initiatives at member - and associated state level. The IST programme will seek active - co-operation with ITEA in software intensive systems. - -ASK_STOCKHOLM I don't know how to find out which work implemented under -EUREKA/ITEA was implemented using Python. I do know of 2 prior IST -projects. FIXME: add URL for ASWAD. A faster Python will enhance -any project done in Python. Also, if this means that EUREKA/ITEA -has more pots of money that they would like to hand us, we'd love -to take it. - - - + implemented under EUREKA/ITEA and in software initiatives at + member and associated state level. The IST programme will seek + active co-operation with ITEA in software intensive systems. + +ASK_STOCKHOLM I don't know how to find out which work implemented +under EUREKA/ITEA was implemented using Python. I do know of 2 prior +IST projects. FIXME: add URL for ASWAD. A faster Python will enhance +any project done in Python. Also, if this means that EUREKA/ITEA has +more pots of money that they would like to hand us, we'd love to take +it. From mwh at codespeak.net Wed Sep 17 16:45:58 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 16:45:58 +0200 (MEST) Subject: [pypy-svn] rev 1343 - pypy/trunk/doc/funding Message-ID: <20030917144558.BB3BF5A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 16:45:58 2003 New Revision: 1343 Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt Log: speeling (who is it that thinks appendix is spelt with two e's?) Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_relevance.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_relevance.txt Wed Sep 17 16:45:58 2003 @@ -81,7 +81,7 @@ providing a dynamic and adaptive environment, suitable for co-operative and distributed development. -Each member of the group is committed to Agile Methodolgies. Some of +Each member of the group is committed to Agile Methodologies. Some of us have experience with Crystal and others with XP, [FIXME add URLS and see if we have any more trained people on the team]two of the more popular ones. Moreover, since we have been active for many years in @@ -94,8 +94,8 @@ is an adaptive workflow framework. A faster Python would seamlessly improve this product, and those applications which are based upon it. -[FIXME: if we want this bit, we want to refere to the AB Strakt - Appendex, that says that we are a cool company, and that we do +[FIXME: if we want this bit, we want to refer to the AB Strakt + Appendix, that says that we are a cool company, and that we do workflow and procurement, and that we are responsible -- blah blah blah. I'll boast about Strakt later, or see if I can steal some boasts from marketing.] @@ -108,7 +108,7 @@ layers can facilitate rapid and broad adoption. PyPy is an Open Language. ASK_STOCKHOLM. So they have just asked us -to make them a languaqe? Or they just want us to use a language to +to make them a language? Or they just want us to use a language to make what we are making? In addition, related foundational research, to be implemented From mwh at codespeak.net Wed Sep 17 16:49:47 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 16:49:47 +0200 (MEST) Subject: [pypy-svn] rev 1344 - pypy/trunk/doc/funding Message-ID: <20030917144947.AC1BF5A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 16:49:47 2003 New Revision: 1344 Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt Log: Just a few De-Capitalizations for this one. Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_relevance.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_relevance.txt Wed Sep 17 16:49:47 2003 @@ -35,8 +35,8 @@ suppliers in building common platforms with support of academic research partners'. -This Project is a collaboration between Academic Researchers, and -Software and Service providers to produce an outcome desired by all +This project is a collaboration between academic researchers, and +software and service providers to produce an outcome desired by all industrial users. ASK_STOCKHOLM 'what's an Industrial User?' Is AB Strakt one? Large companies, who are not software providers, such as Bang and Bang & Olufsen, the Danish maker of high-end audio equipment, @@ -53,7 +53,7 @@ autonomy and composability. PyPy will advance the 'state-of-the-art' in specializing compilers, -with a new architecture for Object Libraries that provide for radical +with a new architecture for object libraries that provide for radical improvements in (the management of) complexity, autonomy (self-hosting), and composability (seamless modularity). @@ -66,10 +66,10 @@ methodologies and ensuring consistency and traceability across the development lifecycle. -Flexibility is one of the Primary goals of PyPy. Our new architecture +Flexibility is one of the primary goals of PyPy. Our new architecture will be the basis of a host of more flexible and extensible ways to create software products. PyPy will, of course, be released as -Free/Open software, under the MIT license. +Free/Open Source software, under the MIT license. FOOTNOTE: Stick MIT License in here. From mwh at codespeak.net Wed Sep 17 16:51:50 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 16:51:50 +0200 (MEST) Subject: [pypy-svn] rev 1345 - pypy/trunk/doc/funding Message-ID: <20030917145150.16E565A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 16:51:50 2003 New Revision: 1345 Modified: pypy/trunk/doc/funding/B3.0_impact.txt Log: line-wrapping (i left the quoted email alone) Modified: pypy/trunk/doc/funding/B3.0_impact.txt ============================================================================== --- pypy/trunk/doc/funding/B3.0_impact.txt (original) +++ pypy/trunk/doc/funding/B3.0_impact.txt Wed Sep 17 16:51:50 2003 @@ -14,20 +14,20 @@ is the acknowledged world leader in handheld, mobile, and embedded devices. But people working in such industries have long desired a high level language with a very small footprint. They wish to use -only the language features which they need. But modern languages -are chock full of features designed for the developer to use when +only the language features which they need. But modern languages are +chock full of features designed for the developer to use when developing software. The new innovative concept of Object Spaces, pioneered by PyPy makes the construction of tiny Object Spaces, -suitable for running on the smallest devices straight-forward. -A Python with greater speed will seemlessly improve the offerings of +suitable for running on the smallest devices straight-forward. A +Python with greater speed will seemlessly improve the offerings of those European Companies who develop using Python. These two reasons make the Python Business Forum quite excited about PyPy. Morever, a great many companies, in deciding what language to develop in, reject Very High Level Languages, despite their known advantages for programmer productivity and code-reuse and maintainablity because -the code produced simply does not run fast enough. If we give them -a *fast* VHLL, they will switch. +the code produced simply does not run fast enough. If we give them a +*fast* VHLL, they will switch. Furthermore, one of the greatest threats to European competitiveness is its dependence upon Proprietary Closed Source Software, mostly made @@ -57,11 +57,11 @@ The second threat Closed Source makes to European competitiveness is more insidious, and more long term. A good workman knows his tools. This is much more than the theoretical knowledge of how his tools -ought to work, according to principles learned in school. The way -car mechanics know how cars work is distinctly different from the -what you would know if you had attended classes on 'the principles of -the internal combustion engine', let alone what you need to know to -just drive the thing. +ought to work, according to principles learned in school. The way car +mechanics know how cars work is distinctly different from the what you +would know if you had attended classes on 'the principles of the +internal combustion engine', let alone what you need to know to just +drive the thing. Right now, in Europe, we don't have enough of the software equivalents of car-mechanics. And most of them live in academia, where they know @@ -71,10 +71,11 @@ family car. It is not as if there is a shortage of people who would be interested -in learning such things, if the source were made available. Many people -have taken this step by learning how CPython does its stuff. But still -there is a barrier. If you want to know how C Python does things, -you need to learn C. C is a notoriously difficult language to learn. +in learning such things, if the source were made available. Many +people have taken this step by learning how CPython does its stuff. +But still there is a barrier. If you want to know how C Python does +things, you need to learn C. C is a notoriously difficult language to +learn. But let me quote from an article posted to the Python-in-Education mailing list. @@ -134,11 +135,12 @@ http://mail.python.org/mailman/listinfo/edu-sig ------------------------------------------------------ -Here is somebody who is hoping we can give him a language he can understand. -Python already is an excellent teaching language. PyPy will be a better one. +Here is somebody who is hoping we can give him a language he can +understand. Python already is an excellent teaching language. PyPy +will be a better one. -This project has to be done at the European or the International level. -That's where we all live. +This project has to be done at the European or the International +level. That's where we all live. Since Education is a primary goal of the project, we will take every opportunity to disseminate PyPy. The source will always be freely @@ -150,15 +152,15 @@ ASK STOCKHOLM -- will the EU pay for us to go to conferences? even ones we were going to attend anyway? -These talks will create interest as well as teach -techniques. We will submit a PEP and push to get PyPy made the reference -implementation of the Python programming language. We will continue -to discuss PyPy on our own mailing lists, as well as other Python -mailing lists such as the Python-in-Education list, and the Usenet -Newsgroup comp.lang.python. We already have an #irc channel, -irc.freenode.net where live online discussions happen, and where -we communicate with each other while we are apart. Beyond that -- -we are willing to take any actions the EU would like to fund. +These talks will create interest as well as teach techniques. We will +submit a PEP and push to get PyPy made the reference implementation of +the Python programming language. We will continue to discuss PyPy on +our own mailing lists, as well as other Python mailing lists such as +the Python-in-Education list, and the Usenet Newsgroup +comp.lang.python. We already have an #irc channel, irc.freenode.net +where live online discussions happen, and where we communicate with +each other while we are apart. Beyond that -- we are willing to take +any actions the EU would like to fund. Ask Stockholm We think you get no money for this in a STREP. True? From mwh at codespeak.net Wed Sep 17 16:59:04 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 16:59:04 +0200 (MEST) Subject: [pypy-svn] rev 1346 - pypy/trunk/doc/funding Message-ID: <20030917145904.AFE805A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 16:59:04 2003 New Revision: 1346 Modified: pypy/trunk/doc/funding/B3.0_impact.txt Log: speeling Modified: pypy/trunk/doc/funding/B3.0_impact.txt ============================================================================== --- pypy/trunk/doc/funding/B3.0_impact.txt (original) +++ pypy/trunk/doc/funding/B3.0_impact.txt Wed Sep 17 16:59:04 2003 @@ -19,13 +19,13 @@ developing software. The new innovative concept of Object Spaces, pioneered by PyPy makes the construction of tiny Object Spaces, suitable for running on the smallest devices straight-forward. A -Python with greater speed will seemlessly improve the offerings of +Python with greater speed will seamlessly improve the offerings of those European Companies who develop using Python. These two reasons make the Python Business Forum quite excited about PyPy. -Morever, a great many companies, in deciding what language to develop +Moreover, a great many companies, in deciding what language to develop in, reject Very High Level Languages, despite their known advantages -for programmer productivity and code-reuse and maintainablity because +for programmer productivity and code-reuse and maintainability because the code produced simply does not run fast enough. If we give them a *fast* VHLL, they will switch. @@ -57,7 +57,7 @@ The second threat Closed Source makes to European competitiveness is more insidious, and more long term. A good workman knows his tools. This is much more than the theoretical knowledge of how his tools -ought to work, according to principles learned in school. The way car +ought to work, according to principles learnt in school. The way car mechanics know how cars work is distinctly different from the what you would know if you had attended classes on 'the principles of the internal combustion engine', let alone what you need to know to just @@ -97,9 +97,9 @@ For whatever reason, the complex built_in and the cmath module, implemented in Python, are part of the early pypy codebase. As I had been spending some -time in the complex realm with PyGeo - a simple verions of the complex +time in the complex realm with PyGeo - a simple version of the complex realm, as these things go - Laura's post gave me the impetus to try to -plugin the pypy implmentations. +plugin the pypy implementations. Only got stuck on the typing issue. My code tests for instance(object,complex). The pypy complexobject, unadorned, is a class - @@ -117,11 +117,11 @@ Presumably - the point is - there will be less cases where the right approach would be an extension module in C or C++, and a sense of -fundamental compromise sould one not be equipped to go there. Many -thousands of folks - using VB and the like - already do involed, highly -performing realworld applications and make nice livings doing so, without -being equipped to do C. I am thinking that pypy would put Python more -squarely in that "space". +fundamental compromise should one not be equipped to go there. Many +thousands of folks - using VB and the like - already do involved, +highly performing real world applications and make nice livings doing +so, without being equipped to do C. I am thinking that pypy would put +Python more squarely in that "space". Is any of this so, or just hope? From mwh at codespeak.net Wed Sep 17 17:03:32 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 17:03:32 +0200 (MEST) Subject: [pypy-svn] rev 1347 - pypy/trunk/doc/funding Message-ID: <20030917150332.2CEF15A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 17:03:31 2003 New Revision: 1347 Modified: pypy/trunk/doc/funding/B3.0_impact.txt Log: nits Modified: pypy/trunk/doc/funding/B3.0_impact.txt ============================================================================== --- pypy/trunk/doc/funding/B3.0_impact.txt (original) +++ pypy/trunk/doc/funding/B3.0_impact.txt Wed Sep 17 17:03:31 2003 @@ -30,13 +30,13 @@ *fast* VHLL, they will switch. Furthermore, one of the greatest threats to European competitiveness -is its dependence upon Proprietary Closed Source Software, mostly made +is its dependence upon proprietary closed source software, mostly made in the United States. This is not only the matter of money being spent in the United States is money that is not being spent here, -although that effect matters as well. There are two more serious +although that affects matters as well. There are two more serious risks. The first is a threat in the present. -Any company which writes its software in a Proprietary, Closed Source +Any company which writes its software in a proprietary, closed source language is dependent upon its software provider. If you have a bug, you must wait for them to fix it. If this bug is not a high priority for them, you can wait a long time. If you have access to the source @@ -54,10 +54,10 @@ these decisions because they make business sense for Microsoft, regardless of the effects on European software developers. -The second threat Closed Source makes to European competitiveness is +The second threat closed source makes to European competitiveness is more insidious, and more long term. A good workman knows his tools. This is much more than the theoretical knowledge of how his tools -ought to work, according to principles learnt in school. The way car +ought to work, according to principles learned in school. The way car mechanics know how cars work is distinctly different from the what you would know if you had attended classes on 'the principles of the internal combustion engine', let alone what you need to know to just @@ -73,7 +73,7 @@ It is not as if there is a shortage of people who would be interested in learning such things, if the source were made available. Many people have taken this step by learning how CPython does its stuff. -But still there is a barrier. If you want to know how C Python does +But still there is a barrier. If you want to know how CPython does things, you need to learn C. C is a notoriously difficult language to learn. @@ -142,7 +142,7 @@ This project has to be done at the European or the International level. That's where we all live. -Since Education is a primary goal of the project, we will take every +Since education is a primary goal of the project, we will take every opportunity to disseminate PyPy. The source will always be freely available from our website. We will continue to give talks about PyPy at EuroPython, Python-UK, OSCON (the International Open Source @@ -157,10 +157,10 @@ the Python programming language. We will continue to discuss PyPy on our own mailing lists, as well as other Python mailing lists such as the Python-in-Education list, and the Usenet Newsgroup -comp.lang.python. We already have an #irc channel, irc.freenode.net -where live online discussions happen, and where we communicate with -each other while we are apart. Beyond that -- we are willing to take -any actions the EU would like to fund. +comp.lang.python. We already have an IRC channel, #pypy on +irc.freenode.net where live online discussions happen, and where we +communicate with each other while we are apart. Beyond that -- we are +willing to take any actions the EU would like to fund. Ask Stockholm We think you get no money for this in a STREP. True? From mwh at codespeak.net Wed Sep 17 17:09:03 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 17:09:03 +0200 (MEST) Subject: [pypy-svn] rev 1348 - pypy/trunk/doc/funding Message-ID: <20030917150903.7EC175A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 17:09:02 2003 New Revision: 1348 Modified: pypy/trunk/doc/funding/B3.1_standards.txt Log: speeling Modified: pypy/trunk/doc/funding/B3.1_standards.txt ============================================================================== --- pypy/trunk/doc/funding/B3.1_standards.txt (original) +++ pypy/trunk/doc/funding/B3.1_standards.txt Wed Sep 17 17:09:02 2003 @@ -1,4 +1,4 @@ -B.3.1 Contributions to standards +.3.1 Contributions to standards ----------------------------------- Describe contributions to national or international standards which @@ -9,18 +9,18 @@ There are currently two implementations of Python in common use. The first one, which we will call CPython, but which is what the world knows as Python, is a C implementation of the Python Programming -langauge which compiles to its own virtual machine. The second one is +language which compiles to its own virtual machine. The second one is Jython, a pure-Python implementation which compiles to the Java virtual machine. There is no ANSI standard or similar for Python. -Right now the defacto standard for the programming language is +Right now the de-facto standard for the programming language is 'whatever CPython does'. This is far from ideal, and questions arise, especially from the developers of Jython as to which CPython language behaviours are essential to the Python language itself, and which are mere accidents of this particular implementation. For example, the garbage-collection behavior of CPython is implemented -by reference-counting ensuring that an object is finalized as soon -as the last reference to it goes away. That would be extremely +by reference-counting ensuring that an object is finalized as soon as +the last reference to it goes away. That would be extremely inconvenient (close to impossible) to implement on standard Java Virtual Machines, which have a deliberately under-specified garbage collector (it can collect anything it pleases whenever it pleases...). @@ -52,12 +52,12 @@ before submitting it for review. A PEP that has not been discussed on python-list at python.org and/or python-dev at python.org will not be accepted. After the authors believe that the PEP is ready, they must -inform the PEP editors (currently Barry Warsaw and David Goodger) that +vinform the PEP editors (currently Barry Warsaw and David Goodger) that it is ready for review. -PEPs are reviewed by Guido van Rossum, the language author, and his chosen -consultants, who may accept or reject a PEP or send it back to the -author(s) for revision. +PEPs are reviewed by Guido van Rossum, the language author, and his +chosen consultants, who may accept or reject a PEP or send it back to +the author(s) for revision. If Guido van Rossum accepts the PEP, then its status is changed to 'Accepted'. If the reference implementation is not already complete, @@ -72,7 +72,7 @@ reference implementation of the Python language. The complete details of how to write a PEP are themselves an -Informational PEP -- PEP #1 in fact. Complete details are to -be found in this appendex: +Informational PEP -- PEP #1 in fact. Complete details are to be found +in this appendix: FIXME include http://www.python.org/peps/pep-0001.html From mwh at codespeak.net Wed Sep 17 17:12:46 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 17:12:46 +0200 (MEST) Subject: [pypy-svn] rev 1349 - pypy/trunk/doc/funding Message-ID: <20030917151246.D06715A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 17:12:45 2003 New Revision: 1349 Modified: pypy/trunk/doc/funding/B3.1_standards.txt Log: one more spelling error Modified: pypy/trunk/doc/funding/B3.1_standards.txt ============================================================================== --- pypy/trunk/doc/funding/B3.1_standards.txt (original) +++ pypy/trunk/doc/funding/B3.1_standards.txt Wed Sep 17 17:12:45 2003 @@ -52,7 +52,7 @@ before submitting it for review. A PEP that has not been discussed on python-list at python.org and/or python-dev at python.org will not be accepted. After the authors believe that the PEP is ready, they must -vinform the PEP editors (currently Barry Warsaw and David Goodger) that +inform the PEP editors (currently Barry Warsaw and David Goodger) that it is ready for review. PEPs are reviewed by Guido van Rossum, the language author, and his From mwh at codespeak.net Wed Sep 17 17:17:47 2003 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 17 Sep 2003 17:17:47 +0200 (MEST) Subject: [pypy-svn] rev 1350 - pypy/trunk/doc/funding Message-ID: <20030917151747.B38DD5A114@thoth.codespeak.net> Author: mwh Date: Wed Sep 17 17:17:47 2003 New Revision: 1350 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt Log: spelling Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Wed Sep 17 17:17:47 2003 @@ -25,7 +25,7 @@ in a distributed network, code generators optimized for number crunching on some processor architectures and re-writing numerical Python packages should be able to prove that PyPy is more than an - academical exercise but very apropriate for industrial strength + academical exercise but very appropriate for industrial strength applications. - Load Balancing in a Multi-Processor environment @@ -69,7 +69,7 @@ implement in WP01 into regular Python libraries. These ones should also work without PyPy, being just plain-Python replacements for existing CPython functionality. -This includes the bytecode compiler, which definately should +This includes the bytecode compiler, which definitely should become a regular Python program, instead of being built into the interpreter. @@ -107,7 +107,7 @@ Produce low-level code out of the data gathered in Part a). Again investigate how this is best done (AST-guided translation or -reverse-engeneering of the low-level control flow gathered by the +reverse-engineering of the low-level control flow gathered by the annotation object space). Compare different low-level environments that we could target (C, Pyrex, others?). @@ -172,10 +172,10 @@ WP06_: Several Object Implementations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Since the standard object space allows coexistance of different +Since the standard object space allows coexistence of different implementations of the same type, we can develop **several object implementations** . -We will develop heuristics to switch between diferent +We will develop heuristics to switch between different implementations during execution. The goal is to study the efficiency of different approaches, with the possibility to change the default implementation @@ -231,7 +231,7 @@ Both will be rewritten as a meta-component that hooks into the translator plus a dedicated run-time component (WP05_). -XXX turning the following into an application for embedded sstems. +XXX turning the following into an application for embedded systems. Low-level targets, tools and releases -------------------------------------------- @@ -266,7 +266,7 @@ environments. Develop in particular a version whose goal is to simulate the existing CPython interpreter to support legacy extension modules. Investigate if the PyPy core can make internal choices that are very - different from CPython's without sacrifying legacy extension modules + different from CPython's without sacrificing legacy extension modules compatibility. @@ -312,8 +312,8 @@ dependant from the available capabilities of the current code generator. The ability of special instructions and other extensions -should be probed using run-time checks. Psyho should be -extended to be aware of paralle instructions, and be +should be probed using run-time checks. Psyco should be +extended to be aware of parallel instructions, and be enabled to emit optimized code for that. @@ -336,8 +336,8 @@ WP1 continues throughout the duration of the project and is carried out by the project coordinator. It involves the collection and monitoring of monthly status reports, reporting to the EU, organising meetings -and maintaining an internal web site. The webiste will make heavy -use of colaborative tools like Wiki pages, a content management system, +and maintaining an internal web site. The website will make heavy +use of collaborative tools like Wiki pages, a content management system, and issue trackers. [[WP102]]: Project Documentation @@ -352,8 +352,8 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ PyPy's own development needs an infrastructure that must continuously be kept up-to-date and further developed. This includes maintenance -of the SubVersion package and tracking of its ongoing development, -maintenance of the SubVersion respository, extensions like automatic +of the Subversion package and tracking of its ongoing development, +maintenance of the Subversion repository, extensions like automatic documentation extraction, and change notification via mailing lists. From arigo at codespeak.net Wed Sep 17 17:39:17 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Sep 2003 17:39:17 +0200 (MEST) Subject: [pypy-svn] rev 1351 - pypy/branch/pypy-noappfile/pypy/objspace/std Message-ID: <20030917153917.08B235A145@thoth.codespeak.net> Author: arigo Date: Wed Sep 17 17:39:17 2003 New Revision: 1351 Modified: pypy/branch/pypy-noappfile/pypy/objspace/std/objspace.py Log: minor check Modified: pypy/branch/pypy-noappfile/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/pypy-noappfile/pypy/objspace/std/objspace.py (original) +++ pypy/branch/pypy-noappfile/pypy/objspace/std/objspace.py Wed Sep 17 17:39:17 2003 @@ -198,6 +198,9 @@ return self.w_None if isinstance(x, W_Object): raise TypeError, "attempt to wrap already wrapped object: %s"%(x,) + if isinstance(x, OperationError): + raise TypeError, ("attempt to wrap already wrapped exception: %s"% + (x,)) if isinstance(x, int): if isinstance(bool, type) and isinstance(x, bool): return self.newbool(x) From arigo at codespeak.net Wed Sep 17 18:08:51 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Sep 2003 18:08:51 +0200 (MEST) Subject: [pypy-svn] rev 1352 - pypy/branch/builtinrefactor/pypy/objspace/std Message-ID: <20030917160851.C95CC5A114@thoth.codespeak.net> Author: arigo Date: Wed Sep 17 18:08:51 2003 New Revision: 1352 Modified: pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py Log: minor check Modified: pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py Wed Sep 17 18:08:51 2003 @@ -175,6 +175,9 @@ return self.w_None if isinstance(x, W_Object): raise TypeError, "attempt to wrap already wrapped object: %s"%(x,) + if isinstance(x, OperationError): + raise TypeError, ("attempt to wrap already wrapped exception: %s"% + (x,)) if isinstance(x, int): if isinstance(bool, type) and isinstance(x, bool): return self.newbool(x) From arigo at codespeak.net Wed Sep 17 18:21:04 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Sep 2003 18:21:04 +0200 (MEST) Subject: [pypy-svn] rev 1353 - pypy/branch/builtinrefactor/pypy/objspace/std Message-ID: <20030917162104.73DC85A145@thoth.codespeak.net> Author: arigo Date: Wed Sep 17 18:21:03 2003 New Revision: 1353 Modified: pypy/branch/builtinrefactor/pypy/objspace/std/stringobject.py Log: one last hack to have string-% behave in stdobjspace Modified: pypy/branch/builtinrefactor/pypy/objspace/std/stringobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/stringobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/stringobject.py Wed Sep 17 18:21:03 2003 @@ -864,7 +864,16 @@ return space.wrap("".join(buf)) def mod_str_tuple(space, w_format, w_args): - raise NotImplementedError + # XXX implement me + format = space.unwrap(w_format) + args = space.unwrap(w_args) + try: + s = format % args + except TypeError, e: + raise OperationError(space.w_TypeError, space.wrap(str(e))) + except ValueError, e: + raise OperationError(space.w_ValueError, space.wrap(str(e))) + return space.wrap(s) def len__String(space, w_str): return space.wrap(len(space.unwrap(w_str))) @@ -956,7 +965,7 @@ return mod_str_tuple(space, w_str, space.newtuple([w_item])) def mod__String_Tuple(space, w_str, w_tuple): - return space.wrap(space.unwrap(w_str)%space.unwrap(w_tuple)) + return mod_str_tuple(space, w_str, w_tuple) # register all methods register_all(vars(), W_StringType) From arigo at codespeak.net Wed Sep 17 18:52:56 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Sep 2003 18:52:56 +0200 (MEST) Subject: [pypy-svn] rev 1354 - in pypy/branch/builtinrefactor/pypy/objspace/std: . test Message-ID: <20030917165256.71D935A145@thoth.codespeak.net> Author: arigo Date: Wed Sep 17 18:52:55 2003 New Revision: 1354 Modified: pypy/branch/builtinrefactor/pypy/objspace/std/boolobject.py pypy/branch/builtinrefactor/pypy/objspace/std/floatobject.py pypy/branch/builtinrefactor/pypy/objspace/std/multimethod.py pypy/branch/builtinrefactor/pypy/objspace/std/objectobject.py pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py pypy/branch/builtinrefactor/pypy/objspace/std/test/test_multimethod.py pypy/branch/builtinrefactor/pypy/objspace/std/userobject.py Log: Still some light refactoring of the multimethod mecanisms. Now we give enough static clues about the types to allow for a complete list of what-to-try-in-which-order to be built for each input argument types. This cleanly separates method selection from method call, as wished by mwh and Samuele. It should also speed up pypy a bit but not dramatically enough to notice I'm afraid. (This is a replay in the builtinrefactor branch of the trunk revision 1131.) Modified: pypy/branch/builtinrefactor/pypy/objspace/std/boolobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/boolobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/boolobject.py Wed Sep 17 18:52:55 2003 @@ -27,6 +27,7 @@ # to an .intval one def delegate__Bool(space, w_bool): return intobject.W_IntObject(space, int(w_bool.boolval)) +delegate__Bool.result_class = intobject.W_IntObject delegate__Bool.priority = PRIORITY_PARENT_TYPE Modified: pypy/branch/builtinrefactor/pypy/objspace/std/floatobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/floatobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/floatobject.py Wed Sep 17 18:52:55 2003 @@ -26,6 +26,7 @@ # int-to-float delegation def delegate__Int(space, w_intobj): return W_FloatObject(space, float(w_intobj.intval)) +delegate__Int.result_class = W_FloatObject delegate__Int.priority = PRIORITY_CHANGE_TYPE Modified: pypy/branch/builtinrefactor/pypy/objspace/std/multimethod.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/multimethod.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/multimethod.py Wed Sep 17 18:52:55 2003 @@ -38,31 +38,184 @@ self.operatorsymbol = operatorsymbol self.dispatch_table = {} self.cache_table = {} + self.cache_delegator_key = None def register(self, function, *types): - for type in types: - if (hasattr(type, 'dispatchclass') and - 'dispatchclass' not in type.__dict__): - raise error, ('looks like you forgot to call\n' - 'registerimplementation(%r)' % type) - # W_ANY can be used as a placeholder to dispatch on any value. functions = self.dispatch_table.setdefault(types, []) if function not in functions: functions.append(function) self.cache_table.clear() + def compile_calllist(self, argclasses, delegate): + """Compile a list of calls to try for the given classes of the + arguments. Return a function that should be called with the + actual arguments.""" + if delegate.key is not self.cache_delegator_key: + self.cache_table.clear() + self.cache_delegator_key = delegate.key + try: + return self.cache_table[argclasses] + except KeyError: + calllist = [] + self.internal_buildcalllist(argclasses, delegate, calllist) + result = self.internal_compilecalllist(argclasses, calllist) + self.cache_table[argclasses] = result + return result + + def internal_compilecalllist(self, argclasses, calllist): + """Translate a call list into the source of a Python function + which is optimized and doesn't do repeated conversions on the + same arguments.""" + if len(calllist) == 1: + fn, conversions = calllist[0] + if conversions.count([]) == len(conversions): + # no conversion, just calling a single function: return + # that function directly + return fn + + #print '**** compile **** ', self.operatorsymbol, [ + # t.__name__ for t in argclasses] + arglist = ['a%d'%i for i in range(len(argclasses))] + ['*extraargs'] + source = ['def do(space,%s):' % ','.join(arglist)] + converted = [{(): 'a%d'%i} for i in range(len(argclasses))] + + def make_conversion(argi, convlist): + if tuple(convlist) in converted[argi]: + return converted[argi][tuple(convlist)] + else: + prev = make_conversion(argi, convlist[:-1]) + new = '%s_%d' % (prev, len(converted[argi])) + fname = all_functions.setdefault(convlist[-1], + 'd%d' % len(all_functions)) + source.append(' %s = %s(space,%s)' % (new, fname, prev)) + converted[argi][tuple(convlist)] = new + return new + + all_functions = {} + has_firstfailure = False + for fn, conversions in calllist: + # make the required conversions + fname = all_functions.setdefault(fn, 'f%d' % len(all_functions)) + arglist = [make_conversion(i, conversions[i]) + for i in range(len(argclasses))] + ['*extraargs'] + source.append( ' try:') + source.append( ' return %s(space,%s)' % ( + fname, ','.join(arglist))) + if has_firstfailure: + source.append(' except FailedToImplement:') + else: + source.append(' except FailedToImplement, firstfailure:') + has_firstfailure = True + source.append( ' pass') + + # complete exhaustion + if has_firstfailure: + source.append(' raise firstfailure') + else: + source.append(' raise FailedToImplement()') + source.append('') + + # compile the function + glob = {'FailedToImplement': FailedToImplement} + for fn, fname in all_functions.items(): + glob[fname] = fn + #for key, value in glob.items(): + # print key, '=', value + #for line in source: + # print line + exec '\n'.join(source) in glob + return glob['do'] + + def internal_buildcalllist(self, argclasses, delegate, calllist): + """Build a list of calls to try for the given classes of the + arguments. The list contains 'calls' of the following form: + (function-to-call, list-of-list-of-converters) + The list of converters contains a list of converter functions per + argument, with [] meaning that no conversion is needed for + that argument.""" + # look for an exact match first + arity = self.arity + assert arity == len(argclasses) + dispatchclasses = tuple([(c,) for c in argclasses]) + choicelist = self.buildchoices(dispatchclasses) + seen_functions = {} + no_conversion = [[]] * arity + for signature, function in choicelist: + calllist.append((function, no_conversion)) + seen_functions[function] = 1 + + # proceed by expanding the last argument by delegation, step by step + # until no longer possible, and then the previous argument, and so on. + expanded_args = () + expanded_dispcls = () + + for argi in range(arity-1, -1, -1): + # growing tuple of dispatch classes we can delegate to + curdispcls = dispatchclasses[argi] + assert len(curdispcls) == 1 + # maps each dispatch class to a list of converters + curargs = {curdispcls[0]: []} + # reduce dispatchclasses to the arguments before this one + # (on which no delegation has been tried yet) + dispatchclasses = dispatchclasses[:argi] + no_conversion = no_conversion[:argi] + + while 1: + choicelist = delegate.buildchoices((curdispcls,)) + # the list is sorted by priority + progress = False + for (t,), function in choicelist: + if function is None: + # this marks a decrease in the priority. + # Don't try delegators with lower priority if + # we have already progressed. + if progress: + break + else: + assert hasattr(function, 'result_class'), ( + "delegator %r must have a result_class" % function) + nt = function.result_class + if nt not in curargs: + curdispcls += (nt,) + srcconvs = curargs[t] + if not getattr(function, 'trivial_delegation',False): + srcconvs = srcconvs + [function] + curargs[nt] = srcconvs + progress = True + else: + if not progress: + break # no progress, and delegators list exhausted + + # progress: try again to dispatch with this new set of types + choicelist = self.buildchoices( + dispatchclasses + (curdispcls,) + expanded_dispcls) + for signature, function in choicelist: + if function not in seen_functions: + seen_functions[function] = 1 + # collect arguments: arguments after position argi... + after_argi = [expanded_args[j][signature[j]] + for j in range(argi+1-arity, 0)] # nb. j<0 + # collect arguments: argument argi... + arg_argi = curargs[signature[argi]] + # collect all arguments + newargs = no_conversion + [arg_argi] + after_argi + # record the call + calllist.append((function, newargs)) + # end of while 1: try on delegating the same argument i + + # proceed to the next argument + expanded_args = (curargs,) + expanded_args + expanded_dispcls = (curdispcls,) + expanded_dispcls + def buildchoices(self, allowedtypes): """Build a list of all possible implementations we can dispatch to, sorted best-first, ignoring delegation.""" # 'types' is a tuple of tuples of classes, one tuple of classes per - # argument. (Delegation needs to call buildchoice() with than one - # class for a single argument.) - try: - result = self.cache_table[allowedtypes] # try from the cache first - except KeyError: - result = self.cache_table[allowedtypes] = [] - self.internal_buildchoices(allowedtypes, (), result) - self.postprocessresult(allowedtypes, result) + # argument. (After delegation, we need to call buildchoice() with + # more than one possible class for a single argument.) + result = [] + self.internal_buildchoices(allowedtypes, (), result) + self.postprocessresult(allowedtypes, result) #print self.operatorsymbol, allowedtypes, result # the result is a list a tuples (function, signature). return result @@ -129,29 +282,33 @@ def __init__(self): MultiMethod.__init__(self, 'delegate', 1, []) - + self.key = object() + + def register(self, function, *types): + AbstractMultiMethod.register(self, function, *types) + self.key = object() # change the key to force recomputation + def postprocessresult(self, allowedtypes, result): - by_priority = {} # classify delegators by priority - - # add delegation from a class to its parent classes + # add delegation from a class to the *first* immediate parent class arg1types, = allowedtypes - parenttypes = [] for t in arg1types: - parenttypes += list(t.__bases__) - if parenttypes: - def delegate_to_parent_classes(space, a, parenttypes=parenttypes): - return [(t, a) for t in parenttypes - if issubclass(a.dispatchclass, t)] - # hard-wire it at priority 0 - by_priority[0] = [((t,), delegate_to_parent_classes) - for t in arg1types] + if t.__bases__: + base = t.__bases__[0] + def delegate_to_parent_class(space, a): + return a + delegate_to_parent_class.trivial_delegation = True + delegate_to_parent_class.result_class = base + delegate_to_parent_class.priority = 0 + # hard-wire it at priority 0 + result.append(((t,), delegate_to_parent_class)) # sort the results in priority order, and insert None marks # between jumps in the priority values. Higher priority values # first. + by_priority = {} # classify delegators by priority for signature, function in result: assert hasattr(function, 'priority'), ( - "delegator function must have a priority") + "delegator %r must have a priority" % function) sublist = by_priority.setdefault(function.priority, []) sublist.append((signature, function)) delegators = by_priority.items() @@ -203,7 +360,7 @@ class BoundMultiMethod: - ASSERT_BASE_TYPE = None + #ASSERT_BASE_TYPE = None def __init__(self, space, multimethod): self.space = space @@ -220,7 +377,7 @@ raise OperationError(*e.args) else: # raise a TypeError for a FailedToImplement - initialtypes = [a.dispatchclass + initialtypes = [a.__class__ for a in args[:self.multimethod.arity]] if len(initialtypes) <= 1: plural = "" @@ -235,93 +392,10 @@ def perform_call(self, args): arity = self.multimethod.arity - extraargs = args[arity:] - - if self.ASSERT_BASE_TYPE: - for a in args[:arity]: - assert hasattr(a, 'dispatchclass'), ( - "multimethod '%s' call with non wrapped argument: %r" % - (self.multimethod.operatorsymbol, a)) - assert issubclass(a.dispatchclass, self.ASSERT_BASE_TYPE), ( - "multimethod '%s' call with non wrapped argument: %r" % - (self.multimethod.operatorsymbol, a)) - - # look for an exact match first - firstfailure = None - types = tuple([(a.dispatchclass,) for a in args]) - choicelist = self.multimethod.buildchoices(types) - for signature, function in choicelist: - try: - return function(self.space, *args) - except FailedToImplement, e: - # we got FailedToImplement, record the first such error - firstfailure = firstfailure or e - - seen_functions = {} - for signature, function in choicelist: - seen_functions[function] = 1 - - # proceed by expanding the last argument by delegation, step by step - # until no longer possible, and then the previous argument, and so on. - expanded_args = () - expanded_types = () - delegate = self.space.delegate.multimethod - - for argi in range(arity-1, -1, -1): - curtypes = types[argi] # growing tuple of types we can delegate to - assert len(curtypes) == 1 - curobjs = {curtypes[0]: args[argi]} # maps them to actual objects - args = args[:argi] # initial segments of arguments before this one - types = types[:argi] # same with types (no deleg tried on them yet) - while 1: - choicelist = delegate.buildchoices((curtypes,)) - # the list is sorted by priority - progress = False - for (t,), function in choicelist: - if function is None: - # this marks a decrease in the priority. - # Don't try delegators with lower priority if - # we have already progressed. - if progress: - break - else: - converted = function(self.space, curobjs[t]) - if not isinstance(converted, list): - converted = [(converted.dispatchclass, - converted)] - for t, a in converted: - if t not in curobjs: - curtypes += (t,) - curobjs[t] = a - progress = True - else: - if not progress: - break # no progress, and delegators list exhausted - - # progress: try again to dispatch with this new set of types - choicelist = self.multimethod.buildchoices( - types + (curtypes,) + expanded_types) - for signature, function in choicelist: - if function not in seen_functions: - seen_functions[function] = 1 - # collect arguments: arguments after position i... - tail = [expanded_args[j][signature[j]] - for j in range(argi+1-arity, 0)] # nb. j<0 - # argments before and up to position i... - newargs= args + (curobjs[signature[argi]],) + tuple(tail) - try: - return function(self.space, *newargs+extraargs) - except FailedToImplement, e: - # record the first FailedToImplement - firstfailure = firstfailure or e - # end of while 1: try on delegating the same argument i - - # proceed to the next argument - expanded_args = (curobjs,) + expanded_args - expanded_types = (curtypes,) + expanded_types - - # complete exhaustion - raise firstfailure or FailedToImplement() + argclasses = tuple([a.__class__ for a in args[:arity]]) + delegate = self.space.delegate.multimethod + fn = self.multimethod.compile_calllist(argclasses, delegate) + return fn(self.space, *args) def is_empty(self): return self.multimethod.is_empty() Modified: pypy/branch/builtinrefactor/pypy/objspace/std/objectobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/objectobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/objectobject.py Wed Sep 17 18:52:55 2003 @@ -15,6 +15,7 @@ # any-to-object delegation is quite trivial, because W_ObjectObject is. def delegate__ANY(space, w_obj): return W_ObjectObject(space) +delegate__ANY.result_class = W_ObjectObject delegate__ANY.priority = PRIORITY_PARENT_TYPE def object_init__Object_ANY_ANY(space, w_object, w_args, w_kwds): Modified: pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/objspace.py Wed Sep 17 18:52:55 2003 @@ -30,7 +30,6 @@ # it may be modified to take 'statictype' instead of requiring it to be # stored in 'implcls' itself assert issubclass(implcls, W_Object) - implcls.__dict__.setdefault('dispatchclass', implcls) ################################################################## Modified: pypy/branch/builtinrefactor/pypy/objspace/std/test/test_multimethod.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/test/test_multimethod.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/test/test_multimethod.py Wed Sep 17 18:52:55 2003 @@ -15,15 +15,17 @@ def from_y_to_x(space, yinstance): return X(yinstance) +from_y_to_x.result_class = X from_y_to_x.priority = 2 -def from_x_to_str_sometimes(space, xinstance): - if xinstance.value: - return w('!' + repr(xinstance.value)) - else: - return [] +def from_x_to_str(space, xinstance): + #if xinstance.value: + return w('!' + repr(xinstance.value)) + #else: + # return [] -from_x_to_str_sometimes.priority = 2 +from_x_to_str.result_class = str +from_x_to_str.priority = 2 class Y: @@ -65,24 +67,27 @@ add.register(add_int_any, int, object) delegate = DelegateMultiMethod() - delegate.register(from_y_to_x, Y) - delegate.register(from_x_to_str_sometimes, X) + delegate.register(from_y_to_x, Y) + delegate.register(from_x_to_str, X) def wrap(self, x): return '' % (x,) w_TypeError = 'w_TypeError' -def w(x, cache={}): - if type(x) in cache: - Stub = cache[type(x)] - else: - Stub = type(type(x))('%s_stub' % type(x).__name__, (type(x),), {}) - Stub.dispatchclass = Stub - cache[type(x)] = Stub - return Stub(x) +##def w(x, cache={}): +## if type(x) in cache: +## Stub = cache[type(x)] +## else: +## Stub = type(type(x))('%s_stub' % type(x).__name__, (type(x),), {}) +## Stub.dispatchclass = Stub +## cache[type(x)] = Stub +## return Stub(x) -X.dispatchclass = X -Y.dispatchclass = Y +##X.dispatchclass = X +##Y.dispatchclass = Y + +def w(x): + return x class TestMultiMethod(test.TestCase): @@ -127,10 +132,10 @@ space = self.space self.assertRaises(OperationError, space.add, w([3]), w(4)) self.assertRaises(OperationError, space.add, w(3.0), w('bla')) - self.assertRaises(OperationError, space.add, X(0), w("spam")) - self.assertRaises(OperationError, space.add, Y(666), w("egg")) + #self.assertRaises(OperationError, space.add, X(0), w("spam")) + #self.assertRaises(OperationError, space.add, Y(666), w("egg")) - def test_delegate_x_to_str_sometimes(self): + def test_delegate_x_to_str(self): space = self.space r = space.add(X(42), w("spam")) self.assertEquals(repr(r), "('add_string_string', '!42', 'spam')") Modified: pypy/branch/builtinrefactor/pypy/objspace/std/userobject.py ============================================================================== --- pypy/branch/builtinrefactor/pypy/objspace/std/userobject.py (original) +++ pypy/branch/builtinrefactor/pypy/objspace/std/userobject.py Wed Sep 17 18:52:55 2003 @@ -42,10 +42,17 @@ try: return _bltin_subclass_cache[cls] except: - subcls = type(W_Object)("%s_sub" % cls.__name__, (cls,), - {'statictype' : W_UserType, - 'bltbase' : cls, - 'dispatchclass': W_UserObject}) + subcls = type(W_Object)("%s_sub" % cls.__name__, (W_UserObject, cls), + {'statictype': W_UserType}) + + # W_UserObject-to-the-parent-builtin-type delegation + def delegate_to_parent_builtin(space, w_userobj): + return w_userobj + delegate_to_parent_builtin.trivial_delegation = True + delegate_to_parent_builtin.result_class = cls + delegate_to_parent_builtin.priority = PRIORITY_PARENT_TYPE + StdObjSpace.delegate.register(delegate_to_parent_builtin, subcls) + _bltin_subclass_cache[cls] = subcls return subcls @@ -73,14 +80,6 @@ return mostspecialized -# W_UserObject-to-the-parent-builtin-type delegation -# So far this is the only delegation that produces a result -# of a variable type. -def delegate__User(space, w_userobj): - return [(w_userobj.bltbase,w_userobj)] -delegate__User.priority = PRIORITY_PARENT_TYPE - - def type__User(space, w_userobj): return w_userobj.w_type From arigo at codespeak.net Wed Sep 17 18:53:15 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Sep 2003 18:53:15 +0200 (MEST) Subject: [pypy-svn] rev 1355 - pypy/branch/builtinrefactor/pypy/tool Message-ID: <20030917165315.951585A145@thoth.codespeak.net> Author: arigo Date: Wed Sep 17 18:53:15 2003 New Revision: 1355 Modified: pypy/branch/builtinrefactor/pypy/tool/fixeol Log: A version of fixeol that finds *.txt files as well, and can correct existing files on your working copy that have inconsistent line endings (This is a replay in the builtinrefactor branch of the trunk revision 1337.) Modified: pypy/branch/builtinrefactor/pypy/tool/fixeol ============================================================================== --- pypy/branch/builtinrefactor/pypy/tool/fixeol (original) +++ pypy/branch/builtinrefactor/pypy/tool/fixeol Wed Sep 17 18:53:15 2003 @@ -1,12 +1,56 @@ #! /usr/bin/env python -import os +import sys, os + +forbidden = range(0,32) +forbidden.remove(9) # tab +forbidden.remove(10) # lf +forbidden.remove(12) # ^L +forbidden.remove(13) # cr + +def looksbinary(data, forbidden = [chr(i) for i in forbidden]): + for c in forbidden: + if c in data: + return True + return False + +# hack to get the platform's native end-of-line format +f = open('@fixeol at tmp.txt', 'w') +print >> f +f.close() +f = open('@fixeol at tmp.txt', 'rb') +native_eol = f.read() +f.close() +os.unlink('@fixeol at tmp.txt') + +def binary2text(filename, native_eol = native_eol): + # convert to the platform's native end-of-line format if needed + f = open(filename, 'rb') + data = f.read() + f.close() + if looksbinary(data): + return False + original = data + data = data.replace('\r\n', '\n') + data = data.replace('\n', native_eol) + data = data.replace('\r', native_eol) + if data != original: + f = open(filename, 'wb') + f.write(data) + f.close() + return True + def fixpyfiles(ignored, dirname, fnames): numpyfiles = 0 for fname in fnames: - if fname.endswith('.py'): - # change end-of-line style of each .py file to 'native' + if fname.endswith('.py') or fname.endswith('.txt'): + # safety check to nail binary files fname = os.path.join(dirname, fname) + if not binary2text(fname): + print >> sys.stderr, "*** warning, looks like a binary file:", + print >> sys.stderr, fname + continue + # change end-of-line style of each .py and .txt file to 'native' os.system('svn propset svn:eol-style native %s' % fname) numpyfiles += 1 if numpyfiles: @@ -21,5 +65,7 @@ g.close() os.system('svn propset svn:ignore -F svn-ignore.tmp %s' % dirname) os.unlink('svn-ignore.tmp') + if '.svn' in fnames: + fnames.remove('.svn') os.path.walk(os.curdir, fixpyfiles, None) From hpk at codespeak.net Wed Sep 17 22:48:52 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 17 Sep 2003 22:48:52 +0200 (MEST) Subject: [pypy-svn] rev 1356 - pypy/trunk/src/pypy Message-ID: <20030917204852.9D1B15A63A@thoth.codespeak.net> Author: hpk Date: Wed Sep 17 22:48:51 2003 New Revision: 1356 Removed: pypy/trunk/src/pypy/ Log: everybody is asleep so we can remove the trunk to replace it with the branch in the next checkin. This way we keep the history of the changes during the builtinrefactor branch. From hpk at codespeak.net Wed Sep 17 22:58:53 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 17 Sep 2003 22:58:53 +0200 (MEST) Subject: [pypy-svn] rev 1357 - in pypy: branch/builtinrefactor/pypy trunk/src/pypy Message-ID: <20030917205853.7E9275ACB4@thoth.codespeak.net> Author: hpk Date: Wed Sep 17 22:58:52 2003 New Revision: 1357 Added: pypy/trunk/src/pypy/ - copied from rev 1356, pypy/branch/builtinrefactor/pypy/ Removed: pypy/branch/builtinrefactor/pypy/ Log: the final merge of the builtinrefactor branch into the trunk. See http://codespeak.net/pipermail/pypy-dev/2003q3/001012.html for in-depth discussion and description of what is new. From hpk at codespeak.net Wed Sep 17 22:58:53 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 17 Sep 2003 22:58:53 +0200 (MEST) Subject: [pypy-svn] rev 1357 - in pypy: branch/builtinrefactor/pypy trunk/src/pypy Message-ID: <20030917205853.6A9C25A63A@thoth.codespeak.net> Author: hpk Date: Wed Sep 17 22:58:52 2003 New Revision: 1357 Added: pypy/trunk/src/pypy/ - copied from rev 1356, pypy/branch/builtinrefactor/pypy/ Removed: pypy/branch/builtinrefactor/pypy/ Log: the final merge of the builtinrefactor branch into the trunk. See http://codespeak.net/pipermail/pypy-dev/2003q3/001012.html for in-depth discussion and description of what is new. From hpk at codespeak.net Wed Sep 17 23:04:16 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 17 Sep 2003 23:04:16 +0200 (MEST) Subject: [pypy-svn] rev 1359 - in pypy/branch: builtinrefactor pypy-noappfile Message-ID: <20030917210416.ED46B5A63A@thoth.codespeak.net> Author: hpk Date: Wed Sep 17 23:04:16 2003 New Revision: 1359 Removed: pypy/branch/builtinrefactor/ pypy/branch/pypy-noappfile/ Log: get rid of unused branches. From arigo at codespeak.net Thu Sep 18 14:57:16 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 18 Sep 2003 14:57:16 +0200 (MEST) Subject: [pypy-svn] rev 1360 - in pypy/trunk/src/pypy/interpreter: . test Message-ID: <20030918125716.65A265A63A@thoth.codespeak.net> Author: arigo Date: Thu Sep 18 14:57:15 2003 New Revision: 1360 Added: pypy/trunk/src/pypy/interpreter/test/test_eval.py pypy/trunk/src/pypy/interpreter/test/test_extmodule.py pypy/trunk/src/pypy/interpreter/test/test_module.py Removed: pypy/trunk/src/pypy/interpreter/test/test_executioncontext.py Modified: pypy/trunk/src/pypy/interpreter/baseobjspace.py pypy/trunk/src/pypy/interpreter/eval.py pypy/trunk/src/pypy/interpreter/module.py pypy/trunk/src/pypy/interpreter/test/test_objspace.py Log: adding unittests and fixing the bugs that these tests discover Modified: pypy/trunk/src/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/baseobjspace.py (original) +++ pypy/trunk/src/pypy/interpreter/baseobjspace.py Thu Sep 18 14:57:15 2003 @@ -21,7 +21,6 @@ def __init__(self): "Basic initialization of objects." - self.appfile_helpers = {} self.initialize() def make_builtins(self): @@ -79,21 +78,6 @@ "Factory function for execution contexts." return ExecutionContext(self) - def gethelper(self, applicationfile): - try: - helper = self.appfile_helpers[applicationfile] - except KeyError: - from appfile import AppHelper - helper = AppHelper(self.gethelperspace(), applicationfile) - self.appfile_helpers[applicationfile] = helper - return helper - - def gethelperspace(self): - # Return the object space to be used for executing helper code. - # A subclass may override this if it wants to use a different - # space to execute helpers (e.g. the annotating space) - return self - # Following is a friendly interface to common object space operations # that can be defined in term of more primitive ones. Subclasses # may also override specific functions for performance. Modified: pypy/trunk/src/pypy/interpreter/eval.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/eval.py (original) +++ pypy/trunk/src/pypy/interpreter/eval.py Thu Sep 18 14:57:15 2003 @@ -2,7 +2,7 @@ This module defines the abstract base classes that support execution: Code and Frame. """ - +from error import OperationError class Code(object): """A code is a compiled version of some source code. @@ -99,6 +99,7 @@ def locals2fast(self): # Copy values from self.w_locals to self.fastlocals_w + assert self.w_locals is not None varnames = self.code.getvarnames() for name, i in zip(varnames, range(len(self.fastlocals_w))): w_name = self.space.wrap(varnames[i]) Modified: pypy/trunk/src/pypy/interpreter/module.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/module.py (original) +++ pypy/trunk/src/pypy/interpreter/module.py Thu Sep 18 14:57:15 2003 @@ -28,3 +28,13 @@ def pypy_setattr(self, w_attr, w_value): self.space.setitem(self.w_dict, w_attr, w_value) + + def pypy_delattr(self, w_attr): + space = self.space + try: + space.delitem(self.w_dict, w_attr) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + # XXX fix error message + raise OperationError(space.w_AttributeError, w_attr) Added: pypy/trunk/src/pypy/interpreter/test/test_eval.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/interpreter/test/test_eval.py Thu Sep 18 14:57:15 2003 @@ -0,0 +1,58 @@ + +import autopath +from pypy.tool import test +from pypy.interpreter.eval import Frame, UNDEFINED +from pypy.interpreter.pycode import PyCode + + +class TestFrame(test.IntTestCase): + def setUp(self): + self.space = test.objspace() + def c(x, y, *args): + pass + code = PyCode()._from_code(c.func_code) + self.f = Frame(self.space, code, numlocals=5) + + def test_fast2locals(self): + w = self.space.wrap + self.f.fast2locals() + self.assertEqual_w(self.f.w_locals, self.space.newdict([])) + + self.f.fastlocals_w[0] = w(5) + self.f.fast2locals() + self.assertEqual_w(self.f.w_locals, self.space.newdict([ + (w('x'), w(5))])) + + self.f.fastlocals_w[2] = w(7) + self.f.fast2locals() + self.assertEqual_w(self.f.w_locals, self.space.newdict([ + (w('x'), w(5)), + (w('args'), w(7))])) + + def sameList(self, l1, l2): + self.assertEqual(len(l1), len(l2)) + for w_1, w_2 in zip(l1, l2): + self.failIf((w_1 is UNDEFINED) != (w_2 is UNDEFINED)) + if w_1 is not UNDEFINED: + self.assertEqual_w(w_1, w_2) + + def test_locals2fast(self): + w = self.space.wrap + self.f.w_locals = self.space.newdict([]) + self.f.locals2fast() + self.sameList(self.f.fastlocals_w, [UNDEFINED]*5) + + self.f.w_locals = self.space.newdict([ + (w('x'), w(5))]) + self.f.locals2fast() + self.sameList(self.f.fastlocals_w, [w(5)] + [UNDEFINED]*4) + + self.f.w_locals = self.space.newdict([ + (w('x'), w(5)), + (w('args'), w(7))]) + self.f.locals2fast() + self.sameList(self.f.fastlocals_w, [w(5), UNDEFINED, w(7), + UNDEFINED, UNDEFINED]) + +if __name__ == '__main__': + test.main() Deleted: /pypy/trunk/src/pypy/interpreter/test/test_executioncontext.py ============================================================================== --- /pypy/trunk/src/pypy/interpreter/test/test_executioncontext.py Thu Sep 18 14:57:15 2003 +++ (empty file) @@ -1,32 +0,0 @@ -import autopath - -from pypy.tool import test -from pypy.interpreter import baseobjspace, executioncontext - - - -class TestExecutionContext(test.TestCase): - - # - # XXX not sure what specifically this class used to test - # XXX turn this into a proper unit test - # - pass - -## def test_trivial1(self): -## # build frame -## space = test.objspace() -## ec = executioncontext.ExecutionContext(space) -## compile = space.builtin.compile -## bytecode = space.unwrap(compile(space.wrap('def f(x): return x+1'), -## space.wrap(''), -## space.wrap('exec'))).co_consts[0] -## w_globals = ec.make_standard_w_globals() -## w_locals = space.newdict([(space.wrap('x'), space.wrap(5))]) -## scopedcode = ScopedCode(space, bytecode, w_globals) -## w_output = scopedcode.eval_frame(w_locals) -## self.assertEquals(space.unwrap(w_output), 6) - - -if __name__ == '__main__': - test.main() Added: pypy/trunk/src/pypy/interpreter/test/test_extmodule.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/interpreter/test/test_extmodule.py Thu Sep 18 14:57:15 2003 @@ -0,0 +1,47 @@ + +import autopath +from pypy.tool import test +from pypy.interpreter.extmodule import ExtModule + + +class TestExtModule(test.IntTestCase): + def setUp(self): + self.space = test.objspace() + + class M(ExtModule): + __name__ = 'm' + constant = 678 + def app_egg(self, x): + return -x + def foo(self, w_spam): + return self.space.neg(w_spam) + self.m = M(self.space) + + def test_app_method(self): + w = self.space.wrap + self.assertEqual_w(self.m.egg(w(42)), w(-42)) + + def test_app_exported(self): + w = self.space.wrap + w_m = w(self.m) + w_result = self.space.call_method(w_m, 'egg', w(42)) + self.assertEqual_w(w_result, w(-42)) + + def test_interp_method(self): + w = self.space.wrap + self.assertEqual_w(self.m.app_foo(w(42)), w(-42)) + + def test_interp_exported(self): + w = self.space.wrap + w_m = w(self.m) + w_result = self.space.call_method(w_m, 'foo', w(42)) + self.assertEqual_w(w_result, w(-42)) + + def test_constant(self): + w = self.space.wrap + w_m = w(self.m) + self.assertEqual_w(self.space.getattr(w_m, w('constant')), w(678)) + + +if __name__ == '__main__': + test.main() Added: pypy/trunk/src/pypy/interpreter/test/test_module.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/interpreter/test/test_module.py Thu Sep 18 14:57:15 2003 @@ -0,0 +1,28 @@ + +import autopath +from pypy.tool import test +from pypy.interpreter.module import Module + + +class TestModule(test.IntTestCase): + def setUp(self): + self.space = test.objspace() + self.m = Module(self.space, self.space.wrap('m')) + + def test_name(self): + w = self.space.wrap + w_m = w(self.m) + self.assertEqual_w(self.space.getattr(w_m, w('__name__')), w('m')) + + def test_attr(self): + w = self.space.wrap + w_m = w(self.m) + self.space.setattr(w_m, w('x'), w(15)) + self.assertEqual_w(self.space.getattr(w_m, w('x')), w(15)) + self.space.delattr(w_m, w('x')) + self.assertRaises_w(self.space.w_AttributeError, + self.space.delattr, w_m, w('x')) + + +if __name__ == '__main__': + test.main() Modified: pypy/trunk/src/pypy/interpreter/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/interpreter/test/test_objspace.py Thu Sep 18 14:57:15 2003 @@ -4,7 +4,7 @@ # this test isn't so much to test that the objspace interface *works* # -- it's more to test that it's *there* -class TestStdObjectSpace(test.TestCase): +class TestObjSpace(test.TestCase): def setUp(self): self.space = test.objspace() @@ -58,5 +58,41 @@ self.failUnless(self.space.is_true(w_true)) self.failIf(self.space.is_true(w_false)) + def test_is_(self): + w_l = self.space.newlist([]) + w_m = self.space.newlist([]) + self.assertEqual(self.space.is_(w_l, w_l), self.space.w_True) + self.assertEqual(self.space.is_(w_l, w_m), self.space.w_False) + + def test_newbool(self): + self.assertEqual(self.space.newbool(0), self.space.w_False) + self.assertEqual(self.space.newbool(1), self.space.w_True) + + def test_unpackiterable(self): + w = self.space.wrap + l = [w(1), w(2), w(3), w(4)] + w_l = self.space.newlist(l) + self.assertEqual(self.space.unpackiterable(w_l), l) + self.assertEqual(self.space.unpackiterable(w_l, 4), l) + self.assertRaises(ValueError, self.space.unpackiterable, w_l, 3) + self.assertRaises(ValueError, self.space.unpackiterable, w_l, 5) + + def test_unpacktuple(self): + w = self.space.wrap + l = [w(1), w(2), w(3), w(4)] + w_l = self.space.newtuple(l) + self.assertEqual(self.space.unpacktuple(w_l), l) + self.assertEqual(self.space.unpacktuple(w_l, 4), l) + self.assertRaises(ValueError, self.space.unpacktuple, w_l, 3) + self.assertRaises(ValueError, self.space.unpacktuple, w_l, 5) + + def test_exception_match(self): + self.failUnless(self.space.exception_match(self.space.w_ValueError, + self.space.w_ValueError)) + self.failUnless(self.space.exception_match(self.space.w_IndexError, + self.space.w_LookupError)) + self.failIf(self.space.exception_match(self.space.w_ValueError, + self.space.w_LookupError)) + if __name__ == '__main__': test.main() From hpk at codespeak.net Thu Sep 18 19:57:51 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 18 Sep 2003 19:57:51 +0200 (MEST) Subject: [pypy-svn] rev 1361 - in pypy/trunk/src/pypy: interpreter module module/test Message-ID: <20030918175751.646D15A271@thoth.codespeak.net> Author: hpk Date: Thu Sep 18 19:57:48 2003 New Revision: 1361 Modified: pypy/trunk/src/pypy/interpreter/baseobjspace.py pypy/trunk/src/pypy/interpreter/error.py pypy/trunk/src/pypy/interpreter/extmodule.py pypy/trunk/src/pypy/module/builtin.py pypy/trunk/src/pypy/module/test/test_builtin.py Log: a first go at introducing classes (or arbitrary app-level constructs instead of just functions and basic types ) for ExtModules. Feel free to modify/enhance. M pypy/interpreter/error.py switched AUTO_DEBUG off because output is a bit annoying M pypy/interpreter/extmodule.py provided a mechanism to compile&exec source code for ExtModule's at app-level. prevent certain class-specific attributes from beeing sent to app-level (__metaclass__, __init__, __new__). M pypy/module/builtin.py added 'xrange' as a class which is acutally compiled&execed during i initilization of the builtin module/class. M pypy/interpreter/baseobjspace.py reordered the initilization of builtins a bit to allow for early execution (which needs space.builtin to be somewhat functional) M pypy/module/test/test_builtin.py added a test to check that the builtin 'xrange' is really a class/type Modified: pypy/trunk/src/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/baseobjspace.py (original) +++ pypy/trunk/src/pypy/interpreter/baseobjspace.py Thu Sep 18 19:57:48 2003 @@ -33,11 +33,13 @@ self.make_sys() from pypy.module import builtin + + # the builtins are iteratively initialized self.builtin = builtin.__builtin__(self) self.w_builtin = self.wrap(self.builtin) - #self.w_builtins = self.getattr(self.w_builtin, self.wrap("__dict__")) self.w_builtins = self.builtin.w_dict + # initialize with "bootstrap types" from objspace (e.g. w_None) for name, value in self.__dict__.items(): if name.startswith('w_'): name = name[2:] @@ -46,6 +48,11 @@ #print "setitem: space instance %-20s into builtins" % name self.setitem(self.w_builtins, self.wrap(name), value) + # only here can we add those builtins that require + # execution of source code -- because this requires + # an almost functional 'builtin' attribute on the space + self.builtin._initcompiledbuiltins() + self.sys._setmodule(self.w_builtin) def make_sys(self): Modified: pypy/trunk/src/pypy/interpreter/error.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/error.py (original) +++ pypy/trunk/src/pypy/interpreter/error.py Thu Sep 18 19:57:48 2003 @@ -1,6 +1,6 @@ import sys -AUTO_DEBUG = 1 +AUTO_DEBUG = 0 class PyPyError(Exception): Modified: pypy/trunk/src/pypy/interpreter/extmodule.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/extmodule.py (original) +++ pypy/trunk/src/pypy/interpreter/extmodule.py Thu Sep 18 19:57:48 2003 @@ -29,6 +29,9 @@ elif hasattr(value, '__get__'): continue # ignore CPython functions + # ignore tricky class-attrs we can't send from interp to app-level + if name in ('__metaclass__', '__init__', '__new__', ): + continue space.call_method(self.w_dict, 'setdefault', space.wrap(name), space.wrap(value)) @@ -36,3 +39,15 @@ def __initclass__(cls): gateway.exportall(RwDictProxy(cls)) # xxx() -> app_xxx() gateway.importall(RwDictProxy(cls)) # app_xxx() -> xxx() + + def _eval_app_source(self, sourcestring): + """ compile/execute a sourcestring in the applevel module dictionary """ + w = self.space.wrap + w_code = self.compile(w(sourcestring), w(''), w('exec')) + code = self.space.unwrap(w_code) + code.exec_code(self.space, self.w_dict, self.w_dict) + + # XXX do we actually want an interp-proxy to the app-level thing here? + # or no interp-level "mirror" at all? + co = compile(sourcestring, '','exec', 4096) + exec co in self.__dict__ Modified: pypy/trunk/src/pypy/module/builtin.py ============================================================================== --- pypy/trunk/src/pypy/module/builtin.py (original) +++ pypy/trunk/src/pypy/module/builtin.py Thu Sep 18 19:57:48 2003 @@ -19,6 +19,10 @@ open = cpy_builtin.open file = cpy_builtin.file + def _initcompiledbuiltins(self): + """ add 'compiled' builtins to app-level dict and interp-level """ + self._eval_app_source(xrange_appsource) + def _actframe(self, index=-1): return self.space.getexecutioncontext().framestack.items[index] @@ -379,31 +383,31 @@ except AttributeError: return False - - def app_xrange(self, start, stop=None, step=1): - class xrange: - def __init__(self, start, stop=None, step=1): - if stop is None: - self.start = 0 - self.stop = start +# source code for the builtin xrange-class +xrange_appsource = """if 1: + class xrange: + def __init__(self, start, stop=None, step=1): + if stop is None: + self.start = 0 + self.stop = start + else: + self.start = start + self.stop = stop + if step == 0: + raise ValueError, 'xrange() step-argument (arg 3) must not be zero' + self.step = step + + def __iter__(self): + def gen(self): + start, stop, step = self.start, self.stop, self.step + i = start + if step > 0: + while i < stop: + yield i + i+=step else: - self.start = start - self.stop = stop - if step == 0: - raise ValueError, 'xrange() step-argument (arg 3) must not be zero' - self.step = step - - def __iter__(self): - def gen(self): - start, stop, step = self.start, self.stop, self.step - i = start - if step > 0: - while i < stop: - yield i - i+=step - else: - while i > stop: - yield i - i+=step - return gen(self) - return xrange(start, stop, step) + while i > stop: + yield i + i+=step + return gen(self) +""" Modified: pypy/trunk/src/pypy/module/test/test_builtin.py ============================================================================== --- pypy/trunk/src/pypy/module/test/test_builtin.py (original) +++ pypy/trunk/src/pypy/module/test/test_builtin.py Thu Sep 18 19:57:48 2003 @@ -52,6 +52,9 @@ self.assertEquals(iter_x.next(), 3) self.assertRaises(StopIteration, iter_x.next) + def test_xrange_has_type_identity(self): + self.assertEquals(type(xrange(1)), type(xrange(1))) + def test_cmp(self): self.assertEquals(cmp(9,9), 0) self.assert_(cmp(0,9) < 0) @@ -88,6 +91,11 @@ import os os.remove(fn) + def test_xrange(self): + self.assert_(hasattr(self.space.builtin, 'xrange')) + self.assertEquals(self.space.builtin.xrange(3).stop, 3) + + if __name__ == '__main__': test.main() From arigo at codespeak.net Fri Sep 19 12:55:14 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 19 Sep 2003 12:55:14 +0200 (MEST) Subject: [pypy-svn] rev 1362 - pypy/trunk/doc/funding Message-ID: <20030919105514.B200E5A2A9@thoth.codespeak.net> Author: arigo Date: Fri Sep 19 12:55:13 2003 New Revision: 1362 Modified: pypy/trunk/doc/funding/B1.0_objectives.txt Log: First try at this one. Modified: pypy/trunk/doc/funding/B1.0_objectives.txt ============================================================================== --- pypy/trunk/doc/funding/B1.0_objectives.txt (original) +++ pypy/trunk/doc/funding/B1.0_objectives.txt Fri Sep 19 12:55:13 2003 @@ -1,9 +1,12 @@ -B.1 Scientific and technological objectives of the project and state of - the art In the event of inconsistency between information given in +B.1 Scientific and technological objectives of the project and state of the art +=============================================================================== + +""" + In the event of inconsistency between information given in Part A and that given in Part B, the Part A version will prevail -Describe in detail the proposed project's S&T objectives. The -objectives should be those achievable within the project, not through +Describe in detail the proposed project's Scientific & Technical objectives. +The objectives should be those achievable within the project, not through subsequent development, and should be stated in a measurable and verifiable form. The progress of the project work will be measured against these goals in later reviews and assessments. Describe the @@ -11,4 +14,243 @@ will enhance the state-of-the-art in that area. (Recommended length ? three pages) +""" + + +Introduction +------------ + +Interpretation is a common implementation technique for a wide range of +programming languages whose primary objective is not efficiency. +Interpreters are well-suited for small-to-medium languages, favouring a +simple and clear engine. + +As a language becomes popular, however, the costs inherent to this +approach are increasingly pointed out. These costs are twofold. Firstly, +no matter how much effort is made to optimize the core components, the +remaining interpretative overhead is large, both in term of processor +cycles and memory consumption. Secondly, interpreters are notoriously +difficult to factor into relatively independent components, which makes +structural changes to a large interpreter cumbersome. While the +efficiency costs are well-known, the costs inherent to the lack +flexibility [REF] are often underestimated by the community of language +users. In practice, what often occurs for newly popular languages is +that the user base creates pressure about the efficiency issue and +motivates a group of developers to write a native Just In Time (JIT) +compiler, adding a significant amount of code and complexity and further +impairing the flexibility. + + +The flexibility goal +-------------------- + +The flexibility problem is pernicious. It affects almost all aspects of +the language implementation: + +* user pressure tends to shape interpreters towards performance at the +expense of other aspects, like small footprints and code simplicity. + +* there are a lot of other trade-offs; experimenting with different +solutions can be close to impossible for aspects that affect the +interpreter globally. These aspects are usually frozen in early design +decisions (memory management, object model, multithreading,...). + +* optimization drives the interpreter code base, an error-prone and +scope-limiting process. It is impossible to reuse the code base for +implementation on new, non-C platforms; you need to start from scratch. + +Moreover, conflicting with performance and manageable complexity, there +is a large number of aspects and features that users would like to see +integrated in the interpreter (or would even do so themselves, if it +were possible at all). These features are well-established as middleware +tools that can be accessed from common languages, but would benefit a +lot from direct support from the language implementation -- which they +generally only have in a couple of experimental languages. Some +examples: + +* distributed execution (SMP or Networked) +* persistency +* FIXME list our application-level goals here + +The first major goal of the PyPy project is to produce an interpreter +with flexibility in mind, and put this flexibility to good use. More +specifically, our objectives are: + +1. to create a implementation of the Python language in Python. + +2. to implement a subset of the above-mentioned features (FIXME which +ones?) as an immediately useful proof-of-concept. + +3. to allow third-parties to implement any other interpreter-level feature +more easily than they would in a traditional interpreter. + +The main building piece behind the proposed flexibility is -- besides +using a high-level language in the first place -- the concept of Object +Space, which captures at the level of the interpreter the semantics of +individual operations between objects. The interpreter's main dispatch +loop handles control flow and supporting data structures (frame stack, +bytecode objects...); each individual operation is dispatched to the +object space. Object spaces are pluggable, which means that we can very +easily replace an object space implementing the normal Python semantics +with another one doing different things, like accessing remote objects +over a network, keeping logs, measuring performance or doing persistence +-- all things that are better implemented in the interpreter itself, so +that they can be fully transparent to the user of the language. + +More theoretically, replacing an object space with another one +performing abstract operations turns our interpreter into an abstract +interpreter at no effort. Such a use will be shown below. + + +The performance goal, phase 1 +----------------------------- + +Flexibility is often associated with poor performances, although +theoretically, high-level languages are more expressive and thus are +open to a wider range of optimization than low-level languages, and +should then outperform them. Of course this trend is not exactly +confirmed in practice so far; the only examples that one could give are +derived from bad or buggy low-level algorithms being repeatedly +reimplemented in C programs while higher-level languages natively +provide the correct algorithms. Also note that languages can be static +or dynamic independently of being high or low-level: high-level +expressive languages can be static and efficient [OCAML] while +inexpressive ones can be dynamic and much less efficient [PROLOG]. + +The PyPy project -- phase one -- aims to write a Python interpreter +using Python itself as the implementation language for flexibility, with +the additional constraint that core parts should be written in a more +static subset of the language, a restriction that opens the door to +static analysis and translation to a low-level language like C. This +approach has already been taken, e.g. for the Scheme language [REF], but +we have an original approach to the translation process that we will +describe later. + +The net result will be an interpreter whose performance is comparable to +today's C-based implementation, but more flexible. The translation is +not a one-shot: the maintained source is the Python one. Moreover, +aspects that were design decisions in the current C Python are now +merely customizable behaviour of the translator. In other words, the +translator is not merely a restricted-Python-to-C translator; it is an +essential piece towards the flexibility goals of the previous section. +Aspects like memory layout of objects are not defined by the PyPy Python +source, but derived from it by the translator with the support of a +small run-time system written in C. It is thus expected that the +translator should be a highly configurable tool, with pluggable +components, that will allow us (and third-parties) to produce customized +C versions of PyPy with tailored trade-offs. Moreover, the exact set of +Python source that the translator will be applied to can be varied as +well, producing C versions with different capabilities. + +Thus, our objectives are: + +4. to produce several different C versions for different identified usages +of Python; + +5. to allow third-parties to produce customized C version easily; + +6. to adapt the translator to target other non-C platforms (Java and/or +.NET and/or other). + +Moreover, we will also consider targetting a virtual machine that +already has a high-performance JIT compiler, possibly Self (see below). + + +JIT compilers +------------- + +JIT compilers have been reasonably well studied; an account of their +history is given in [REF:jit-history.ps]. But actually writing a JIT for +a given language is generally a major task. Different techniques to ease +this path have been recently explored; let us cite: + +* to implement a dynamic language in a flexible way, it can be written +on top of another one, e.g. as a dynamic translator that produces +bytecodes for an existing virtual machine. If the virtual machine is +already equipped with state-of-the-art JIT compilation, it is possible +to leverage its benefits to the new language. This path is not only much +sorter than designing a complete custom JIT, but it is also easier to +maintain and evolve. This idea is explored for the Self virtual machine +in [REF:http://research.sun.com/research/kanban/oopsla-vm-wkshp.pdf]. As +pointed out in that paper, some source language features may not match +any of the target virtual machine features. When this issue araises, we +are left with the hard problem of refactoring an efficient JIT-based +virtual machine. + +* a completely different approach: making it easier to derive a JIT from +an existing C interpreter. DynamoRIO [REF] instrumentates the execution +of compiled programs and optimizes them dynamically. It has been +extended with specific support for interpreters. With minimal amounts of +changes in the source of an interpreter, it can significantly reduce the +processor-time interpretative overhead +[REF:http://www.ai.mit.edu/~gregs/dynamorio.htm]. While this offers a +highly competitive gain/effort ratio, performance does not reach the +levels of a hand-crafted JIT. + + +The performance goal, phase 2 +----------------------------- + +Building on phase 1, we aim to add a JIT compiler to PyPy. This will be +done using a different technique than described in the previous +paragraph: our goal is to *generate* the JIT compiler from the +high-level Python source of PyPy. + +This will be accomplished by integrating the technology developed in +Psyco [REF], a prototype JIT for the Python programming language. It is +more precisely a specializing JIT compiler based on abstract +interpretation. + +Although Psyco was hand-written, large parts have a one-to-one +correspondence to whole sections of the C Python interpreter. This is +uncommon for JITs, but Psyco's good results give ample proof-of-concept. +Moreover, this property can be explicitely related to the DynamoRIO +project cited above, which instrumentates the interpreter itself. +Indeed, the one-to-one correspondance between parts of C Python and +Psyco is as follows: to each expression in C Python corresponds a more +complex expression in Psyco, which does instrumentation and +optimizations. The difference with DynamoRIO is that the latter analyses +at run-time the machine code resulting from the compilation of the +interpreter. In Psyco the one-to-one correspondance is with the C source +instead. It could certainly have been done by automated analysis of the +C source of C Python, but this is difficult and error-prone. + +In the PyPy project, on the other hand, the source code of the +interpreter will be in Python instead of C. It is much easier to analyse +safely. Thus, the plan is to raise the level further and rely on yet +another customization of the translator, to let it emit the complex +Psyco-style expressions instead of (or in addition to) the normal C +expressions that would be the direct translation. + +The last objective is thus: + +7. to provide a flexible Python interpreter running at JIT speed. + + +The translator +-------------- + +Finally, if this were still needed, let us further demonstrate the +benefits of writing an interpreter flexibly in a high-level language. So +far, the translator sounds like a magic piece of difficult-to-write +software. This is not so. Analysing pieces of Python source code would +not be a complex task anyway. Some type inference is needed, but it can +be kept simple by suitably restricting the amount of allowed dynamism. + +However, instead of analysing source code, we propose another approach: +doing the type inference by abstract interpretation [REF]. This is +essentially the way Psyco works, albeit it does it at run-time. [FIXME: +has it already been done elsewhere?] [otherwise explain in a FOOTNOTE: +the essential idea is to "interpret" the Python source code that we want +to translate; all manipulated objects are abstracted into their type +only. Operations between two such "abstract objects" return a new +"abstract object", which represents the type of the result.] As we have +seen above, abstract interpretation is easily acheived by plugging a +custom object space into the PyPy code base. The translator can be built +as an extension of this "type inference" object space, emitting +translated code as a side-effect along the analysis. Thus the translator +is nothing more than PyPy itself with a custom object space: when you +use this special version of PyPy to "interpret" a Python function, it is +in fact translated... +In summary, to translate PyPy itself to C, we will be using PyPy! From arigo at codespeak.net Fri Sep 19 12:55:26 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 19 Sep 2003 12:55:26 +0200 (MEST) Subject: [pypy-svn] rev 1363 - pypy/trunk/doc/funding Message-ID: <20030919105526.3C7BF5AD0D@thoth.codespeak.net> Author: arigo Date: Fri Sep 19 12:55:24 2003 New Revision: 1363 Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt pypy/trunk/doc/funding/B3.0_impact.txt pypy/trunk/doc/funding/B3.1_standards.txt pypy/trunk/doc/funding/B4.1_subcontracting.txt pypy/trunk/doc/funding/B4.2_other_countries.txt Log: A few remarks Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_relevance.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_relevance.txt Fri Sep 19 12:55:24 2003 @@ -52,13 +52,16 @@ integration, addressing non-functional aspects, complexity, autonomy and composability. -PyPy will advance the 'state-of-the-art' in specializing compilers, +PyPy will advance the 'state-of-the-art' in high level +implementation of languages, introducing high level methods +and concepts in the domain, producing a development environment +and methods with as-yet-unmatched flexibility, with a new architecture for object libraries that provide for radical improvements in (the management of) complexity, autonomy (self-hosting), and composability (seamless modularity). - -ASK_STOCKHOLM DO I HAVE TO EXPLAIN THIS MORE? +ASK_STOCKHOLM DO I HAVE TO EXPLAIN THIS MORE? + b) Open and modular development environments, enabling flexibility and extensibility with new or sector-specific tools (e.g. intelligent distributed decision support), @@ -127,6 +130,10 @@ research that has no direct applications, but goes under the heading 'it is nice to learn stuff'? + FIXME + Emphasis "quantitative analysis" as per WP06 and WP07 where we + describe PyPy as a research and comparison platform + Work should where appropriate, enhance and complement work implemented under EUREKA/ITEA and in software initiatives at member and associated state level. The IST programme will seek Modified: pypy/trunk/doc/funding/B3.0_impact.txt ============================================================================== --- pypy/trunk/doc/funding/B3.0_impact.txt (original) +++ pypy/trunk/doc/funding/B3.0_impact.txt Fri Sep 19 12:55:24 2003 @@ -154,7 +154,7 @@ These talks will create interest as well as teach techniques. We will submit a PEP and push to get PyPy made the reference implementation of -the Python programming language. We will continue to discuss PyPy on +the Python programming language(*). We will continue to discuss PyPy on our own mailing lists, as well as other Python mailing lists such as the Python-in-Education list, and the Usenet Newsgroup comp.lang.python. We already have an IRC channel, #pypy on @@ -162,6 +162,13 @@ communicate with each other while we are apart. Beyond that -- we are willing to take any actions the EU would like to fund. + FIXME + (*) emphasis on this one? If this really happens we'll arguably + become the software European Project with the largest distribution + and user base ever (XXX rephrase :-) + Also mention that the issue (and the word 'PEP') is discussed + later in B3.1_standards.txt? + Ask Stockholm We think you get no money for this in a STREP. True? ASK STOCKHOLM -- Modified: pypy/trunk/doc/funding/B3.1_standards.txt ============================================================================== --- pypy/trunk/doc/funding/B3.1_standards.txt (original) +++ pypy/trunk/doc/funding/B3.1_standards.txt Fri Sep 19 12:55:24 2003 @@ -71,6 +71,11 @@ likely, though not certain, that PyPy will become the standard reference implementation of the Python language. + FIXME + either say 'very likely if all the goals described in the project + are met' or give some more precise conditions, e.g. existence of a + PyPy version largely compatible with existing C extension modules + The complete details of how to write a PEP are themselves an Informational PEP -- PEP #1 in fact. Complete details are to be found in this appendix: Modified: pypy/trunk/doc/funding/B4.1_subcontracting.txt ============================================================================== --- pypy/trunk/doc/funding/B4.1_subcontracting.txt (original) +++ pypy/trunk/doc/funding/B4.1_subcontracting.txt Fri Sep 19 12:55:24 2003 @@ -15,3 +15,9 @@ ASK_STOCKHOLM Do I have to discuss this? + FIXME + should we mention that other people out of the project + are likely to contribute patches for specific parts, + as is always the case in this kind of open source project? + maybe these contribution would not be significant to the + primary WP goals. Modified: pypy/trunk/doc/funding/B4.2_other_countries.txt ============================================================================== --- pypy/trunk/doc/funding/B4.2_other_countries.txt (original) +++ pypy/trunk/doc/funding/B4.2_other_countries.txt Fri Sep 19 12:55:24 2003 @@ -21,3 +21,5 @@ .. [*] Armin Rigo is Swiss too. Switzerland and the EU have general agreement for education and research programs. Do they cover this case? + + FIXME: what about Guido? From tismer at codespeak.net Fri Sep 19 23:32:20 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Fri, 19 Sep 2003 23:32:20 +0200 (MEST) Subject: [pypy-svn] rev 1366 - pypy/trunk/doc/funding Message-ID: <20030919213220.20B4E5C354@thoth.codespeak.net> Author: tismer Date: Fri Sep 19 23:32:19 2003 New Revision: 1366 Modified: pypy/trunk/doc/funding/B1.0_objectives.txt Log: edited to work as reST Modified: pypy/trunk/doc/funding/B1.0_objectives.txt ============================================================================== --- pypy/trunk/doc/funding/B1.0_objectives.txt (original) +++ pypy/trunk/doc/funding/B1.0_objectives.txt Fri Sep 19 23:32:19 2003 @@ -48,16 +48,16 @@ the language implementation: * user pressure tends to shape interpreters towards performance at the -expense of other aspects, like small footprints and code simplicity. + expense of other aspects, like small footprints and code simplicity. * there are a lot of other trade-offs; experimenting with different -solutions can be close to impossible for aspects that affect the -interpreter globally. These aspects are usually frozen in early design -decisions (memory management, object model, multithreading,...). + solutions can be close to impossible for aspects that affect the + interpreter globally. These aspects are usually frozen in early design + decisions (memory management, object model, multithreading,...). * optimization drives the interpreter code base, an error-prone and -scope-limiting process. It is impossible to reuse the code base for -implementation on new, non-C platforms; you need to start from scratch. + scope-limiting process. It is impossible to reuse the code base for + implementation on new, non-C platforms; you need to start from scratch. Moreover, conflicting with performance and manageable complexity, there is a large number of aspects and features that users would like to see @@ -79,10 +79,10 @@ 1. to create a implementation of the Python language in Python. 2. to implement a subset of the above-mentioned features (FIXME which -ones?) as an immediately useful proof-of-concept. + ones?) as an immediately useful proof-of-concept. 3. to allow third-parties to implement any other interpreter-level feature -more easily than they would in a traditional interpreter. + more easily than they would in a traditional interpreter. The main building piece behind the proposed flexibility is -- besides using a high-level language in the first place -- the concept of Object @@ -145,12 +145,12 @@ Thus, our objectives are: 4. to produce several different C versions for different identified usages -of Python; + of Python; 5. to allow third-parties to produce customized C version easily; 6. to adapt the translator to target other non-C platforms (Java and/or -.NET and/or other). + .NET and/or other). Moreover, we will also consider targetting a virtual machine that already has a high-performance JIT compiler, possibly Self (see below). @@ -165,27 +165,27 @@ this path have been recently explored; let us cite: * to implement a dynamic language in a flexible way, it can be written -on top of another one, e.g. as a dynamic translator that produces -bytecodes for an existing virtual machine. If the virtual machine is -already equipped with state-of-the-art JIT compilation, it is possible -to leverage its benefits to the new language. This path is not only much -sorter than designing a complete custom JIT, but it is also easier to -maintain and evolve. This idea is explored for the Self virtual machine -in [REF:http://research.sun.com/research/kanban/oopsla-vm-wkshp.pdf]. As -pointed out in that paper, some source language features may not match -any of the target virtual machine features. When this issue araises, we -are left with the hard problem of refactoring an efficient JIT-based -virtual machine. + on top of another one, e.g. as a dynamic translator that produces + bytecodes for an existing virtual machine. If the virtual machine is + already equipped with state-of-the-art JIT compilation, it is possible + to leverage its benefits to the new language. This path is not only much + sorter than designing a complete custom JIT, but it is also easier to + maintain and evolve. This idea is explored for the Self virtual machine + in [REF:http://research.sun.com/research/kanban/oopsla-vm-wkshp.pdf]. As + pointed out in that paper, some source language features may not match + any of the target virtual machine features. When this issue araises, we + are left with the hard problem of refactoring an efficient JIT-based + virtual machine. * a completely different approach: making it easier to derive a JIT from -an existing C interpreter. DynamoRIO [REF] instrumentates the execution -of compiled programs and optimizes them dynamically. It has been -extended with specific support for interpreters. With minimal amounts of -changes in the source of an interpreter, it can significantly reduce the -processor-time interpretative overhead -[REF:http://www.ai.mit.edu/~gregs/dynamorio.htm]. While this offers a -highly competitive gain/effort ratio, performance does not reach the -levels of a hand-crafted JIT. + an existing C interpreter. DynamoRIO [REF] instrumentates the execution + of compiled programs and optimizes them dynamically. It has been + extended with specific support for interpreters. With minimal amounts of + changes in the source of an interpreter, it can significantly reduce the + processor-time interpretative overhead + [REF:http://www.ai.mit.edu/~gregs/dynamorio.htm]. While this offers a + highly competitive gain/effort ratio, performance does not reach the + levels of a hand-crafted JIT. The performance goal, phase 2 From hpk at codespeak.net Sat Sep 20 11:15:32 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 20 Sep 2003 11:15:32 +0200 (MEST) Subject: [pypy-svn] rev 1367 - pypy/trunk/doc/funding Message-ID: <20030920091532.F21A65C35D@thoth.codespeak.net> Author: hpk Date: Sat Sep 20 11:15:32 2003 New Revision: 1367 Modified: pypy/trunk/doc/funding/B1.0_objectives.txt Log: - insert a XXX paragraph for Armin and others to judge in the introduction - fixed a typo reported by Florian Schulze Modified: pypy/trunk/doc/funding/B1.0_objectives.txt ============================================================================== --- pypy/trunk/doc/funding/B1.0_objectives.txt (original) +++ pypy/trunk/doc/funding/B1.0_objectives.txt Sat Sep 20 11:15:32 2003 @@ -23,7 +23,14 @@ Interpretation is a common implementation technique for a wide range of programming languages whose primary objective is not efficiency. Interpreters are well-suited for small-to-medium languages, favouring a -simple and clear engine. +simple and clear engine. XXX aren't there "large" interpreted languages? + +XXX armin et.al..: isn't the key in the following para that today's language design is +"monolithic" and not configurable, i.e. all bigger programming languages are not +designed in a modular way and thus give the user no choice which components/aspects +they want to use for their target application? But that today's applications +are to be developed for an increasing variety of devices and environments (Big-Iron, cars, +watches, mobile phones, banks, whatever). What about a security-goal btw? As a language becomes popular, however, the costs inherent to this approach are increasingly pointed out. These costs are twofold. Firstly, @@ -169,7 +176,7 @@ bytecodes for an existing virtual machine. If the virtual machine is already equipped with state-of-the-art JIT compilation, it is possible to leverage its benefits to the new language. This path is not only much - sorter than designing a complete custom JIT, but it is also easier to + shorter than designing a complete custom JIT, but it is also easier to maintain and evolve. This idea is explored for the Self virtual machine in [REF:http://research.sun.com/research/kanban/oopsla-vm-wkshp.pdf]. As pointed out in that paper, some source language features may not match From pedronis at codespeak.net Sat Sep 20 15:19:32 2003 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 20 Sep 2003 15:19:32 +0200 (MEST) Subject: [pypy-svn] rev 1368 - pypy/trunk/doc Message-ID: <20030920131932.A40C45C361@thoth.codespeak.net> Author: pedronis Date: Sat Sep 20 15:19:32 2003 New Revision: 1368 Modified: pypy/trunk/doc/readme.txt (props changed) Log: fix eol. From pedronis at codespeak.net Sat Sep 20 15:27:22 2003 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 20 Sep 2003 15:27:22 +0200 (MEST) Subject: [pypy-svn] rev 1369 - pypy/trunk/doc/funding Message-ID: <20030920132722.E829A5C362@thoth.codespeak.net> Author: pedronis Date: Sat Sep 20 15:27:22 2003 New Revision: 1369 Modified: pypy/trunk/doc/funding/plan.txt (contents, props changed) Log: set eol-style. Modified: pypy/trunk/doc/funding/plan.txt ============================================================================== --- pypy/trunk/doc/funding/plan.txt (original) +++ pypy/trunk/doc/funding/plan.txt Sat Sep 20 15:27:22 2003 @@ -1,217 +1,217 @@ -========================== -Draft of a PyPy work plan -========================== - - -1. The !PyPy Interpreter ---------------------------- - -The goal is to make a complete Python interpreter that runs under any -existing Python implementation. - -a) develop and complete the !PyPy interpreter itself, as a regular - Python program, until it contains all the parts of CPython that we don't - want to move to b). Further investigate the unorthodox multimethod - concepts that the standard object space is based on, and how to hook in - the bytecode compiler. - -b) translate all other parts of !CPython into regular Python libraries. - These ones should also work without !PyPy, being just plain-Python - replacements for existing !CPython functionality. This includes the - bytecode compiler. - - -2. Translation of !RPython ------------------------------- - -The goal is to be able to translate arbitrary RPython source code (e.g. -the one produced in 1a) into low-level code (C, Pyrex, Java, others). -This includes making a stand-alone, not-!PyPy-related tool for general -optimization of arbitrary but suitably restricted Python application or -parts thereof. - -a) analyse code to produce the relevant typing information. Investigate - if we can use the annotation object space only or if additional - AST-based control flow analysis is needed. Give a formal definition - of RPython. - -b) produce low-level code out of the data gathered in (a). Again - investigate how this is best done (AST-guided translation or - reverse-engeneering of the low-level control flow gathered by the - annotation object space). Compare different low-level environment that - we could target (C, Pyrex, others?). - - -3. Bootstrapping !PyPy --------------------------- - -The goal is to put (1) and (2) together. - -a) investigate the particular problems specific to the global - translation of !PyPy, as opposed to general to any !RPython program. - According to the requirements and insights of (2) we will probably have - to redesign specific parts of !PyPy, e.g. make the various - app-level/interp-level interface designs converge. - -b) build the low-level-specific run-time components of !PyPy, most - notably the object layout, the memory management, possibly threading - support, and multimethod dispatch. Here, if we target C code, important - parts can be directly re-used from !CPython. - - -4. High-performance !PyPy-Python ------------------------------------ - -The goal is to optimize `3. Bootstrapping !PyPy`_ in possibly various ways, building on its -flexibility to go beyond !CPython. - -a) develop several object implementations for the same types, as - explicitely allowed by the standard object space, and develop heuristics - to switch between implementations during execution. - -b) identify which optimizations would benefit from support from the - translator (2). These are the optimizations not easily available to - !CPython because they would require large-scale code rewrites. - -c) for each issue, work on several solutions when no one is obviously - better than the other ones. The meta-programming underlying (b) -- - namely the work on the translator instead of on the resulting code -- is - what gives us the possibility of actually implementing several very - different schemes. - -d) integrate existing technology that traditionally depended on closely - following !CPython's code base, notably Psyco and Stackless. Rewrite each - one as a meta-component that hooks into the translator (2) plus a - dedicated run-time component (3b). Further develop these technologies - based on the results gathered in (c), e.g. identify when these - technologies would guide specific choices among the solutions developed - in (a) and (b). - - -Annex to (a) -~~~~~~~~~~~~ - -Some major uses for several implementations of the built-in types: - -* dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, - or with string-only or integer-only keys. Dictionaries with specific - support for "on-change" callbacks (useful for Psyco). - -* strings as plain immutable memory buffers vs. immutable but more - complex data structures (see functional languages) vs. internally - mutable data structures (e.g. Psyco's concatenated strings) - -* ints as machine words vs. two machine words vs. internal longs vs. - external bignum library (investigate if completely unifying ints and - longs is possible in the Python language at this stage). - -* etc. (lists as range() or chained lists, ...) - -The above are mostly independent from any particular low-level run-time -environment. - - -Annex to (b) -~~~~~~~~~~~~ - -Here are some of the main issues and tricks. Note that compatibility -with legacy C extensions can be acheived by choosing, for each of the -following issues, the same one as !CPython did. - -* object layout and memory management strategy or strategies, e.g. - reference counting vs. Boehm garbage collection vs. our own. Includes - speed vs. data size trade-offs. - -* code size vs. speed trade-offs (e.g. whether the final interpreter - should still include compact precompiled bytecode or be completely - translated into C). - -* the complex issue of threading (global interpreter lock vs. - alternatives). - -* multimethod dispatching - -* pointer tagging, e.g. encoding an integer object as a pointer with a - special value instead of a real pointer to a data structure representing - the integer. - -The above are mostly specific to a particular low-level run-time. - - -5. Low-level targets, tools and releases --------------------------------------------- - -The goal is to identify, among those low-level targets that are in -widespread use (e.g. workstation usage vs. web server vs. -high-performance computing vs. memory-starved hand-held device; C/Unix -vs. Java vs. .NET environment), which ones would benefit most from a -high-performance Python interpreter. For each of these, focus will be -given to: - -a) develop the translation process, run-time and those optimizations - that depend on low-level details. - -b) design interfaces for extension modules. Some can be very general - (e.g. a pure Python one that should allow generic third-party code to - hook into the !PyPy interpreter source code without worrying about the - translation process). Others depend on the low-level environment and on - the choices made for the issues of (4). - -c) combine different solutions for the different issues discussed in - (4). Gather statistics with real-work Python application. Compare the - results. This is where the flexibility of the whole project is vital. - Typically, very different trade-offs need to be made on different - environments. - -d) most importantly, develop tools to easily allow third-parties to - repeat (c) in their own domain and build their own tailored versions of - !PyPy. - -e) release a few official versions pre-tailored for various common - environments. Develop in particular a version whose goal is to simulate - the existing !CPython interpreter to support legacy extension modules. - Investigate if the !PyPy core can make internal choices that are very - different from !CPython's without sacrifying legacy extension modules - compatibility. - - -6. Infrastructure ---------------------- - -The goal is to address the development and maintenance issues. - -a) !PyPy's own development needs an infrastructure that must - continuously be kept up-to-date and further developed. - -b) write tests. All parts of !PyPy should be extensively covered by - stress tests. Investigate the use of test-coverage analysers. - -c) investigate means of keeping !PyPy in sync with the future - developments of !CPython, e.g. ways to relate pieces of !PyPy source and - pieces of !CPython source. Look for existing solutions. - - -7. Extension of !PyPy -------------------------- - -The goal is to add functionalities in !PyPy that are not present in -existing Python implementations. This is an open goal. We only list a -few promizing directions: - -a) build alternate object spaces provides features that are essentially - language-transparent, e.g. distributed computing (via a network proxy - object space), compatibility layers (e.g. a Python-1.5.2-compliant - object space), persistance (via a persistant object space). - -b) build language features that rely on translator support (2), i.e. - which can be turned on or off during the production of individual - versions of !PyPy, e.g. Stackless and continuations. - -c) work on the interaction between the compiler and the main loop to - allow custom opcodes to be defined, generated by the compiler, and - interpreted by the main loop, thus allowing syntactic extension of the - language by user code - -d) conversely, develop interfaces to use object spaces without the main - loop to provide Python-like object semantics to other programming - languages, using their own syntax and execution environment, e.g. Java. +========================== +Draft of a PyPy work plan +========================== + + +1. The !PyPy Interpreter +--------------------------- + +The goal is to make a complete Python interpreter that runs under any +existing Python implementation. + +a) develop and complete the !PyPy interpreter itself, as a regular + Python program, until it contains all the parts of CPython that we don't + want to move to b). Further investigate the unorthodox multimethod + concepts that the standard object space is based on, and how to hook in + the bytecode compiler. + +b) translate all other parts of !CPython into regular Python libraries. + These ones should also work without !PyPy, being just plain-Python + replacements for existing !CPython functionality. This includes the + bytecode compiler. + + +2. Translation of !RPython +------------------------------ + +The goal is to be able to translate arbitrary RPython source code (e.g. +the one produced in 1a) into low-level code (C, Pyrex, Java, others). +This includes making a stand-alone, not-!PyPy-related tool for general +optimization of arbitrary but suitably restricted Python application or +parts thereof. + +a) analyse code to produce the relevant typing information. Investigate + if we can use the annotation object space only or if additional + AST-based control flow analysis is needed. Give a formal definition + of RPython. + +b) produce low-level code out of the data gathered in (a). Again + investigate how this is best done (AST-guided translation or + reverse-engeneering of the low-level control flow gathered by the + annotation object space). Compare different low-level environment that + we could target (C, Pyrex, others?). + + +3. Bootstrapping !PyPy +-------------------------- + +The goal is to put (1) and (2) together. + +a) investigate the particular problems specific to the global + translation of !PyPy, as opposed to general to any !RPython program. + According to the requirements and insights of (2) we will probably have + to redesign specific parts of !PyPy, e.g. make the various + app-level/interp-level interface designs converge. + +b) build the low-level-specific run-time components of !PyPy, most + notably the object layout, the memory management, possibly threading + support, and multimethod dispatch. Here, if we target C code, important + parts can be directly re-used from !CPython. + + +4. High-performance !PyPy-Python +----------------------------------- + +The goal is to optimize `3. Bootstrapping !PyPy`_ in possibly various ways, building on its +flexibility to go beyond !CPython. + +a) develop several object implementations for the same types, as + explicitely allowed by the standard object space, and develop heuristics + to switch between implementations during execution. + +b) identify which optimizations would benefit from support from the + translator (2). These are the optimizations not easily available to + !CPython because they would require large-scale code rewrites. + +c) for each issue, work on several solutions when no one is obviously + better than the other ones. The meta-programming underlying (b) -- + namely the work on the translator instead of on the resulting code -- is + what gives us the possibility of actually implementing several very + different schemes. + +d) integrate existing technology that traditionally depended on closely + following !CPython's code base, notably Psyco and Stackless. Rewrite each + one as a meta-component that hooks into the translator (2) plus a + dedicated run-time component (3b). Further develop these technologies + based on the results gathered in (c), e.g. identify when these + technologies would guide specific choices among the solutions developed + in (a) and (b). + + +Annex to (a) +~~~~~~~~~~~~ + +Some major uses for several implementations of the built-in types: + +* dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, + or with string-only or integer-only keys. Dictionaries with specific + support for "on-change" callbacks (useful for Psyco). + +* strings as plain immutable memory buffers vs. immutable but more + complex data structures (see functional languages) vs. internally + mutable data structures (e.g. Psyco's concatenated strings) + +* ints as machine words vs. two machine words vs. internal longs vs. + external bignum library (investigate if completely unifying ints and + longs is possible in the Python language at this stage). + +* etc. (lists as range() or chained lists, ...) + +The above are mostly independent from any particular low-level run-time +environment. + + +Annex to (b) +~~~~~~~~~~~~ + +Here are some of the main issues and tricks. Note that compatibility +with legacy C extensions can be acheived by choosing, for each of the +following issues, the same one as !CPython did. + +* object layout and memory management strategy or strategies, e.g. + reference counting vs. Boehm garbage collection vs. our own. Includes + speed vs. data size trade-offs. + +* code size vs. speed trade-offs (e.g. whether the final interpreter + should still include compact precompiled bytecode or be completely + translated into C). + +* the complex issue of threading (global interpreter lock vs. + alternatives). + +* multimethod dispatching + +* pointer tagging, e.g. encoding an integer object as a pointer with a + special value instead of a real pointer to a data structure representing + the integer. + +The above are mostly specific to a particular low-level run-time. + + +5. Low-level targets, tools and releases +-------------------------------------------- + +The goal is to identify, among those low-level targets that are in +widespread use (e.g. workstation usage vs. web server vs. +high-performance computing vs. memory-starved hand-held device; C/Unix +vs. Java vs. .NET environment), which ones would benefit most from a +high-performance Python interpreter. For each of these, focus will be +given to: + +a) develop the translation process, run-time and those optimizations + that depend on low-level details. + +b) design interfaces for extension modules. Some can be very general + (e.g. a pure Python one that should allow generic third-party code to + hook into the !PyPy interpreter source code without worrying about the + translation process). Others depend on the low-level environment and on + the choices made for the issues of (4). + +c) combine different solutions for the different issues discussed in + (4). Gather statistics with real-work Python application. Compare the + results. This is where the flexibility of the whole project is vital. + Typically, very different trade-offs need to be made on different + environments. + +d) most importantly, develop tools to easily allow third-parties to + repeat (c) in their own domain and build their own tailored versions of + !PyPy. + +e) release a few official versions pre-tailored for various common + environments. Develop in particular a version whose goal is to simulate + the existing !CPython interpreter to support legacy extension modules. + Investigate if the !PyPy core can make internal choices that are very + different from !CPython's without sacrifying legacy extension modules + compatibility. + + +6. Infrastructure +--------------------- + +The goal is to address the development and maintenance issues. + +a) !PyPy's own development needs an infrastructure that must + continuously be kept up-to-date and further developed. + +b) write tests. All parts of !PyPy should be extensively covered by + stress tests. Investigate the use of test-coverage analysers. + +c) investigate means of keeping !PyPy in sync with the future + developments of !CPython, e.g. ways to relate pieces of !PyPy source and + pieces of !CPython source. Look for existing solutions. + + +7. Extension of !PyPy +------------------------- + +The goal is to add functionalities in !PyPy that are not present in +existing Python implementations. This is an open goal. We only list a +few promizing directions: + +a) build alternate object spaces provides features that are essentially + language-transparent, e.g. distributed computing (via a network proxy + object space), compatibility layers (e.g. a Python-1.5.2-compliant + object space), persistance (via a persistant object space). + +b) build language features that rely on translator support (2), i.e. + which can be turned on or off during the production of individual + versions of !PyPy, e.g. Stackless and continuations. + +c) work on the interaction between the compiler and the main loop to + allow custom opcodes to be defined, generated by the compiler, and + interpreted by the main loop, thus allowing syntactic extension of the + language by user code + +d) conversely, develop interfaces to use object spaces without the main + loop to provide Python-like object semantics to other programming + languages, using their own syntax and execution environment, e.g. Java. From lac at codespeak.net Sat Sep 20 15:38:06 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Sat, 20 Sep 2003 15:38:06 +0200 (MEST) Subject: [pypy-svn] rev 1370 - pypy/trunk/doc/funding Message-ID: <20030920133806.1F51A5B356@thoth.codespeak.net> Author: lac Date: Sat Sep 20 15:38:05 2003 New Revision: 1370 Added: pypy/trunk/doc/funding/Bookfp6.pdf Log: Myer W Morron's Guide to actually writing things that the Commission will fund. And don't I wish I had read it a month ago. Added: pypy/trunk/doc/funding/Bookfp6.pdf ============================================================================== Files (empty file) and pypy/trunk/doc/funding/Bookfp6.pdf Sat Sep 20 15:38:05 2003 differ From tismer at codespeak.net Sat Sep 20 15:46:21 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 20 Sep 2003 15:46:21 +0200 (MEST) Subject: [pypy-svn] rev 1371 - pypy/trunk/doc/funding Message-ID: <20030920134621.E2AD05C363@thoth.codespeak.net> Author: tismer Date: Sat Sep 20 15:46:21 2003 New Revision: 1371 Added: pypy/trunk/doc/funding/B6.wp11_small_os.txt - copied unchanged from rev 1365, pypy/trunk/doc/funding/B6.wp10_small_os.txt pypy/trunk/doc/funding/renamewp.py Removed: pypy/trunk/doc/funding/B6.wp10_small_os.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/crossreferences.asc pypy/trunk/doc/funding/mkxref.py Log: renamed WP10 to WP11 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Sat Sep 20 15:46:21 2003 @@ -253,7 +253,7 @@ the target platform. -WP10_: A small Operating System for an Embedded Device +WP11_: A small Operating System for an Embedded Device ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *we are thinking of a printer here, but don't have facts yet* Deleted: /pypy/trunk/doc/funding/B6.wp10_small_os.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp10_small_os.txt Sat Sep 20 15:46:21 2003 +++ (empty file) @@ -1,58 +0,0 @@ -.. include:: crossreferences.asc - -Workpackage Description -======================== - -================================ ========================== -**Workpackage number** 10 --------------------------------- -------------------------- -**Start date or starting event** not before WP05_ -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y Z ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 2 2 2 -================================= ======= ======= ======= ======= ======= ======= ======= ======= - - -**Objectives** - -Title: A small Operating System for an Embedded Device - -XXX the following is still WP09, write it!!! - -- Supporting a completely different platform -- Providing an efficient, high-level Language for - embedded Devices -- Proving the flexibility of the PyPy idea - - -**Description of work** - -- Analyse the differences between the new architecture and - PyPy's current. Deside whether to optimize for code size - or applicatin speed. - -- Develop the translation process, run-time and those optimisations - that depend on low-level details. - -- Implement a simulator for the embedded device. - -- Implement interfaces to the needed external libraries. - -- Combine different solutions for the different issues discussed in - High-performance (WP06_, WP07_, WP08_). - Gather statistics with real-work Python applications. - Compare the results. This is where the flexibility of the whole - project is vital. Typically, very different trade-offs need to be - made on different environments. - - -**Deliverables** - -- DL??? Python running on the embedded device. -- DL??? Passing all applicable elements of the standard test suite. - -**Milestones and Expected Result** - Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Sat Sep 20 15:46:21 2003 @@ -7,4 +7,4 @@ .. _WP07: B6.wp07_translator_optimisations.html .. _WP08: B6.wp08_integrate_technology.html .. _WP09: B6.wp09_embedded_device.html -.. _WP10: B6.wp10_small_os.html +.. _WP11: B6.wp11_small_os.html Modified: pypy/trunk/doc/funding/mkxref.py ============================================================================== --- pypy/trunk/doc/funding/mkxref.py (original) +++ pypy/trunk/doc/funding/mkxref.py Sat Sep 20 15:46:21 2003 @@ -11,7 +11,7 @@ START_STR = "B6.wp".lower() -def get_wps_from_dir(): +def get_wps_from_dir(as_html=False): res = {} for fname in os.listdir("."): flow = fname.lower() @@ -19,8 +19,12 @@ if flow.startswith(START_STR) and ext == ".txt": num = flow[len(START_STR):].split("_", 1)[0] if len(num)==2 and num.isdigit(): - pre, ext = os.path.splitext(fname) - res["WP"+num] = pre + ".html" + if as_html: + pre, ext = os.path.splitext(fname) + newfname = pre + ".html" + else: + newfname = fname + res["WP"+num] = newfname return res REF_FILE = "B6.1_plan_introduction.txt" @@ -65,7 +69,7 @@ def build_xref_file(): pre, old, post = split_xref_text() wps = get_wps_from_file() - wps.update(get_wps_from_dir()) + wps.update(get_wps_from_dir(as_html=True)) wplist = wps.items() wplist.sort() mid = [ ".. _%s: %s\n" % (name, url) for name, url in wplist] Added: pypy/trunk/doc/funding/renamewp.py ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/renamewp.py Sat Sep 20 15:46:21 2003 @@ -0,0 +1,55 @@ +# renaming a working package. +usage = """usage: +python renamewp.py old new +where old and new are decimal WP numbers in the range 1-99. +""" + +import os, sys, mkxref + +def do_rename(old, new): + oldS = "WP%02d" % old + newS = "WP%02d" % new + wpfiles = mkxref.get_wps_from_dir() + if wpfiles.has_key(oldS): + if wpfiles.has_key(newS): + raise ValueError, "cannot rename because %s is in the way!" % wpfiles[newS] + oldName = wpfiles[oldS] + p = len(mkxref.START_STR) + newName = "%s%02d%s" % (oldName[:p], new, oldName[p+2:]) + cmd = 'svn move "%s" "%s"' % (oldName, newName) + print ">>>", cmd + err = os.system(cmd) + if err: + print "aborting due to the above error" + sys.exit(1) + del wpfiles[oldS] + wpfiles[newS] = newName + workfiles = wpfiles.values() + workfiles.append(mkxref.REF_FILE) + find = oldS+"_" + repl = newS+"_" + for fname in workfiles: + txt = file(fname).read() + if txt.find(find) >= 0: + print "rewriting %s" % fname + txt = txt.replace(find, repl) + file(fname, "w").write(txt) + print "re-creating cross references" + mkxref.build_xref_file() + print "done." + + +def main(): + args = sys.argv[1:] + try: + old, new = map(int, args) + if min(old, new) < 1 or max(old, new) > 99: + raise ValueError + except: + print usage + sys.exit(1) + do_rename(old, new) + +if __name__ == "__main__": + main() + \ No newline at end of file From tismer at codespeak.net Sat Sep 20 15:47:21 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 20 Sep 2003 15:47:21 +0200 (MEST) Subject: [pypy-svn] rev 1372 - pypy/trunk/doc/funding Message-ID: <20030920134721.45CDA5C364@thoth.codespeak.net> Author: tismer Date: Sat Sep 20 15:47:20 2003 New Revision: 1372 Added: pypy/trunk/doc/funding/B6.wp10_embedded_device.txt - copied unchanged from rev 1370, pypy/trunk/doc/funding/B6.wp09_embedded_device.txt Removed: pypy/trunk/doc/funding/B6.wp09_embedded_device.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/crossreferences.asc Log: renamed WP09 to WP10 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Sat Sep 20 15:47:20 2003 @@ -243,7 +243,7 @@ the abstraction and ease-of-use of Python. -WP09_: Porting PyPy to an Embedded Device +WP10_: Porting PyPy to an Embedded Device ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dependant from the actual processor architecture, PyPy's @@ -258,7 +258,7 @@ *we are thinking of a printer here, but don't have facts yet* -In extent to WP09_, it makes sense to write the whole +In extent to WP10_, it makes sense to write the whole operating system in PyPy. We need a low-level extension to the code generator that allows to access I/O ports. Interrupts need to be supported as well, and primitive Deleted: /pypy/trunk/doc/funding/B6.wp09_embedded_device.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp09_embedded_device.txt Sat Sep 20 15:47:20 2003 +++ (empty file) @@ -1,56 +0,0 @@ -.. include:: crossreferences.asc - -Workpackage Description -======================== - -================================ ========================== -**Workpackage number** 09 --------------------------------- -------------------------- -**Start date or starting event** not before WP05_ -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y Z ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 2 2 2 -================================= ======= ======= ======= ======= ======= ======= ======= ======= - - -**Objectives** - -Title: Porting PyPy to an Embedded Device - -- Supporting a completely different platform -- Providing an efficient, high-level Language for - embedded Devices -- Proving the flexibility of the PyPy idea - - -**Description of work** - -- Analyse the differences between the new architecture and - PyPy's current. Deside whether to optimize for code size - or applicatin speed. - -- Develop the translation process, run-time and those optimisations - that depend on low-level details. - -- Implement a simulator for the embedded device. - -- Implement interfaces to the needed external libraries. - -- Combine different solutions for the different issues discussed in - High-performance (WP06_, WP07_, WP08_). - Gather statistics with real-work Python applications. - Compare the results. This is where the flexibility of the whole - project is vital. Typically, very different trade-offs need to be - made on different environments. - - -**Deliverables** - -- DL??? Python running on the embedded device. -- DL??? Passing all applicable elements of the standard test suite. - -**Milestones and Expected Result** - Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Sat Sep 20 15:47:20 2003 @@ -6,5 +6,5 @@ .. _WP06: B6.wp06_several_obj_impl.html .. _WP07: B6.wp07_translator_optimisations.html .. _WP08: B6.wp08_integrate_technology.html -.. _WP09: B6.wp09_embedded_device.html +.. _WP10: B6.wp10_embedded_device.html .. _WP11: B6.wp11_small_os.html From tismer at codespeak.net Sat Sep 20 15:57:59 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 20 Sep 2003 15:57:59 +0200 (MEST) Subject: [pypy-svn] rev 1373 - pypy/trunk/doc/funding Message-ID: <20030920135759.5C6A45C365@thoth.codespeak.net> Author: tismer Date: Sat Sep 20 15:57:58 2003 New Revision: 1373 Added: pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt - copied, changed from rev 1370, pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt - copied, changed from rev 1370, pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt Removed: pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/crossreferences.asc Log: splitted Psyco and Stackless into two packages. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Sat Sep 20 15:57:58 2003 @@ -204,13 +204,20 @@ - optimisation towards minimal code size for embedded systems. -WP08_: Integration of Existing Technology -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Integration of Existing Technology +----------------------------------- There are existing projects, notably Psyco and Stackless, which have been traditionally depended on closely following CPython's code base. +Both will be rewritten as a meta-component that hooks into the +translator plus a dedicated run-time component (WP05_). + + +WP08_: Integration of Psyco +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Psyco provides techniques to increase the performance of C Python by generating specialized machine code at run-time. Developing C Psyco has proved that more flexibility would be of @@ -219,6 +226,9 @@ RPython code, enabling yet more efficient optimisations and allowing new processor architectures to be targeted. +WP09_: Integration of Stackless +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Stackless Python has implemented high speed multitasking in a single thread for C Python in two different ways: Continuation passing and stack switching. Both ways of @@ -227,9 +237,6 @@ Stackless Python and should enable PyPy for load-balancing between different machines. -Both will be rewritten as a meta-component that hooks into the -translator plus a dedicated run-time component (WP05_). - Supporting Embedded Devices ------------------------------------------ Copied: pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt (from rev 1370, pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt (original) +++ pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt Sat Sep 20 15:57:58 2003 @@ -18,17 +18,12 @@ **Objectives** -Title: Integration of Existing Technology +Title: Integration of Psyco - Integration of Psyco's technology into PyPy. - Optimization by Creating Specialized Machine Code at Run-time. - Adressing new Processor Architectures -- Integration of Stackless's Technology into PyPy. - - - Enable Massive Parallelism in a Single Thread - - Allow Pickling of a Running Program - **Description of work** @@ -43,18 +38,6 @@ - Implement a dedicated run-time component (WP05_) for Psyco. - - -- Determine which parts of Stackless need to be written in RPython. - -- Design Stackless as a meta-component - that hooks into the translator (WP03_) - -- Implement Stackless partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP05_) for Stackless. - - Further develop these technologies based on the results gathered in WP07_, e.g. identify when these technologies would guide specific choices among the solutions developed in WP05_ and studying several solutions in WP06_. @@ -62,7 +45,7 @@ **Deliverables** -- DL??? to be defined. Speed boost, architectures, green threads, pickling. +- DL??? to be defined. Speed boost, architectures. - DL??? Passing all applicable elements of the standard test suite. **Milestones and Expected Result** Deleted: /pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt Sat Sep 20 15:57:58 2003 +++ (empty file) @@ -1,69 +0,0 @@ -.. include:: crossreferences.asc - -Workpackage Description -======================== - -================================ ========================== -**Workpackage number** 08 --------------------------------- -------------------------- -**Start date or starting event** 0 -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** AR CT X ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 4 2 2 -================================= ======= ======= ======= ======= ======= ======= ======= ======= - - -**Objectives** - -Title: Integration of Existing Technology - -- Integration of Psyco's technology into PyPy. - - - Optimization by Creating Specialized Machine Code at Run-time. - - Adressing new Processor Architectures -- Integration of Stackless's Technology into PyPy. - - - Enable Massive Parallelism in a Single Thread - - Allow Pickling of a Running Program - - -**Description of work** - - -- Determine which parts of Psyco need to be written in RPython. - -- Design Psyco as a meta-component - that hooks into the translator (WP03_) - -- Implement Psyco partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP05_) for Psyco. - - - -- Determine which parts of Stackless need to be written in RPython. - -- Design Stackless as a meta-component - that hooks into the translator (WP03_) - -- Implement Stackless partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP05_) for Stackless. - -- Further develop these technologies based on the results gathered in WP07_, - e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP06_. - - -**Deliverables** - -- DL??? to be defined. Speed boost, architectures, green threads, pickling. -- DL??? Passing all applicable elements of the standard test suite. - -**Milestones and Expected Result** - Copied: pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt (from rev 1370, pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp08_integrate_technology.txt (original) +++ pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt Sat Sep 20 15:57:58 2003 @@ -4,7 +4,7 @@ ======================== ================================ ========================== -**Workpackage number** 08 +**Workpackage number** 09 -------------------------------- -------------------------- **Start date or starting event** 0 ================================ ========================== @@ -18,12 +18,8 @@ **Objectives** -Title: Integration of Existing Technology +Title: Integration of Stackless -- Integration of Psyco's technology into PyPy. - - - Optimization by Creating Specialized Machine Code at Run-time. - - Adressing new Processor Architectures - Integration of Stackless's Technology into PyPy. - Enable Massive Parallelism in a Single Thread @@ -33,18 +29,6 @@ **Description of work** -- Determine which parts of Psyco need to be written in RPython. - -- Design Psyco as a meta-component - that hooks into the translator (WP03_) - -- Implement Psyco partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP05_) for Psyco. - - - - Determine which parts of Stackless need to be written in RPython. - Design Stackless as a meta-component @@ -62,7 +46,7 @@ **Deliverables** -- DL??? to be defined. Speed boost, architectures, green threads, pickling. +- DL??? to be defined. Green threads, pickling. - DL??? Passing all applicable elements of the standard test suite. **Milestones and Expected Result** Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Sat Sep 20 15:57:58 2003 @@ -5,6 +5,7 @@ .. _WP05: B6.wp05_bootstrap_runtime.html .. _WP06: B6.wp06_several_obj_impl.html .. _WP07: B6.wp07_translator_optimisations.html -.. _WP08: B6.wp08_integrate_technology.html +.. _WP08: B6.wp08_integrate_psyco.html +.. _WP09: B6.wp09_integrate_stackless.html .. _WP10: B6.wp10_embedded_device.html .. _WP11: B6.wp11_small_os.html From tismer at codespeak.net Sat Sep 20 16:27:05 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 20 Sep 2003 16:27:05 +0200 (MEST) Subject: [pypy-svn] rev 1374 - pypy/trunk/doc/funding Message-ID: <20030920142705.E19905B671@thoth.codespeak.net> Author: tismer Date: Sat Sep 20 16:27:05 2003 New Revision: 1374 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt Log: small improvements Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Sat Sep 20 16:27:05 2003 @@ -208,12 +208,15 @@ ----------------------------------- There are existing projects, notably Psyco and Stackless, -which have been traditionally depended on closely +which have been traditionally dependent on closely following CPython's code base. Both will be rewritten as a meta-component that hooks into the translator plus a dedicated run-time component (WP05_). +As a side effect, after successful completion of the +PyPy project, they no longer need to exist as stand-alone +projects. WP08_: Integration of Psyco ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From pedronis at codespeak.net Sat Sep 20 17:22:40 2003 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 20 Sep 2003 17:22:40 +0200 (MEST) Subject: [pypy-svn] rev 1376 - pypy/trunk/doc/funding Message-ID: <20030920152240.E10F15C36A@thoth.codespeak.net> Author: pedronis Date: Sat Sep 20 17:22:40 2003 New Revision: 1376 Modified: pypy/trunk/doc/funding/mkxref.py (contents, props changed) pypy/trunk/doc/funding/renamewp.py (contents, props changed) pypy/trunk/doc/funding/uncrappify.py (contents, props changed) Log: set eol-style. Modified: pypy/trunk/doc/funding/mkxref.py ============================================================================== --- pypy/trunk/doc/funding/mkxref.py (original) +++ pypy/trunk/doc/funding/mkxref.py Sat Sep 20 17:22:40 2003 @@ -1,85 +1,85 @@ -# mkxref.py -# -# Generate crossreferences.asc -# as a global include file for all text files here. -# - -import os, sys, re - -# find all work package files, like -# B6.wp02_cpython_to_python.txt - -START_STR = "B6.wp".lower() - -def get_wps_from_dir(as_html=False): - res = {} - for fname in os.listdir("."): - flow = fname.lower() - pre, ext = os.path.splitext(flow) - if flow.startswith(START_STR) and ext == ".txt": - num = flow[len(START_STR):].split("_", 1)[0] - if len(num)==2 and num.isdigit(): - if as_html: - pre, ext = os.path.splitext(fname) - newfname = pre + ".html" - else: - newfname = fname - res["WP"+num] = newfname - return res - -REF_FILE = "B6.1_plan_introduction.txt" - -def get_wps_from_file(): - # read the file and find all WPnn_ references - txt = file(REF_FILE).read() - res = {} - for piece in txt.split("WP"): - num = piece.split("_", 1)[0] - if len(num)==2 and num.isdigit(): - res["WP"+num] = "" - return res - -# read the existing xref file, find the WP section and replace it. - -XREF_FILE = "crossreferences.asc" - -def split_xref_text(): - """ split the xref file into three parts: - before, in, and after the workpackages - """ - try: - lines = file(XREF_FILE).readlines() - except IOError: - lines = [] - pre = [] - pat = "..\s+_WP\d\d\:|.. attention::" - for line in lines: - if re.match(pat, line): - break - pre.append(line) - post = lines[len(pre):] - mid = [] - for line in post[:]: - if re.match(pat, line): - mid.append(post.pop(0)) - else: - break - return pre, mid, post - -def build_xref_file(): - pre, old, post = split_xref_text() - wps = get_wps_from_file() - wps.update(get_wps_from_dir(as_html=True)) - wplist = wps.items() - wplist.sort() - mid = [ ".. _%s: %s\n" % (name, url) for name, url in wplist] - empties = [name for name, url in wplist if not url] - if empties: - n = len(empties) - mid.append(".. attention:: **%d file%s missing: %s**\n" % - (n, "s"[0:n>1], " ".join(empties))) - if mid != old: - file(XREF_FILE, "w").writelines( pre + mid + post) - -if __name__ == "__main__": +# mkxref.py +# +# Generate crossreferences.asc +# as a global include file for all text files here. +# + +import os, sys, re + +# find all work package files, like +# B6.wp02_cpython_to_python.txt + +START_STR = "B6.wp".lower() + +def get_wps_from_dir(as_html=False): + res = {} + for fname in os.listdir("."): + flow = fname.lower() + pre, ext = os.path.splitext(flow) + if flow.startswith(START_STR) and ext == ".txt": + num = flow[len(START_STR):].split("_", 1)[0] + if len(num)==2 and num.isdigit(): + if as_html: + pre, ext = os.path.splitext(fname) + newfname = pre + ".html" + else: + newfname = fname + res["WP"+num] = newfname + return res + +REF_FILE = "B6.1_plan_introduction.txt" + +def get_wps_from_file(): + # read the file and find all WPnn_ references + txt = file(REF_FILE).read() + res = {} + for piece in txt.split("WP"): + num = piece.split("_", 1)[0] + if len(num)==2 and num.isdigit(): + res["WP"+num] = "" + return res + +# read the existing xref file, find the WP section and replace it. + +XREF_FILE = "crossreferences.asc" + +def split_xref_text(): + """ split the xref file into three parts: + before, in, and after the workpackages + """ + try: + lines = file(XREF_FILE).readlines() + except IOError: + lines = [] + pre = [] + pat = "..\s+_WP\d\d\:|.. attention::" + for line in lines: + if re.match(pat, line): + break + pre.append(line) + post = lines[len(pre):] + mid = [] + for line in post[:]: + if re.match(pat, line): + mid.append(post.pop(0)) + else: + break + return pre, mid, post + +def build_xref_file(): + pre, old, post = split_xref_text() + wps = get_wps_from_file() + wps.update(get_wps_from_dir(as_html=True)) + wplist = wps.items() + wplist.sort() + mid = [ ".. _%s: %s\n" % (name, url) for name, url in wplist] + empties = [name for name, url in wplist if not url] + if empties: + n = len(empties) + mid.append(".. attention:: **%d file%s missing: %s**\n" % + (n, "s"[0:n>1], " ".join(empties))) + if mid != old: + file(XREF_FILE, "w").writelines( pre + mid + post) + +if __name__ == "__main__": build_xref_file() \ No newline at end of file Modified: pypy/trunk/doc/funding/renamewp.py ============================================================================== --- pypy/trunk/doc/funding/renamewp.py (original) +++ pypy/trunk/doc/funding/renamewp.py Sat Sep 20 17:22:40 2003 @@ -1,55 +1,55 @@ -# renaming a working package. -usage = """usage: -python renamewp.py old new -where old and new are decimal WP numbers in the range 1-99. -""" - -import os, sys, mkxref - -def do_rename(old, new): - oldS = "WP%02d" % old - newS = "WP%02d" % new - wpfiles = mkxref.get_wps_from_dir() - if wpfiles.has_key(oldS): - if wpfiles.has_key(newS): - raise ValueError, "cannot rename because %s is in the way!" % wpfiles[newS] - oldName = wpfiles[oldS] - p = len(mkxref.START_STR) - newName = "%s%02d%s" % (oldName[:p], new, oldName[p+2:]) - cmd = 'svn move "%s" "%s"' % (oldName, newName) - print ">>>", cmd - err = os.system(cmd) - if err: - print "aborting due to the above error" - sys.exit(1) - del wpfiles[oldS] - wpfiles[newS] = newName - workfiles = wpfiles.values() - workfiles.append(mkxref.REF_FILE) - find = oldS+"_" - repl = newS+"_" - for fname in workfiles: - txt = file(fname).read() - if txt.find(find) >= 0: - print "rewriting %s" % fname - txt = txt.replace(find, repl) - file(fname, "w").write(txt) - print "re-creating cross references" - mkxref.build_xref_file() - print "done." - - -def main(): - args = sys.argv[1:] - try: - old, new = map(int, args) - if min(old, new) < 1 or max(old, new) > 99: - raise ValueError - except: - print usage - sys.exit(1) - do_rename(old, new) - -if __name__ == "__main__": - main() +# renaming a working package. +usage = """usage: +python renamewp.py old new +where old and new are decimal WP numbers in the range 1-99. +""" + +import os, sys, mkxref + +def do_rename(old, new): + oldS = "WP%02d" % old + newS = "WP%02d" % new + wpfiles = mkxref.get_wps_from_dir() + if wpfiles.has_key(oldS): + if wpfiles.has_key(newS): + raise ValueError, "cannot rename because %s is in the way!" % wpfiles[newS] + oldName = wpfiles[oldS] + p = len(mkxref.START_STR) + newName = "%s%02d%s" % (oldName[:p], new, oldName[p+2:]) + cmd = 'svn move "%s" "%s"' % (oldName, newName) + print ">>>", cmd + err = os.system(cmd) + if err: + print "aborting due to the above error" + sys.exit(1) + del wpfiles[oldS] + wpfiles[newS] = newName + workfiles = wpfiles.values() + workfiles.append(mkxref.REF_FILE) + find = oldS+"_" + repl = newS+"_" + for fname in workfiles: + txt = file(fname).read() + if txt.find(find) >= 0: + print "rewriting %s" % fname + txt = txt.replace(find, repl) + file(fname, "w").write(txt) + print "re-creating cross references" + mkxref.build_xref_file() + print "done." + + +def main(): + args = sys.argv[1:] + try: + old, new = map(int, args) + if min(old, new) < 1 or max(old, new) > 99: + raise ValueError + except: + print usage + sys.exit(1) + do_rename(old, new) + +if __name__ == "__main__": + main() \ No newline at end of file Modified: pypy/trunk/doc/funding/uncrappify.py ============================================================================== --- pypy/trunk/doc/funding/uncrappify.py (original) +++ pypy/trunk/doc/funding/uncrappify.py Sat Sep 20 17:22:40 2003 @@ -1,14 +1,14 @@ -# very simple repair to Armin's mess :-) - -import os, sys - -if sys.platform != "win32": - raise TypeError, "this won't work on %s" % sys.platform - -for fname in os.listdir("."): - pre, ext = os.path.splitext(fname) - if ext.lower() in (".txt", ".py"): - txt = file(fname, "rb").read() - txt = txt.replace("\r\n\n\r\n\n", "\r\n") - file(fname, "wb").write(txt) +# very simple repair to Armin's mess :-) + +import os, sys + +if sys.platform != "win32": + raise TypeError, "this won't work on %s" % sys.platform + +for fname in os.listdir("."): + pre, ext = os.path.splitext(fname) + if ext.lower() in (".txt", ".py"): + txt = file(fname, "rb").read() + txt = txt.replace("\r\n\n\r\n\n", "\r\n") + file(fname, "wb").write(txt) \ No newline at end of file From pedronis at codespeak.net Sat Sep 20 17:48:10 2003 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 20 Sep 2003 17:48:10 +0200 (MEST) Subject: [pypy-svn] rev 1377 - pypy/trunk/doc/funding Message-ID: <20030920154810.C35215C36B@thoth.codespeak.net> Author: pedronis Date: Sat Sep 20 17:48:10 2003 New Revision: 1377 Modified: pypy/trunk/doc/funding/crossreferences.asc (contents, props changed) Log: set eol-style. should be ok. Christian's scripts use text mode. Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Sat Sep 20 17:48:10 2003 @@ -1,11 +1,11 @@ -.. _WP01: B6.wp01_interpreter_completion.html -.. _WP02: B6.wp02_cpython_to_python.html -.. _WP03: B6.wp03_translation_of_rpython.html -.. _WP04: B6.wp04_bootstrap_redesign.html -.. _WP05: B6.wp05_bootstrap_runtime.html -.. _WP06: B6.wp06_several_obj_impl.html -.. _WP07: B6.wp07_translator_optimisations.html -.. _WP08: B6.wp08_integrate_psyco.html -.. _WP09: B6.wp09_integrate_stackless.html -.. _WP10: B6.wp10_embedded_device.html -.. _WP11: B6.wp11_small_os.html +.. _WP01: B6.wp01_interpreter_completion.html +.. _WP02: B6.wp02_cpython_to_python.html +.. _WP03: B6.wp03_translation_of_rpython.html +.. _WP04: B6.wp04_bootstrap_redesign.html +.. _WP05: B6.wp05_bootstrap_runtime.html +.. _WP06: B6.wp06_several_obj_impl.html +.. _WP07: B6.wp07_translator_optimisations.html +.. _WP08: B6.wp08_integrate_psyco.html +.. _WP09: B6.wp09_integrate_stackless.html +.. _WP10: B6.wp10_embedded_device.html +.. _WP11: B6.wp11_small_os.html From tismer at codespeak.net Sat Sep 20 19:57:19 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 20 Sep 2003 19:57:19 +0200 (MEST) Subject: [pypy-svn] rev 1378 - pypy/trunk/doc/funding Message-ID: <20030920175719.587385C377@thoth.codespeak.net> Author: tismer Date: Sat Sep 20 19:57:18 2003 New Revision: 1378 Added: pypy/trunk/doc/funding/STREP-template.doc (contents, props changed) pypy/trunk/doc/funding/STREP-template.pdf (contents, props changed) Log: added this fine document about preparing a strep. The PDF was created by a Win2PDF demo, see the annoyingly added last page :-) Added: pypy/trunk/doc/funding/STREP-template.doc ============================================================================== Binary file. No diff available. Added: pypy/trunk/doc/funding/STREP-template.pdf ============================================================================== Binary file. No diff available. From tismer at codespeak.net Sat Sep 20 20:47:42 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 20 Sep 2003 20:47:42 +0200 (MEST) Subject: [pypy-svn] rev 1379 - pypy/trunk/doc/funding Message-ID: <20030920184742.26E9B5C385@thoth.codespeak.net> Author: tismer Date: Sat Sep 20 20:47:42 2003 New Revision: 1379 Added: pypy/trunk/doc/funding/default.css Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp11_small_os.txt Log: moving foward. I think I have to add an extra WP which investigates the necessary extensions for writing an OS. See if we can create ROMable code See if we can write a prioritized scheduler See if we can write an IP stack on our own! Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Sat Sep 20 20:47:42 2003 @@ -273,6 +273,11 @@ to the code generator that allows to access I/O ports. Interrupts need to be supported as well, and primitive access to persistant storage. +Ideally, we can create a single threaded PyPy OS with +a prioritized scheduler that runs the OS parts together +with multiple Python application. The latter depends on +WP09_ very much. + Load Balancing in a Multi-Processor environment --------------------------------------------------- Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp11_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp11_small_os.txt Sat Sep 20 20:47:42 2003 @@ -20,38 +20,21 @@ Title: A small Operating System for an Embedded Device -XXX the following is still WP09, write it!!! -- Supporting a completely different platform -- Providing an efficient, high-level Language for - embedded Devices -- Proving the flexibility of the PyPy idea +- Building all parts of a small Operating System **Description of work** -- Analyse the differences between the new architecture and - PyPy's current. Deside whether to optimize for code size - or applicatin speed. - -- Develop the translation process, run-time and those optimisations - that depend on low-level details. - -- Implement a simulator for the embedded device. - -- Implement interfaces to the needed external libraries. - -- Combine different solutions for the different issues discussed in - High-performance (WP06_, WP07_, WP08_). - Gather statistics with real-work Python applications. - Compare the results. This is where the flexibility of the whole - project is vital. Typically, very different trade-offs need to be - made on different environments. +- Analyse the existing device drivers for the embedded device + and re-code them in Python.### +*Gack! It looks like I have to have another working package +that adds scheduling and interrupt support to PyPy!* **Deliverables** -- DL??? Python running on the embedded device. +- DL??? Python implements the whole OS of the embedded device. - DL??? Passing all applicable elements of the standard test suite. **Milestones and Expected Result** Added: pypy/trunk/doc/funding/default.css ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/default.css Sat Sep 20 20:47:42 2003 @@ -0,0 +1,224 @@ +/* +:Author: David Goodger +:Contact: goodger at users.sourceforge.net +:date: $Date: 2003/09/01 15:27:01 $ +:version: $Revision: 1.35 $ +:copyright: This stylesheet has been placed in the public domain. + +Default cascading style sheet for the HTML output of Docutils. +*/ + +.first { + margin-top: 0 } + +.last { + margin-bottom: 0 } + +a.toc-backref { + text-decoration: none ; + color: black } + +dd { + margin-bottom: 0.5em } + +div.abstract { + margin: 2em 5em } + +div.abstract p.topic-title { + font-weight: bold ; + text-align: center } + +div.attention, div.caution, div.danger, div.error, div.hint, +div.important, div.note, div.tip, div.warning, div.admonition { + margin: 2em ; + border: medium outset ; + padding: 1em } + +div.attention p.admonition-title, div.caution p.admonition-title, +div.danger p.admonition-title, div.error p.admonition-title, +div.warning p.admonition-title { + color: red ; + font-weight: bold ; + font-family: sans-serif } + +div.hint p.admonition-title, div.important p.admonition-title, +div.note p.admonition-title, div.tip p.admonition-title, +div.admonition p.admonition-title { + font-weight: bold ; + font-family: sans-serif } + +div.dedication { + margin: 2em 5em ; + text-align: center ; + font-style: italic } + +div.dedication p.topic-title { + font-weight: bold ; + font-style: normal } + +div.figure { + margin-left: 2em } + +div.footer, div.header { + font-size: smaller } + +div.sidebar { + margin-left: 1em ; + border: medium outset ; + padding: 0em 1em ; + background-color: #ffffee ; + width: 40% ; + float: right ; + clear: right } + +div.sidebar p.rubric { + font-family: sans-serif ; + font-size: medium } + +div.system-messages { + margin: 5em } + +div.system-messages h1 { + color: red } + +div.system-message { + border: medium outset ; + padding: 1em } + +div.system-message p.system-message-title { + color: red ; + font-weight: bold } + +div.topic { + margin: 2em } + +h1.title { + text-align: center } + +h2.subtitle { + text-align: center } + +hr { + width: 75% } + +ol.simple, ul.simple { + margin-bottom: 1em } + +ol.arabic { + list-style: decimal } + +ol.loweralpha { + list-style: lower-alpha } + +ol.upperalpha { + list-style: upper-alpha } + +ol.lowerroman { + list-style: lower-roman } + +ol.upperroman { + list-style: upper-roman } + +p.attribution { + text-align: right ; + margin-left: 50% } + +p.caption { + font-style: italic } + +p.credits { + font-style: italic ; + font-size: smaller } + +p.label { + white-space: nowrap } + +p.rubric { + font-weight: bold ; + font-size: larger ; + color: maroon ; + text-align: center } + +p.sidebar-title { + font-family: sans-serif ; + font-weight: bold ; + font-size: larger } + +p.sidebar-subtitle { + font-family: sans-serif ; + font-weight: bold } + +p.topic-title { + font-weight: bold } + +pre.address { + margin-bottom: 0 ; + margin-top: 0 ; + font-family: serif ; + font-size: 100% } + +pre.line-block { + font-family: serif ; + font-size: 100% } + +pre.literal-block, pre.doctest-block { + margin-left: 2em ; + margin-right: 2em ; + background-color: #eeeeee } + +span.classifier { + font-family: sans-serif ; + font-style: oblique } + +span.classifier-delimiter { + font-family: sans-serif ; + font-weight: bold } + +span.interpreted { + font-family: sans-serif } + +span.option { + white-space: nowrap } + +span.option-argument { + font-style: italic } + +span.pre { + white-space: pre } + +span.problematic { + color: red } + +table { + margin-top: 0.5em ; + margin-bottom: 0.5em } + +table.citation { + border-left: solid thin gray ; + padding-left: 0.5ex } + +table.docinfo { + margin: 2em 4em } + +table.footnote { + border-left: solid thin black ; + padding-left: 0.5ex } + +td, th { + padding-left: 0.5em ; + padding-right: 0.5em ; + vertical-align: top } + +th.docinfo-name, th.field-name { + font-weight: bold ; + text-align: left ; + white-space: nowrap } + +h1 tt, h2 tt, h3 tt, h4 tt, h5 tt, h6 tt { + font-size: 100% } + +tt { + background-color: #eeeeee } + +ul.auto-toc { + list-style-type: none } From tismer at codespeak.net Sat Sep 20 21:02:49 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 20 Sep 2003 21:02:49 +0200 (MEST) Subject: [pypy-svn] rev 1380 - pypy/trunk/doc/funding Message-ID: <20030920190249.4152F5C386@thoth.codespeak.net> Author: tismer Date: Sat Sep 20 21:02:48 2003 New Revision: 1380 Modified: pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt pypy/trunk/doc/funding/B6.wp11_small_os.txt Log: Augmented the Stackless stuff with support for interrupts and such. This avoids having yet another WP, and if I want to implement an OS at all, then one without threads, just tasklets! Modified: pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt (original) +++ pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt Sat Sep 20 21:02:48 2003 @@ -22,9 +22,9 @@ - Integration of Stackless's Technology into PyPy. - - Enable Massive Parallelism in a Single Thread - - Allow Pickling of a Running Program +- Enable Massive Parallelism in a Single Thread +- Allow Pickling of a Running Program **Description of work** @@ -39,6 +39,9 @@ - Implement a dedicated run-time component (WP05_) for Stackless. +- Implement a single-threaded, pre-emptive scheduler with + priorities. This is necessary to support small OSes, like WP11_. + - Further develop these technologies based on the results gathered in WP07_, e.g. identify when these technologies would guide specific choices among the solutions developed in WP05_ and studying several solutions in WP06_. Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp11_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp11_small_os.txt Sat Sep 20 21:02:48 2003 @@ -26,11 +26,23 @@ **Description of work** -- Analyse the existing device drivers for the embedded device - and re-code them in Python.### +- Analyse the existing device drivers for the embedded device. + Figure out which drivers should be interfaced to, which + parts of the BIOS must be interfaced, and what has to be + recoded in Python. At least all drivers which have + blocking calls must be recoded in in Python. + +- Recode the according device drivers in Python. Design tests + which ensure functional compatibility with the original ones, + as far as these exist. + +- Extend the simulator of WP10_ accordingly. + +- Implement the OS kernel as a set of prioritized tasklets + (WP09_) + +- Test the implementation both on the simulator and on the real hardware. -*Gack! It looks like I have to have another working package -that adds scheduling and interrupt support to PyPy!* **Deliverables** From tismer at codespeak.net Sat Sep 20 21:17:50 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 20 Sep 2003 21:17:50 +0200 (MEST) Subject: [pypy-svn] rev 1381 - pypy/trunk/doc/funding Message-ID: <20030920191750.1844D5C389@thoth.codespeak.net> Author: tismer Date: Sat Sep 20 21:17:49 2003 New Revision: 1381 Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt Log: small additions. Please **review** Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp11_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp11_small_os.txt Sat Sep 20 21:17:49 2003 @@ -32,6 +32,10 @@ recoded in Python. At least all drivers which have blocking calls must be recoded in in Python. +- Check whether we need to implement a TCP/IP stack and if this + work is doable in reasonable time. If not, then cancel this + WP in favor of another application WP to be done instead. + - Recode the according device drivers in Python. Design tests which ensure functional compatibility with the original ones, as far as these exist. @@ -41,6 +45,12 @@ - Implement the OS kernel as a set of prioritized tasklets (WP09_) +- Implement a simple file system which is accessible by + Python, only. + +- Implement a few basic protocols like FTP in order to provide + basic access to the internal data of the embedded device. + - Test the implementation both on the simulator and on the real hardware. From tismer at codespeak.net Sat Sep 20 21:40:52 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 20 Sep 2003 21:40:52 +0200 (MEST) Subject: [pypy-svn] rev 1382 - pypy/trunk/doc/funding Message-ID: <20030920194052.687155C38B@thoth.codespeak.net> Author: tismer Date: Sat Sep 20 21:40:51 2003 New Revision: 1382 Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt Log: small corrections Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp11_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp11_small_os.txt Sat Sep 20 21:40:51 2003 @@ -4,7 +4,7 @@ ======================== ================================ ========================== -**Workpackage number** 10 +**Workpackage number** 11 -------------------------------- -------------------------- **Start date or starting event** not before WP05_ ================================ ========================== @@ -43,7 +43,7 @@ - Extend the simulator of WP10_ accordingly. - Implement the OS kernel as a set of prioritized tasklets - (WP09_) + (using the support from WP09_) - Implement a simple file system which is accessible by Python, only. From lac at codespeak.net Sun Sep 21 15:53:40 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Sun, 21 Sep 2003 15:53:40 +0200 (MEST) Subject: [pypy-svn] rev 1383 - pypy/trunk/doc/funding Message-ID: <20030921135340.149605A2A9@thoth.codespeak.net> Author: lac Date: Sun Sep 21 15:53:40 2003 New Revision: 1383 Modified: pypy/trunk/doc/funding/B2.0.0_background.txt Log: Added note saying why I am moving this. Modified: pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_background.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_background.txt Sun Sep 21 15:53:40 2003 @@ -1,6 +1,10 @@ B2.0.0 background ================= +THIS WE NEED TO MAKE APPENDIX A. We can canabalise as we like for +other sections. But the Message from Stockholm is, 'don't give them +a background like this. The evaluators get bored' + The relevance of Python ----------------------- From tismer at codespeak.net Mon Sep 22 11:44:53 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 11:44:53 +0200 (MEST) Subject: [pypy-svn] rev 1384 - pypy/trunk/doc/funding Message-ID: <20030922094453.ADD995B356@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 11:44:52 2003 New Revision: 1384 Added: pypy/trunk/doc/funding/wp-tablebegin.asc pypy/trunk/doc/funding/wp-tableend.asc pypy/trunk/doc/funding/wp-template.htm Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt pypy/trunk/doc/funding/B6.wp10_embedded_device.txt pypy/trunk/doc/funding/B6.wp11_small_os.txt pypy/trunk/doc/funding/crossreferences.asc pypy/trunk/doc/funding/renamewp.py Log: Changed the layout of all working packages. They now use some raw html inclusions, which produce nearly exact the result that we want to see after import of the HTML into Open Office. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 11:44:52 2003 @@ -95,22 +95,6 @@ optimisation of arbitrary but suitably restricted Python application or parts thereof. -**Part a): Code Analysis** - -Analyse code to produce the relevant typing information. Investigate -if we can use the annotation object space only or if additional -AST-based control flow analysis is needed. Give a formal definition -of RPython. - -**Part b): Code Production** - -Produce low-level code out of the data gathered in Part a). Again -investigate how this is best done (AST-guided translation or -reverse-engineering of the low-level control flow gathered by the -annotation object space). Compare different low-level environments that -we could target (C, Pyrex, others?). - - _`Bootstrapping` PyPy ------------------------------ @@ -248,7 +232,7 @@ memory size, which either limits the power of software that is implemented for these platforms, or enforces use of low-level approaches like C/C++ or Java. -PyPy is especially suited to support such platform, since +PyPy is especially suited to support such platforms, since it can produce highly effective, compact code, while retaining the abstraction and ease-of-use of Python. @@ -279,13 +263,6 @@ WP09_ very much. -Load Balancing in a Multi-Processor environment ---------------------------------------------------- - - -*XXX to be finished, collecting snippets* - - Numerical Applications ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ PyPy is extremely flexible, since it supports static @@ -320,6 +297,12 @@ enabled to emit optimized code for that. +Load Balancing in a Multi-Processor environment +--------------------------------------------------- + +*XXX to be finished, collecting snippets* + + Infrastructure ------------------ Modified: pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt (original) +++ pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt Mon Sep 22 11:44:52 2003 @@ -1,29 +1,34 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Development and Completion of the PyPy Interpreter +.. |wp| replace:: WP01 +.. |start| replace:: 0 +.. |p1| replace:: X +.. |m1| replace:: 3 +.. |p2| replace:: Y +.. |m2| replace:: 3 +.. |p3| replace:: Z +.. |m3| replace:: 3 +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 01 --------------------------------- -------------------------- -**Start date or starting event** 0 -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y Z ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 3 3 3 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: Development and Completion of the PyPy Interpreter - The PyPy interpreter itself should be developed and completed as a regular Python program. This package includes all parts of CPython that we don't want to move to WP02_. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Description of work** - Coordinate with WP02_ which parts of CPython @@ -33,12 +38,18 @@ concepts that the standard object space is based on, and how to hook in the bytecode compiler. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Deliverables** - DL01: First complete Python Implementation +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Milestones and Expected Result** - Initial Prototype + +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt (original) +++ pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt Mon Sep 22 11:44:52 2003 @@ -1,30 +1,34 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Translation of CPython into regular Python +.. |wp| replace:: WP02 +.. |start| replace:: 0 +.. |p1| replace:: X +.. |m1| replace:: 3 +.. |p2| replace:: Y +.. |m2| replace:: 3 +.. |p3| replace:: Z +.. |m3| replace:: 3 +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 02 --------------------------------- -------------------------- -**Start date or starting event** 0 -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y Z ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 3 3 3 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: Translation of CPython into regular Python - Translate all parts of CPython which we don't want to implement in WP01_ into regular Python libraries. Note: These ones should also work without PyPy, being just plain-Python replacements for existing CPython functionality. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Description of work** @@ -34,14 +38,19 @@ become a regular Python program, instead of being built into the interpreter. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Deliverables** - DL01: First complete Python Implementation - DL??? Passing all applicable elements of the standard test suite. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Milestones and Expected Result** - Initial Prototype +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt (original) +++ pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Mon Sep 22 11:44:52 2003 @@ -1,31 +1,35 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Translation of RPython +.. |wp| replace:: WP03 +.. |start| replace:: 0 +.. |p1| replace:: X +.. |m1| replace:: 3 +.. |p2| replace:: Y +.. |m2| replace:: 3 +.. |p3| replace:: |e| +.. |m3| replace:: |e| +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 03 --------------------------------- -------------------------- -**Start date or starting event** end of WP02_/WP03_ -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 3 3 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: Translation of RPython - - Ability to translate arbitrary RPython source code (e.g. the one produced in WP01_ into low-level code (C, Pyrex, Java, others). - Creation of a stand-alone, not-PyPy-related tool for general optimization of arbitrary but suitably restricted Python application or parts thereof. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Description of work** @@ -48,12 +52,18 @@ - Provide some measure for the efficiency of the generated code, in comparison to Standard Python. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Deliverables** - DL02: Stand-alone Python Interpreter - DL03: Tool for Analysis of restricted Python Programs - DL??? Passing all applicable elements of the standard test suite. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Milestones and Expected Result** - Initial Complete Self-Contained PyPy Interpreter Modified: pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt (original) +++ pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt Mon Sep 22 11:44:52 2003 @@ -1,31 +1,37 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Specific Analysis and Redesign +.. |wp| replace:: WP04 +.. |start| replace:: 0 +.. |p1| replace:: X +.. |m1| replace:: 3 +.. |p2| replace:: Y +.. |m2| replace:: 3 +.. |p3| replace:: |e| +.. |m3| replace:: |e| +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 04 --------------------------------- -------------------------- -**Start date or starting event** WP03_ -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 3 3 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: Specific Analysis and Redesign - - Putting interpreter (WP01_, WP02_) and translator (WP03_) together. - Enhancing the PyPy Design. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Description of work** +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Part a): Specific Analysis and Redesign** @@ -36,12 +42,14 @@ - Re-iterate this until we agree to have reached an optimum solution. - +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Deliverables** - +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Milestones and Expected Result** - +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt (original) +++ pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Mon Sep 22 11:44:52 2003 @@ -1,31 +1,34 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Low Level PyPy Runtime +.. |wp| replace:: WP05 +.. |start| replace:: WP03_ +.. |p1| replace:: X +.. |m1| replace:: 2 +.. |p2| replace:: Y +.. |m2| replace:: 2 +.. |p3| replace:: |e| +.. |m3| replace:: |e| +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 05 --------------------------------- -------------------------- -**Start date or starting event** WP03_ -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 2 2 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: Low Level PyPy Runtime - - Providing a Runtime Library for PyPy +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Description of work** - In order to give a working environment to the translated RPython program, we need to build the low-level-specific **run-time** components of PyPy. @@ -41,13 +44,17 @@ If producing C code is a target, important parts can be directly re-used from CPython. - +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Deliverables** DL??? Runtime Library for PyPy DL??? Passing all applicable elements of the standard test suite. -**Milestones and Expected Result** +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc +**Milestones and Expected Result** +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt (original) +++ pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt Mon Sep 22 11:44:52 2003 @@ -1,29 +1,33 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Several Object Implementations +.. |wp| replace:: WP06 +.. |start| replace:: WP03_ +.. |p1| replace:: X +.. |m1| replace:: 2 +.. |p2| replace:: Y +.. |m2| replace:: 2 +.. |p3| replace:: |e| +.. |m3| replace:: |e| +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 06 --------------------------------- -------------------------- -**Start date or starting event** 0 -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 2 2 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: Several Object Implementations - - Providing several object implementations for - general optimisation - application specific alternatives +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Description of work** @@ -50,11 +54,18 @@ * Create reports and merge the results back into the optimization effort. As of general interest, submit the reports as publication. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Deliverables** - DL??? to be defined. Publication. - DL??? Passing all applicable elements of the standard test suite for every tested alternative. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Milestones and Expected Result** +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt (original) +++ pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt Mon Sep 22 11:44:52 2003 @@ -1,28 +1,33 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Translator Optimisations +.. |wp| replace:: WP07 +.. |start| replace:: WP03_ +.. |p1| replace:: X +.. |m1| replace:: 4 +.. |p2| replace:: Y +.. |m2| replace:: 4 +.. |p3| replace:: |e| +.. |m3| replace:: |e| +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 07 --------------------------------- -------------------------- -**Start date or starting event** 0 -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 4 4 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: Translator Optimisations - - Identification and Implementation of Optimisations through modifications of the Translator. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Description of work** The following alternatives are mostly specific to a particular low-level run-time. @@ -53,14 +58,20 @@ * Create reports and merge the results back into the optimization effort. As of general interest, submit the reports as publication. -Note that compatibility with legacy C extensions can be acheived by choosing, +Note that compatibility with legacy C extensions can be achieved by choosing, for each of the following issues, the same one as CPython did. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Deliverables** - DL??? to be defined. Publication. - DL??? Passing all applicable elements of the standard test suite. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Milestones and Expected Result** +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt (original) +++ pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt Mon Sep 22 11:44:52 2003 @@ -1,32 +1,36 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Integration of Psyco +.. |wp| replace:: WP08 +.. |start| replace:: WP07_ +.. |p1| replace:: AR +.. |m1| replace:: 6 +.. |p2| replace:: Y +.. |m2| replace:: 4 +.. |p3| replace:: Y +.. |m3| replace:: 4 +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 08 --------------------------------- -------------------------- -**Start date or starting event** 0 -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** AR CT X ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 4 2 2 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: Integration of Psyco - - Integration of Psyco's technology into PyPy. - Optimization by Creating Specialized Machine Code at Run-time. - Adressing new Processor Architectures -**Description of work** +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc +**Description of work** - Determine which parts of Psyco need to be written in RPython. @@ -42,11 +46,17 @@ e.g. identify when these technologies would guide specific choices among the solutions developed in WP05_ and studying several solutions in WP06_. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Deliverables** - DL??? to be defined. Speed boost, architectures. - DL??? Passing all applicable elements of the standard test suite. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Milestones and Expected Result** +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt (original) +++ pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt Mon Sep 22 11:44:52 2003 @@ -1,33 +1,37 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Integration of Stackless +.. |wp| replace:: WP09 +.. |start| replace:: WP07_ +.. |p1| replace:: CT +.. |m1| replace:: 6 +.. |p2| replace:: Y +.. |m2| replace:: 4 +.. |p3| replace:: Y +.. |m3| replace:: 4 +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 09 --------------------------------- -------------------------- -**Start date or starting event** 0 -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** AR CT X ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 4 2 2 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: Integration of Stackless - - Integration of Stackless's Technology into PyPy. - Enable Massive Parallelism in a Single Thread - Allow Pickling of a Running Program -**Description of work** +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc +**Description of work** - Determine which parts of Stackless need to be written in RPython. @@ -46,6 +50,8 @@ e.g. identify when these technologies would guide specific choices among the solutions developed in WP05_ and studying several solutions in WP06_. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Deliverables** @@ -54,3 +60,4 @@ **Milestones and Expected Result** +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp10_embedded_device.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp10_embedded_device.txt (original) +++ pypy/trunk/doc/funding/B6.wp10_embedded_device.txt Mon Sep 22 11:44:52 2003 @@ -1,20 +1,24 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: Integration of Stackless +.. |wp| replace:: WP10 +.. |start| replace:: WP05_ +.. |p1| replace:: X +.. |m1| replace:: 3 +.. |p2| replace:: Y +.. |m2| replace:: 3 +.. |p3| replace:: Y +.. |m3| replace:: 3 +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 09 --------------------------------- -------------------------- -**Start date or starting event** not before WP05_ -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y Z ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 2 2 2 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** @@ -25,6 +29,8 @@ embedded Devices - Proving the flexibility of the PyPy idea +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Description of work** @@ -46,11 +52,17 @@ project is vital. Typically, very different trade-offs need to be made on different environments. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Deliverables** - DL??? Python running on the embedded device. - DL??? Passing all applicable elements of the standard test suite. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Milestones and Expected Result** +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp11_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp11_small_os.txt Mon Sep 22 11:44:52 2003 @@ -1,28 +1,31 @@ .. include:: crossreferences.asc -Workpackage Description -======================== +.. |title| replace:: A small Operating System for an Embedded Device +.. |wp| replace:: WP11 +.. |start| replace:: WP10_ +.. |p1| replace:: CT +.. |m1| replace:: 6 +.. |p2| replace:: Y +.. |m2| replace:: 4 +.. |p3| replace:: Y +.. |m3| replace:: 4 +.. |p4| replace:: |e| +.. |m4| replace:: |e| +.. |p5| replace:: |e| +.. |m5| replace:: |e| +.. |p6| replace:: |e| +.. |m6| replace:: |e| -================================ ========================== -**Workpackage number** 11 --------------------------------- -------------------------- -**Start date or starting event** not before WP05_ -================================ ========================== - -================================= ======= ======= ======= ======= ======= ======= ======= ======= -**Participant id** X Y Z ---------------------------------- ------- ------- ------- ------- ------- ------- ------- ------- -**Person-months per participant** 2 2 2 -================================= ======= ======= ======= ======= ======= ======= ======= ======= +.. include:: wp-toptable.asc +.. include:: wp-tablebegin.asc **Objectives** -Title: A small Operating System for an Embedded Device - - - Building all parts of a small Operating System +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Description of work** @@ -53,11 +56,17 @@ - Test the implementation both on the simulator and on the real hardware. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc **Deliverables** - DL??? Python implements the whole OS of the embedded device. - DL??? Passing all applicable elements of the standard test suite. +.. include:: wp-tableend.asc +.. include:: wp-tablebegin.asc + **Milestones and Expected Result** +.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 11:44:52 2003 @@ -9,3 +9,5 @@ .. _WP09: B6.wp09_integrate_stackless.html .. _WP10: B6.wp10_embedded_device.html .. _WP11: B6.wp11_small_os.html + +.. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file Modified: pypy/trunk/doc/funding/renamewp.py ============================================================================== --- pypy/trunk/doc/funding/renamewp.py (original) +++ pypy/trunk/doc/funding/renamewp.py Mon Sep 22 11:44:52 2003 @@ -24,8 +24,8 @@ sys.exit(1) del wpfiles[oldS] wpfiles[newS] = newName - workfiles = wpfiles.values() - workfiles.append(mkxref.REF_FILE) + workfiles = os.listdir(".") + workfiles = [fname for fname in workfiles if os.path.splitext(fname)[-1].lower() == ".txt"] find = oldS+"_" repl = newS+"_" for fname in workfiles: Added: pypy/trunk/doc/funding/wp-tablebegin.asc ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/wp-tablebegin.asc Mon Sep 22 11:44:52 2003 @@ -0,0 +1,11 @@ + +.. start a surounding single table cell with a minimum height. + +.. raw:: html + +



+

+ + + + + +
Added: pypy/trunk/doc/funding/wp-tableend.asc ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/wp-tableend.asc Mon Sep 22 11:44:52 2003 @@ -0,0 +1,8 @@ + +.. end a single surrounding table. + +.. raw:: html + +
Added: pypy/trunk/doc/funding/wp-template.htm ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/wp-template.htm Mon Sep 22 11:44:52 2003 @@ -0,0 +1,112 @@ + + + + + + + + + + + + + +

+|title|

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+

Workpackage number +

+ 		+

|wp|

+ 		+

Start date or starting + event:

+ 		+

|start|

+
+

Participant id

+ 		+

|p1|

+ 		+

|p2|

+ 		+

|p3|

+ 		+

|p4|

+ 		+

|p5|

+ 		+

|p6|

+
+

Person-months per + participant:

+ 		+

|m1|

+ 		+

|m2|

+ 		+

|m3|

+ 		+

|m4|

+ 		+

|m5|

+ 		+

|m6|

+
+



+

+ + + + + +
+


+

+
+



+

+

Note +that this file is intentionally named ending in “.htm”, +because I used to remove all the generated crap by “del +*.html”.

+ + \ No newline at end of file From tismer at codespeak.net Mon Sep 22 11:47:27 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 11:47:27 +0200 (MEST) Subject: [pypy-svn] rev 1385 - pypy/trunk/doc/funding Message-ID: <20030922094727.C13895B356@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 11:47:27 2003 New Revision: 1385 Added: pypy/trunk/doc/funding/wp-toptable.asc Log: sorry, forgot this one Added: pypy/trunk/doc/funding/wp-toptable.asc ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/wp-toptable.asc Mon Sep 22 11:47:27 2003 @@ -0,0 +1,172 @@ + +.. create a nearly exact table for the WPs +.. note that you must define *all* the |xxx| references. + +.. raw:: html + +

+ +

+ +|title| +============== + +.. raw:: html + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+

Workpackage number +

+ 		+ + +|wp| + +.. raw:: html + + + +

Start date or starting + event:

+ 		+ + +|start| + +.. raw:: html + + +
+

Participant id

+ 		+ + +|p1| + +.. raw:: html + + + + + +|p2| + +.. raw:: html + + + + + +|p3| + +.. raw:: html + + + + + +|p4| + +.. raw:: html + + + + + +|p5| + +.. raw:: html + + + + + +|p6| + +.. raw:: html + + +
+

Person-months per + participant:

+ 		+ + +|m1| + +.. raw:: html + + + + + +|m2| + +.. raw:: html + + + + + +|m3| + +.. raw:: html + + + + + +|m4| + +.. raw:: html + + + + + +|m5| + +.. raw:: html + + + + + +|m6| + +.. raw:: html + + +
From tismer at codespeak.net Mon Sep 22 12:03:26 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 12:03:26 +0200 (MEST) Subject: [pypy-svn] rev 1386 - pypy/trunk/doc/funding Message-ID: <20030922100326.D32795B9CA@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 12:03:26 2003 New Revision: 1386 Modified: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Log: layout less bold Modified: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt (original) +++ pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Mon Sep 22 12:03:26 2003 @@ -33,14 +33,14 @@ **Description of work** -**Part a): Code Analysis** +*Part a): Code Analysis* - Analyse code to produce the relevant typing information. - Investigate whether we can use the annotation object space only or if additional AST-based control flow analysis is needed. - Give a formal definition of RPython. -**Part b): Code Production** +*Part b): Code Production* - Compare different low-level environments that we could target (C, Pyrex, others?). Select at least From arigo at codespeak.net Mon Sep 22 15:02:18 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Sep 2003 15:02:18 +0200 (MEST) Subject: [pypy-svn] rev 1387 - pypy/trunk/src/pypy/tool Message-ID: <20030922130218.A72C25B9CA@thoth.codespeak.net> Author: arigo Date: Mon Sep 22 15:02:18 2003 New Revision: 1387 Modified: pypy/trunk/src/pypy/tool/fixeol Log: Fixed the broken end-of-line-normalizer algorithm Modified: pypy/trunk/src/pypy/tool/fixeol ============================================================================== --- pypy/trunk/src/pypy/tool/fixeol (original) +++ pypy/trunk/src/pypy/tool/fixeol Mon Sep 22 15:02:18 2003 @@ -31,27 +31,29 @@ return False original = data data = data.replace('\r\n', '\n') + data = data.replace('\r', '\n') data = data.replace('\n', native_eol) - data = data.replace('\r', native_eol) if data != original: f = open(filename, 'wb') f.write(data) f.close() return True +def asserttextfile(fname): + # safety check to nail binary files + if not binary2text(fname): + print >> sys.stderr, "*** warning, looks like a binary file:", + print >> sys.stderr, fname + return + # change end-of-line style of each .py and .txt file to 'native' + os.system('svn propset svn:eol-style native %s' % fname) + def fixpyfiles(ignored, dirname, fnames): numpyfiles = 0 for fname in fnames: if fname.endswith('.py') or fname.endswith('.txt'): - # safety check to nail binary files - fname = os.path.join(dirname, fname) - if not binary2text(fname): - print >> sys.stderr, "*** warning, looks like a binary file:", - print >> sys.stderr, fname - continue - # change end-of-line style of each .py and .txt file to 'native' - os.system('svn propset svn:eol-style native %s' % fname) + asserttextfile(os.path.join(dirname, fname)) numpyfiles += 1 if numpyfiles: # ignore '*.pyc' in any directory containing .py files @@ -68,4 +70,9 @@ if '.svn' in fnames: fnames.remove('.svn') -os.path.walk(os.curdir, fixpyfiles, None) +if __name__ == '__main__': + if len(sys.argv) > 1: + for fname in sys.argv[1:]: + asserttextfile(fname) + else: + os.path.walk(os.curdir, fixpyfiles, None) From tismer at codespeak.net Mon Sep 22 18:50:10 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 18:50:10 +0200 (MEST) Subject: [pypy-svn] rev 1388 - pypy/trunk/doc/funding Message-ID: <20030922165010.E58775C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 18:50:10 2003 New Revision: 1388 Added: pypy/trunk/doc/funding/B6.4_workpackage_list.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt pypy/trunk/doc/funding/B6.wp10_embedded_device.txt pypy/trunk/doc/funding/crossreferences.asc Log: initial list of all working packages Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 18:50:10 2003 @@ -360,3 +360,5 @@ source. This work is carried out by hand at the moment. We do look for existing solutions which allow to automate this effort at least partially. + +WP12_ WP13_ WP14_ WP15_ WP16_ WP17_ WP18_ WP19_ Added: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 18:50:10 2003 @@ -0,0 +1,46 @@ +.. include:: crossreferences.asc + +Workpackage List +==================== + +=============== ===================================================== ================== ============= =========== ========= ============== +Work-package No Workpackage title Lead contractor No Person-months Start month End month Deliverable No +=============== ===================================================== ================== ============= =========== ========= ============== +WP01_ Development and Completion of the PyPy Interpreter +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP02_ Translation of CPython into regular Python +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP03_ Translation of RPython +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP04_ Specific Analysis and Redesign +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP05_ Low Level PyPy Runtime +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP06_ Several Object Implementations +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP07_ Translator Optimisations +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP08_ Integration of Psyco +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP09_ Integration of Stackless +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP10_ Porting PyPy to an Embedded Device +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP11_ A small Operating System for an Embedded Device +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP12_ Code Generator for SIMD Architecture +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP13_ Enhanced Numerical Package +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP14_ Load Balancing in a Multi-Processor environment +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP15_ Coordination and Management +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP16_ Project Documentation +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP17_ Maintenance of Tools +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP18_ Quality Control +--------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- +WP19_ Synchronisation with Standard Python +=============== ===================================================== ================== ============= =========== ========= ============== Modified: pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt (original) +++ pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Mon Sep 22 18:50:10 2003 @@ -4,9 +4,9 @@ .. |wp| replace:: WP05 .. |start| replace:: WP03_ .. |p1| replace:: X -.. |m1| replace:: 2 +.. |m1| replace:: 3 .. |p2| replace:: Y -.. |m2| replace:: 2 +.. |m2| replace:: 3 .. |p3| replace:: |e| .. |m3| replace:: |e| .. |p4| replace:: |e| Modified: pypy/trunk/doc/funding/B6.wp10_embedded_device.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp10_embedded_device.txt (original) +++ pypy/trunk/doc/funding/B6.wp10_embedded_device.txt Mon Sep 22 18:50:10 2003 @@ -1,6 +1,6 @@ .. include:: crossreferences.asc -.. |title| replace:: Integration of Stackless +.. |title| replace:: Porting PyPy to an Embedded Device .. |wp| replace:: WP10 .. |start| replace:: WP05_ .. |p1| replace:: X @@ -22,8 +22,6 @@ **Objectives** -Title: Porting PyPy to an Embedded Device - - Supporting a completely different platform - Providing an efficient, high-level Language for embedded Devices Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 18:50:10 2003 @@ -9,5 +9,14 @@ .. _WP09: B6.wp09_integrate_stackless.html .. _WP10: B6.wp10_embedded_device.html .. _WP11: B6.wp11_small_os.html +.. _WP12: +.. _WP13: +.. _WP14: +.. _WP15: +.. _WP16: +.. _WP17: +.. _WP18: +.. _WP19: +.. attention:: **8 files missing: WP12 WP13 WP14 WP15 WP16 WP17 WP18 WP19** .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file From tismer at codespeak.net Mon Sep 22 18:55:27 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 18:55:27 +0200 (MEST) Subject: [pypy-svn] rev 1389 - pypy/trunk/doc/funding Message-ID: <20030922165527.363D35C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 18:55:26 2003 New Revision: 1389 Added: pypy/trunk/doc/funding/B6.wp07_several_obj_impl.txt (props changed) - copied unchanged from rev 1387, pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt pypy/trunk/doc/funding/B6.wp08_translator_optimisations.txt (props changed) - copied unchanged from rev 1387, pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt (contents, props changed) - copied, changed from rev 1387, pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt pypy/trunk/doc/funding/B6.wp99_integrate_stackless.txt (contents, props changed) - copied, changed from rev 1387, pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt Removed: pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/B6.wp10_embedded_device.txt pypy/trunk/doc/funding/B6.wp11_small_os.txt pypy/trunk/doc/funding/crossreferences.asc Log: starting the big renamer. Step 1: rotate Stackless from 9 to 6 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 18:55:26 2003 @@ -153,7 +153,7 @@ produces the actual machine code. This layer is highly configurable. -WP06_: Several Object Implementations +WP07_: Several Object Implementations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Since the standard object space allows coexistence of different implementations of the same type, we can develop @@ -166,7 +166,7 @@ Some object layouts will further exist in parallel, if their efficiency is highly application dependant.. -WP07_: Translator Optimisations +WP08_: Translator Optimisations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It should be identified, which **optimisations** would benefit from support from the translator. These are the optimisations @@ -176,7 +176,7 @@ to use reference counting together with garbage collection, or whether to go for garbage collection, only. -The meta-programming underlying WP07_ +The meta-programming underlying WP08_ -- namely the work on the translator instead of on the resulting code -- is what gives us the possibility of actually implementing several very different schemes. The outcome of this effort is of course not unique, @@ -202,7 +202,7 @@ PyPy project, they no longer need to exist as stand-alone projects. -WP08_: Integration of Psyco +WP09_: Integration of Psyco ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Psyco provides techniques to increase the performance of C @@ -213,7 +213,7 @@ RPython code, enabling yet more efficient optimisations and allowing new processor architectures to be targeted. -WP09_: Integration of Stackless +WP99_: Integration of Stackless ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Stackless Python has implemented high speed multitasking in @@ -260,7 +260,7 @@ Ideally, we can create a single threaded PyPy OS with a prioritized scheduler that runs the OS parts together with multiple Python application. The latter depends on -WP09_ very much. +WP99_ very much. Numerical Applications Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 18:55:26 2003 @@ -16,13 +16,13 @@ --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP05_ Low Level PyPy Runtime --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP06_ Several Object Implementations +WP07_ Several Object Implementations --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP07_ Translator Optimisations +WP08_ Translator Optimisations --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP08_ Integration of Psyco +WP09_ Integration of Psyco --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP09_ Integration of Stackless +WP99_ Integration of Stackless --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP10_ Porting PyPy to an Embedded Device --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- Deleted: /pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp06_several_obj_impl.txt Mon Sep 22 18:55:26 2003 +++ (empty file) @@ -1,71 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Several Object Implementations -.. |wp| replace:: WP06 -.. |start| replace:: WP03_ -.. |p1| replace:: X -.. |m1| replace:: 2 -.. |p2| replace:: Y -.. |m2| replace:: 2 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Providing several object implementations for - - general optimisation - - application specific alternatives - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -The following objects are mostly independent from any particular low-level run-time -environment. - -* Implement dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, - or with string-only or integer-only keys. Dictionaries with specific - support for "on-change" callbacks (useful for Psyco). - -* Implement strings as plain immutable memory buffers vs. immutable but more - complex data structures (see functional languages) vs. internally - mutable data structures (e.g. Psyco's concatenated strings) - -* Implement ints as machine words vs. two machine words vs. internal longs vs. - external bignum library (investigate if completely unifying ints and - longs is possible in the Python language at this stage). - -* Consider further objects and do multiple implementations of them. - -* Run various performance tests on the above, check for an - overall improvement or application dependency. - -* Create reports and merge the results back into the optimization effort. - As of general interest, submit the reports as publication. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Publication. -- DL??? Passing all applicable elements of the standard test - suite for every tested alternative. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp07_translator_optimisations.txt Mon Sep 22 18:55:26 2003 +++ (empty file) @@ -1,77 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Translator Optimisations -.. |wp| replace:: WP07 -.. |start| replace:: WP03_ -.. |p1| replace:: X -.. |m1| replace:: 4 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Identification and Implementation of Optimisations - through modifications of the Translator. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -The following alternatives are mostly specific to a particular low-level run-time. - -* Implement and compare different object layout and memory management strategy - or strategies, e.g. reference counting vs. Boehm garbage collection vs. our own. - Includes speed vs. data size trade-offs. - -* Study approaches concerning code size vs. speed trade-offs (e.g. whether the - final interpreter should still include compact precompiled bytecode or be - completely translated into C). - -* Solve the complex issue of threading (global interpreter lock vs. - alternatives). - -* Enhance multimethod dispatching. - -* Implement schemes of pointer tagging, e.g. encoding an integer object as a pointer with a - special value instead of a real pointer to a data structure representing - the integer. - -* Run various performance tests on the above, check for an - overall improvement or application dependency. - -* For each issue, work on **several solutions** when no one is obviously - better than the other ones. - -* Create reports and merge the results back into the optimization effort. - As of general interest, submit the reports as publication. - -Note that compatibility with legacy C extensions can be achieved by choosing, -for each of the following issues, the same one as CPython did. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Publication. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt Mon Sep 22 18:55:26 2003 +++ (empty file) @@ -1,62 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Integration of Psyco -.. |wp| replace:: WP08 -.. |start| replace:: WP07_ -.. |p1| replace:: AR -.. |m1| replace:: 6 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: Y -.. |m3| replace:: 4 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Integration of Psyco's technology into PyPy. - - - Optimization by Creating Specialized Machine Code at Run-time. - - Adressing new Processor Architectures - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Determine which parts of Psyco need to be written in RPython. - -- Design Psyco as a meta-component - that hooks into the translator (WP03_) - -- Implement Psyco partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP05_) for Psyco. - -- Further develop these technologies based on the results gathered in WP07_, - e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP06_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Speed boost, architectures. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Copied: pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt (from rev 1387, pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp08_integrate_psyco.txt (original) +++ pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt Mon Sep 22 18:55:26 2003 @@ -2,7 +2,7 @@ .. |title| replace:: Integration of Psyco .. |wp| replace:: WP08 -.. |start| replace:: WP07_ +.. |start| replace:: WP08_ .. |p1| replace:: AR .. |m1| replace:: 6 .. |p2| replace:: Y @@ -42,9 +42,9 @@ - Implement a dedicated run-time component (WP05_) for Psyco. -- Further develop these technologies based on the results gathered in WP07_, +- Further develop these technologies based on the results gathered in WP08_, e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP06_. + solutions developed in WP05_ and studying several solutions in WP07_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc Deleted: /pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt Mon Sep 22 18:55:26 2003 +++ (empty file) @@ -1,63 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Integration of Stackless -.. |wp| replace:: WP09 -.. |start| replace:: WP07_ -.. |p1| replace:: CT -.. |m1| replace:: 6 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: Y -.. |m3| replace:: 4 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Integration of Stackless's Technology into PyPy. - -- Enable Massive Parallelism in a Single Thread - -- Allow Pickling of a Running Program - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Determine which parts of Stackless need to be written in RPython. - -- Design Stackless as a meta-component - that hooks into the translator (WP03_) - -- Implement Stackless partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP05_) for Stackless. - -- Implement a single-threaded, pre-emptive scheduler with - priorities. This is necessary to support small OSes, like WP11_. - -- Further develop these technologies based on the results gathered in WP07_, - e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP06_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Green threads, pickling. -- DL??? Passing all applicable elements of the standard test suite. - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp10_embedded_device.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp10_embedded_device.txt (original) +++ pypy/trunk/doc/funding/B6.wp10_embedded_device.txt Mon Sep 22 18:55:26 2003 @@ -44,7 +44,7 @@ - Implement interfaces to the needed external libraries. - Combine different solutions for the different issues discussed in - High-performance (WP06_, WP07_, WP08_). + High-performance (WP07_, WP08_, WP09_). Gather statistics with real-work Python applications. Compare the results. This is where the flexibility of the whole project is vital. Typically, very different trade-offs need to be Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp11_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp11_small_os.txt Mon Sep 22 18:55:26 2003 @@ -46,7 +46,7 @@ - Extend the simulator of WP10_ accordingly. - Implement the OS kernel as a set of prioritized tasklets - (using the support from WP09_) + (using the support from WP99_) - Implement a simple file system which is accessible by Python, only. Copied: pypy/trunk/doc/funding/B6.wp99_integrate_stackless.txt (from rev 1387, pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp09_integrate_stackless.txt (original) +++ pypy/trunk/doc/funding/B6.wp99_integrate_stackless.txt Mon Sep 22 18:55:26 2003 @@ -2,7 +2,7 @@ .. |title| replace:: Integration of Stackless .. |wp| replace:: WP09 -.. |start| replace:: WP07_ +.. |start| replace:: WP08_ .. |p1| replace:: CT .. |m1| replace:: 6 .. |p2| replace:: Y @@ -46,9 +46,9 @@ - Implement a single-threaded, pre-emptive scheduler with priorities. This is necessary to support small OSes, like WP11_. -- Further develop these technologies based on the results gathered in WP07_, +- Further develop these technologies based on the results gathered in WP08_, e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP06_. + solutions developed in WP05_ and studying several solutions in WP07_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 18:55:26 2003 @@ -3,10 +3,9 @@ .. _WP03: B6.wp03_translation_of_rpython.html .. _WP04: B6.wp04_bootstrap_redesign.html .. _WP05: B6.wp05_bootstrap_runtime.html -.. _WP06: B6.wp06_several_obj_impl.html -.. _WP07: B6.wp07_translator_optimisations.html -.. _WP08: B6.wp08_integrate_psyco.html -.. _WP09: B6.wp09_integrate_stackless.html +.. _WP07: B6.wp07_several_obj_impl.html +.. _WP08: B6.wp08_translator_optimisations.html +.. _WP09: B6.wp09_integrate_psyco.html .. _WP10: B6.wp10_embedded_device.html .. _WP11: B6.wp11_small_os.html .. _WP12: @@ -17,6 +16,7 @@ .. _WP17: .. _WP18: .. _WP19: +.. _WP99: B6.wp99_integrate_stackless.html .. attention:: **8 files missing: WP12 WP13 WP14 WP15 WP16 WP17 WP18 WP19** .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file From tismer at codespeak.net Mon Sep 22 18:55:53 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 18:55:53 +0200 (MEST) Subject: [pypy-svn] rev 1390 - pypy/trunk/doc/funding Message-ID: <20030922165553.5496C5C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 18:55:52 2003 New Revision: 1390 Added: pypy/trunk/doc/funding/B6.wp06_integrate_stackless.txt (props changed) - copied unchanged from rev 1389, pypy/trunk/doc/funding/B6.wp99_integrate_stackless.txt Removed: pypy/trunk/doc/funding/B6.wp99_integrate_stackless.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/B6.wp11_small_os.txt pypy/trunk/doc/funding/crossreferences.asc Log: end of rotating Stackless 9 to 6 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 18:55:52 2003 @@ -213,7 +213,7 @@ RPython code, enabling yet more efficient optimisations and allowing new processor architectures to be targeted. -WP99_: Integration of Stackless +WP06_: Integration of Stackless ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Stackless Python has implemented high speed multitasking in @@ -260,7 +260,7 @@ Ideally, we can create a single threaded PyPy OS with a prioritized scheduler that runs the OS parts together with multiple Python application. The latter depends on -WP99_ very much. +WP06_ very much. Numerical Applications Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 18:55:52 2003 @@ -22,7 +22,7 @@ --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP09_ Integration of Psyco --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP99_ Integration of Stackless +WP06_ Integration of Stackless --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP10_ Porting PyPy to an Embedded Device --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp11_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp11_small_os.txt Mon Sep 22 18:55:52 2003 @@ -46,7 +46,7 @@ - Extend the simulator of WP10_ accordingly. - Implement the OS kernel as a set of prioritized tasklets - (using the support from WP99_) + (using the support from WP06_) - Implement a simple file system which is accessible by Python, only. Deleted: /pypy/trunk/doc/funding/B6.wp99_integrate_stackless.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp99_integrate_stackless.txt Mon Sep 22 18:55:52 2003 +++ (empty file) @@ -1,63 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Integration of Stackless -.. |wp| replace:: WP09 -.. |start| replace:: WP08_ -.. |p1| replace:: CT -.. |m1| replace:: 6 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: Y -.. |m3| replace:: 4 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Integration of Stackless's Technology into PyPy. - -- Enable Massive Parallelism in a Single Thread - -- Allow Pickling of a Running Program - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Determine which parts of Stackless need to be written in RPython. - -- Design Stackless as a meta-component - that hooks into the translator (WP03_) - -- Implement Stackless partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP05_) for Stackless. - -- Implement a single-threaded, pre-emptive scheduler with - priorities. This is necessary to support small OSes, like WP11_. - -- Further develop these technologies based on the results gathered in WP08_, - e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP07_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Green threads, pickling. -- DL??? Passing all applicable elements of the standard test suite. - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 18:55:52 2003 @@ -3,6 +3,7 @@ .. _WP03: B6.wp03_translation_of_rpython.html .. _WP04: B6.wp04_bootstrap_redesign.html .. _WP05: B6.wp05_bootstrap_runtime.html +.. _WP06: B6.wp06_integrate_stackless.html .. _WP07: B6.wp07_several_obj_impl.html .. _WP08: B6.wp08_translator_optimisations.html .. _WP09: B6.wp09_integrate_psyco.html @@ -16,7 +17,6 @@ .. _WP17: .. _WP18: .. _WP19: -.. _WP99: B6.wp99_integrate_stackless.html .. attention:: **8 files missing: WP12 WP13 WP14 WP15 WP16 WP17 WP18 WP19** .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file From tismer at codespeak.net Mon Sep 22 19:08:59 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 19:08:59 +0200 (MEST) Subject: [pypy-svn] rev 1391 - pypy/trunk/doc/funding Message-ID: <20030922170859.A27CE5C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 19:08:55 2003 New Revision: 1391 Added: pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt (contents, props changed) - copied, changed from rev 1390, pypy/trunk/doc/funding/B6.wp06_integrate_stackless.txt pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt (props changed) - copied unchanged from rev 1390, pypy/trunk/doc/funding/B6.wp07_several_obj_impl.txt pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt (props changed) - copied unchanged from rev 1390, pypy/trunk/doc/funding/B6.wp08_translator_optimisations.txt pypy/trunk/doc/funding/B6.wp51_embedded_device.txt (contents, props changed) - copied, changed from rev 1390, pypy/trunk/doc/funding/B6.wp10_embedded_device.txt Removed: pypy/trunk/doc/funding/B6.wp06_integrate_stackless.txt pypy/trunk/doc/funding/B6.wp07_several_obj_impl.txt pypy/trunk/doc/funding/B6.wp08_translator_optimisations.txt pypy/trunk/doc/funding/B6.wp10_embedded_device.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt pypy/trunk/doc/funding/B6.wp11_small_os.txt pypy/trunk/doc/funding/crossreferences.asc Log: big renaming Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 19:08:55 2003 @@ -153,7 +153,7 @@ produces the actual machine code. This layer is highly configurable. -WP07_: Several Object Implementations +WP42_: Several Object Implementations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Since the standard object space allows coexistence of different implementations of the same type, we can develop @@ -166,7 +166,7 @@ Some object layouts will further exist in parallel, if their efficiency is highly application dependant.. -WP08_: Translator Optimisations +WP43_: Translator Optimisations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It should be identified, which **optimisations** would benefit from support from the translator. These are the optimisations @@ -176,7 +176,7 @@ to use reference counting together with garbage collection, or whether to go for garbage collection, only. -The meta-programming underlying WP08_ +The meta-programming underlying WP43_ -- namely the work on the translator instead of on the resulting code -- is what gives us the possibility of actually implementing several very different schemes. The outcome of this effort is of course not unique, @@ -213,7 +213,7 @@ RPython code, enabling yet more efficient optimisations and allowing new processor architectures to be targeted. -WP06_: Integration of Stackless +WP41_: Integration of Stackless ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Stackless Python has implemented high speed multitasking in @@ -237,7 +237,7 @@ the abstraction and ease-of-use of Python. -WP10_: Porting PyPy to an Embedded Device +WP51_: Porting PyPy to an Embedded Device ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dependant from the actual processor architecture, PyPy's @@ -252,7 +252,7 @@ *we are thinking of a printer here, but don't have facts yet* -In extent to WP10_, it makes sense to write the whole +In extent to WP51_, it makes sense to write the whole operating system in PyPy. We need a low-level extension to the code generator that allows to access I/O ports. Interrupts need to be supported as well, and primitive @@ -260,7 +260,7 @@ Ideally, we can create a single threaded PyPy OS with a prioritized scheduler that runs the OS parts together with multiple Python application. The latter depends on -WP06_ very much. +WP41_ very much. Numerical Applications @@ -361,4 +361,4 @@ do look for existing solutions which allow to automate this effort at least partially. -WP12_ WP13_ WP14_ WP15_ WP16_ WP17_ WP18_ WP19_ +WP53_ WP54_ WP55_ WP15_ WP16_ WP17_ WP18_ WP19_ Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 19:08:55 2003 @@ -16,23 +16,23 @@ --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP05_ Low Level PyPy Runtime --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP07_ Several Object Implementations +WP42_ Several Object Implementations --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP08_ Translator Optimisations +WP43_ Translator Optimisations --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP09_ Integration of Psyco --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP06_ Integration of Stackless +WP41_ Integration of Stackless --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP10_ Porting PyPy to an Embedded Device +WP51_ Porting PyPy to an Embedded Device --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP11_ A small Operating System for an Embedded Device --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP12_ Code Generator for SIMD Architecture +WP53_ Code Generator for SIMD Architecture --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP13_ Enhanced Numerical Package +WP54_ Enhanced Numerical Package --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP14_ Load Balancing in a Multi-Processor environment +WP55_ Load Balancing in a Multi-Processor environment --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP15_ Coordination and Management --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- Deleted: /pypy/trunk/doc/funding/B6.wp06_integrate_stackless.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp06_integrate_stackless.txt Mon Sep 22 19:08:55 2003 +++ (empty file) @@ -1,63 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Integration of Stackless -.. |wp| replace:: WP09 -.. |start| replace:: WP08_ -.. |p1| replace:: CT -.. |m1| replace:: 6 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: Y -.. |m3| replace:: 4 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Integration of Stackless's Technology into PyPy. - -- Enable Massive Parallelism in a Single Thread - -- Allow Pickling of a Running Program - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Determine which parts of Stackless need to be written in RPython. - -- Design Stackless as a meta-component - that hooks into the translator (WP03_) - -- Implement Stackless partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP05_) for Stackless. - -- Implement a single-threaded, pre-emptive scheduler with - priorities. This is necessary to support small OSes, like WP11_. - -- Further develop these technologies based on the results gathered in WP08_, - e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP07_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Green threads, pickling. -- DL??? Passing all applicable elements of the standard test suite. - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp07_several_obj_impl.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp07_several_obj_impl.txt Mon Sep 22 19:08:55 2003 +++ (empty file) @@ -1,71 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Several Object Implementations -.. |wp| replace:: WP06 -.. |start| replace:: WP03_ -.. |p1| replace:: X -.. |m1| replace:: 2 -.. |p2| replace:: Y -.. |m2| replace:: 2 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Providing several object implementations for - - general optimisation - - application specific alternatives - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -The following objects are mostly independent from any particular low-level run-time -environment. - -* Implement dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, - or with string-only or integer-only keys. Dictionaries with specific - support for "on-change" callbacks (useful for Psyco). - -* Implement strings as plain immutable memory buffers vs. immutable but more - complex data structures (see functional languages) vs. internally - mutable data structures (e.g. Psyco's concatenated strings) - -* Implement ints as machine words vs. two machine words vs. internal longs vs. - external bignum library (investigate if completely unifying ints and - longs is possible in the Python language at this stage). - -* Consider further objects and do multiple implementations of them. - -* Run various performance tests on the above, check for an - overall improvement or application dependency. - -* Create reports and merge the results back into the optimization effort. - As of general interest, submit the reports as publication. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Publication. -- DL??? Passing all applicable elements of the standard test - suite for every tested alternative. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp08_translator_optimisations.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp08_translator_optimisations.txt Mon Sep 22 19:08:55 2003 +++ (empty file) @@ -1,77 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Translator Optimisations -.. |wp| replace:: WP07 -.. |start| replace:: WP03_ -.. |p1| replace:: X -.. |m1| replace:: 4 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Identification and Implementation of Optimisations - through modifications of the Translator. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -The following alternatives are mostly specific to a particular low-level run-time. - -* Implement and compare different object layout and memory management strategy - or strategies, e.g. reference counting vs. Boehm garbage collection vs. our own. - Includes speed vs. data size trade-offs. - -* Study approaches concerning code size vs. speed trade-offs (e.g. whether the - final interpreter should still include compact precompiled bytecode or be - completely translated into C). - -* Solve the complex issue of threading (global interpreter lock vs. - alternatives). - -* Enhance multimethod dispatching. - -* Implement schemes of pointer tagging, e.g. encoding an integer object as a pointer with a - special value instead of a real pointer to a data structure representing - the integer. - -* Run various performance tests on the above, check for an - overall improvement or application dependency. - -* For each issue, work on **several solutions** when no one is obviously - better than the other ones. - -* Create reports and merge the results back into the optimization effort. - As of general interest, submit the reports as publication. - -Note that compatibility with legacy C extensions can be achieved by choosing, -for each of the following issues, the same one as CPython did. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Publication. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt (original) +++ pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt Mon Sep 22 19:08:55 2003 @@ -2,7 +2,7 @@ .. |title| replace:: Integration of Psyco .. |wp| replace:: WP08 -.. |start| replace:: WP08_ +.. |start| replace:: WP43_ .. |p1| replace:: AR .. |m1| replace:: 6 .. |p2| replace:: Y @@ -42,9 +42,9 @@ - Implement a dedicated run-time component (WP05_) for Psyco. -- Further develop these technologies based on the results gathered in WP08_, +- Further develop these technologies based on the results gathered in WP43_, e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP07_. + solutions developed in WP05_ and studying several solutions in WP42_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc Deleted: /pypy/trunk/doc/funding/B6.wp10_embedded_device.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp10_embedded_device.txt Mon Sep 22 19:08:55 2003 +++ (empty file) @@ -1,66 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Porting PyPy to an Embedded Device -.. |wp| replace:: WP10 -.. |start| replace:: WP05_ -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Y -.. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Supporting a completely different platform -- Providing an efficient, high-level Language for - embedded Devices -- Proving the flexibility of the PyPy idea - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Analyse the differences between the new architecture and - PyPy's current. Deside whether to optimize for code size - or applicatin speed. - -- Develop the translation process, run-time and those optimisations - that depend on low-level details. - -- Implement a simulator for the embedded device. - -- Implement interfaces to the needed external libraries. - -- Combine different solutions for the different issues discussed in - High-performance (WP07_, WP08_, WP09_). - Gather statistics with real-work Python applications. - Compare the results. This is where the flexibility of the whole - project is vital. Typically, very different trade-offs need to be - made on different environments. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? Python running on the embedded device. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp11_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp11_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp11_small_os.txt Mon Sep 22 19:08:55 2003 @@ -2,7 +2,7 @@ .. |title| replace:: A small Operating System for an Embedded Device .. |wp| replace:: WP11 -.. |start| replace:: WP10_ +.. |start| replace:: WP51_ .. |p1| replace:: CT .. |m1| replace:: 6 .. |p2| replace:: Y @@ -43,10 +43,10 @@ which ensure functional compatibility with the original ones, as far as these exist. -- Extend the simulator of WP10_ accordingly. +- Extend the simulator of WP51_ accordingly. - Implement the OS kernel as a set of prioritized tasklets - (using the support from WP06_) + (using the support from WP41_) - Implement a simple file system which is accessible by Python, only. Copied: pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt (from rev 1390, pypy/trunk/doc/funding/B6.wp06_integrate_stackless.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp06_integrate_stackless.txt (original) +++ pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt Mon Sep 22 19:08:55 2003 @@ -2,7 +2,7 @@ .. |title| replace:: Integration of Stackless .. |wp| replace:: WP09 -.. |start| replace:: WP08_ +.. |start| replace:: WP43_ .. |p1| replace:: CT .. |m1| replace:: 6 .. |p2| replace:: Y @@ -46,9 +46,9 @@ - Implement a single-threaded, pre-emptive scheduler with priorities. This is necessary to support small OSes, like WP11_. -- Further develop these technologies based on the results gathered in WP08_, +- Further develop these technologies based on the results gathered in WP43_, e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP07_. + solutions developed in WP05_ and studying several solutions in WP42_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc Copied: pypy/trunk/doc/funding/B6.wp51_embedded_device.txt (from rev 1390, pypy/trunk/doc/funding/B6.wp10_embedded_device.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp10_embedded_device.txt (original) +++ pypy/trunk/doc/funding/B6.wp51_embedded_device.txt Mon Sep 22 19:08:55 2003 @@ -44,7 +44,7 @@ - Implement interfaces to the needed external libraries. - Combine different solutions for the different issues discussed in - High-performance (WP07_, WP08_, WP09_). + High-performance (WP42_, WP43_, WP09_). Gather statistics with real-work Python applications. Compare the results. This is where the flexibility of the whole project is vital. Typically, very different trade-offs need to be Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 19:08:55 2003 @@ -3,20 +3,20 @@ .. _WP03: B6.wp03_translation_of_rpython.html .. _WP04: B6.wp04_bootstrap_redesign.html .. _WP05: B6.wp05_bootstrap_runtime.html -.. _WP06: B6.wp06_integrate_stackless.html -.. _WP07: B6.wp07_several_obj_impl.html -.. _WP08: B6.wp08_translator_optimisations.html .. _WP09: B6.wp09_integrate_psyco.html -.. _WP10: B6.wp10_embedded_device.html .. _WP11: B6.wp11_small_os.html -.. _WP12: -.. _WP13: -.. _WP14: .. _WP15: .. _WP16: .. _WP17: .. _WP18: .. _WP19: -.. attention:: **8 files missing: WP12 WP13 WP14 WP15 WP16 WP17 WP18 WP19** +.. _WP41: B6.wp41_integrate_stackless.html +.. _WP42: B6.wp42_several_obj_impl.html +.. _WP43: B6.wp43_translator_optimisations.html +.. _WP51: B6.wp51_embedded_device.html +.. _WP53: +.. _WP54: +.. _WP55: +.. attention:: **8 files missing: WP15 WP16 WP17 WP18 WP19 WP53 WP54 WP55** .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file From tismer at codespeak.net Mon Sep 22 19:09:48 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 19:09:48 +0200 (MEST) Subject: [pypy-svn] rev 1392 - pypy/trunk/doc/funding Message-ID: <20030922170948.661775C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 19:09:44 2003 New Revision: 1392 Added: pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt (props changed) - copied unchanged from rev 1390, pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt (contents, props changed) - copied, changed from rev 1391, pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt Removed: pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt pypy/trunk/doc/funding/B6.wp51_embedded_device.txt pypy/trunk/doc/funding/crossreferences.asc Log: big renaming Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 19:09:44 2003 @@ -78,7 +78,7 @@ --------------------------- -WP03_: Translation of RPython +WP31_: Translation of RPython ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ RPython is a restricted version of Python, which is much @@ -87,7 +87,7 @@ else, like C source, Pyrex code, or even passed to an assembly code generator. The Definition of RPython is slightly in flux and should -be adjusted during WP03_. +be adjusted during WP31_. The goal is to be able to translate arbitrary RPython source code (e.g. the one produced in WP01_ into low-level code (C, Pyrex, Java, others). @@ -99,7 +99,7 @@ ------------------------------ The goal is to put interpreter (WP01_, WP02_) and translator -(WP03_) together. +(WP31_) together. The interpreter is written as an RPython program. The translator has to translate this program into some low-level language. @@ -202,7 +202,7 @@ PyPy project, they no longer need to exist as stand-alone projects. -WP09_: Integration of Psyco +WP44_: Integration of Psyco ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Psyco provides techniques to increase the performance of C Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 19:09:44 2003 @@ -10,7 +10,7 @@ --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP02_ Translation of CPython into regular Python --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP03_ Translation of RPython +WP31_ Translation of RPython --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP04_ Specific Analysis and Redesign --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- @@ -20,7 +20,7 @@ --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP43_ Translator Optimisations --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP09_ Integration of Psyco +WP44_ Integration of Psyco --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP41_ Integration of Stackless --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- Deleted: /pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp03_translation_of_rpython.txt Mon Sep 22 19:09:44 2003 +++ (empty file) @@ -1,70 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Translation of RPython -.. |wp| replace:: WP03 -.. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Ability to translate arbitrary RPython source code (e.g. - the one produced in WP01_ into low-level code (C, Pyrex, Java, others). -- Creation of a stand-alone, not-PyPy-related tool for general - optimization of arbitrary but suitably restricted Python application or - parts thereof. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -*Part a): Code Analysis* - -- Analyse code to produce the relevant typing information. -- Investigate whether we can use the annotation object space - only or if additional AST-based control flow analysis is needed. -- Give a formal definition of RPython. - -*Part b): Code Production* - -- Compare different low-level environments that - we could target (C, Pyrex, others?). Select at least - one suitable target. -- Produce low-level code out of the data gathered in Part a). -- Again investigate how this is best done (AST-guided translation or - reverse-engeneering of the low-level control flow gathered by the - annotation object space). -- Provide some measure for the efficiency of the generated - code, in comparison to Standard Python. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL02: Stand-alone Python Interpreter -- DL03: Tool for Analysis of restricted Python Programs -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- Initial Complete Self-Contained PyPy Interpreter -- Initial Measures of Efficiency Modified: pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt (original) +++ pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt Mon Sep 22 19:09:44 2003 @@ -22,7 +22,7 @@ **Objectives** -- Putting interpreter (WP01_, WP02_) and translator (WP03_) together. +- Putting interpreter (WP01_, WP02_) and translator (WP31_) together. - Enhancing the PyPy Design. .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt (original) +++ pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Mon Sep 22 19:09:44 2003 @@ -2,7 +2,7 @@ .. |title| replace:: Low Level PyPy Runtime .. |wp| replace:: WP05 -.. |start| replace:: WP03_ +.. |start| replace:: WP31_ .. |p1| replace:: X .. |m1| replace:: 3 .. |p2| replace:: Y Deleted: /pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt Mon Sep 22 19:09:44 2003 +++ (empty file) @@ -1,62 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Integration of Psyco -.. |wp| replace:: WP08 -.. |start| replace:: WP43_ -.. |p1| replace:: AR -.. |m1| replace:: 6 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: Y -.. |m3| replace:: 4 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Integration of Psyco's technology into PyPy. - - - Optimization by Creating Specialized Machine Code at Run-time. - - Adressing new Processor Architectures - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Determine which parts of Psyco need to be written in RPython. - -- Design Psyco as a meta-component - that hooks into the translator (WP03_) - -- Implement Psyco partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP05_) for Psyco. - -- Further develop these technologies based on the results gathered in WP43_, - e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP42_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Speed boost, architectures. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt (original) +++ pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt Mon Sep 22 19:09:44 2003 @@ -36,7 +36,7 @@ - Determine which parts of Stackless need to be written in RPython. - Design Stackless as a meta-component - that hooks into the translator (WP03_) + that hooks into the translator (WP31_) - Implement Stackless partially in RPython, partially in Python, favoring the latter. Modified: pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt (original) +++ pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt Mon Sep 22 19:09:44 2003 @@ -2,7 +2,7 @@ .. |title| replace:: Several Object Implementations .. |wp| replace:: WP06 -.. |start| replace:: WP03_ +.. |start| replace:: WP31_ .. |p1| replace:: X .. |m1| replace:: 2 .. |p2| replace:: Y Modified: pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt (original) +++ pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt Mon Sep 22 19:09:44 2003 @@ -2,7 +2,7 @@ .. |title| replace:: Translator Optimisations .. |wp| replace:: WP07 -.. |start| replace:: WP03_ +.. |start| replace:: WP31_ .. |p1| replace:: X .. |m1| replace:: 4 .. |p2| replace:: Y Copied: pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt (from rev 1391, pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp09_integrate_psyco.txt (original) +++ pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt Mon Sep 22 19:09:44 2003 @@ -35,7 +35,7 @@ - Determine which parts of Psyco need to be written in RPython. - Design Psyco as a meta-component - that hooks into the translator (WP03_) + that hooks into the translator (WP31_) - Implement Psyco partially in RPython, partially in Python, favoring the latter. Modified: pypy/trunk/doc/funding/B6.wp51_embedded_device.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp51_embedded_device.txt (original) +++ pypy/trunk/doc/funding/B6.wp51_embedded_device.txt Mon Sep 22 19:09:44 2003 @@ -44,7 +44,7 @@ - Implement interfaces to the needed external libraries. - Combine different solutions for the different issues discussed in - High-performance (WP42_, WP43_, WP09_). + High-performance (WP42_, WP43_, WP44_). Gather statistics with real-work Python applications. Compare the results. This is where the flexibility of the whole project is vital. Typically, very different trade-offs need to be Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 19:09:44 2003 @@ -1,18 +1,18 @@ .. _WP01: B6.wp01_interpreter_completion.html .. _WP02: B6.wp02_cpython_to_python.html -.. _WP03: B6.wp03_translation_of_rpython.html .. _WP04: B6.wp04_bootstrap_redesign.html .. _WP05: B6.wp05_bootstrap_runtime.html -.. _WP09: B6.wp09_integrate_psyco.html .. _WP11: B6.wp11_small_os.html .. _WP15: .. _WP16: .. _WP17: .. _WP18: .. _WP19: +.. _WP31: B6.wp31_translation_of_rpython.html .. _WP41: B6.wp41_integrate_stackless.html .. _WP42: B6.wp42_several_obj_impl.html .. _WP43: B6.wp43_translator_optimisations.html +.. _WP44: B6.wp44_integrate_psyco.html .. _WP51: B6.wp51_embedded_device.html .. _WP53: .. _WP54: From tismer at codespeak.net Mon Sep 22 19:10:42 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 19:10:42 +0200 (MEST) Subject: [pypy-svn] rev 1393 - pypy/trunk/doc/funding Message-ID: <20030922171042.4E0645C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 19:10:37 2003 New Revision: 1393 Added: pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt (props changed) - copied unchanged from rev 1390, pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt (contents, props changed) - copied, changed from rev 1392, pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt pypy/trunk/doc/funding/B6.wp33_bootstrap_runtime.txt (props changed) - copied unchanged from rev 1392, pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Removed: pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt pypy/trunk/doc/funding/B6.wp51_embedded_device.txt pypy/trunk/doc/funding/crossreferences.asc Log: big renaming Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 19:10:37 2003 @@ -53,7 +53,7 @@ The goal is to make a complete Python interpreter that runs under any existing Python implementation. -WP01_: Development and Completion of the PyPy Interpreter +WP21_: Development and Completion of the PyPy Interpreter ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The PyPy interpreter itself should be developed and completed as a regular Python/RPython program. This package includes all @@ -66,7 +66,7 @@ WP02_: Translation of CPython into regular Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Translate all other parts of CPython which we don't want to -implement in WP01_ into regular Python libraries. +implement in WP21_ into regular Python libraries. These ones should also work without PyPy, being just plain-Python replacements for existing CPython functionality. This includes the bytecode compiler, which definitely should @@ -90,7 +90,7 @@ be adjusted during WP31_. The goal is to be able to translate arbitrary RPython source code (e.g. -the one produced in WP01_ into low-level code (C, Pyrex, Java, others). +the one produced in WP21_ into low-level code (C, Pyrex, Java, others). This includes making a stand-alone, not-PyPy-related tool for general optimisation of arbitrary but suitably restricted Python application or parts thereof. @@ -98,19 +98,19 @@ _`Bootstrapping` PyPy ------------------------------ -The goal is to put interpreter (WP01_, WP02_) and translator +The goal is to put interpreter (WP21_, WP02_) and translator (WP31_) together. The interpreter is written as an RPython program. The translator has to translate this program into some low-level language. The resulting program then needs to be supported by a special -runtime library. The work-flow of WP04_ is repetitive, +runtime library. The work-flow of WP32_ is repetitive, since it will not be possible to "get it right" in the first attempt. Analysis and redesign will have to be repeated until the result is satisfactory. -WP04_: Specific Analysis and Redesign +WP32_: Specific Analysis and Redesign ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The global translation of PyPy is going to raise particular @@ -125,7 +125,7 @@ until we reach overall convergence. -WP05_: Low Level PyPy Runtime +WP33_: Low Level PyPy Runtime ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In order to give a working environment to the translated RPython @@ -196,7 +196,7 @@ following CPython's code base. Both will be rewritten as a meta-component that hooks into the -translator plus a dedicated run-time component (WP05_). +translator plus a dedicated run-time component (WP33_). As a side effect, after successful completion of the PyPy project, they no longer need to exist as stand-alone Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 19:10:37 2003 @@ -6,15 +6,15 @@ =============== ===================================================== ================== ============= =========== ========= ============== Work-package No Workpackage title Lead contractor No Person-months Start month End month Deliverable No =============== ===================================================== ================== ============= =========== ========= ============== -WP01_ Development and Completion of the PyPy Interpreter +WP21_ Development and Completion of the PyPy Interpreter --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP02_ Translation of CPython into regular Python --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP31_ Translation of RPython --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP04_ Specific Analysis and Redesign +WP32_ Specific Analysis and Redesign --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP05_ Low Level PyPy Runtime +WP33_ Low Level PyPy Runtime --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- WP42_ Several Object Implementations --------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- Deleted: /pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp01_interpreter_completion.txt Mon Sep 22 19:10:37 2003 +++ (empty file) @@ -1,55 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Development and Completion of the PyPy Interpreter -.. |wp| replace:: WP01 -.. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Z -.. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -The PyPy interpreter itself should be developed and completed -as a regular Python program. This package includes all -parts of CPython that we don't want to move to WP02_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Coordinate with WP02_ which parts of CPython - should be implemented in RPython. -- Implementation of all C modules and C objects -- Further investigation of the unorthodox multimethod - concepts that the standard object space is based on, - and how to hook in the bytecode compiler. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL01: First complete Python Implementation - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- Initial Prototype - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt (original) +++ pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt Mon Sep 22 19:10:37 2003 @@ -23,7 +23,7 @@ **Objectives** Translate all parts of CPython which we don't want to -implement in WP01_ into regular Python libraries. +implement in WP21_ into regular Python libraries. Note: These ones should also work without PyPy, being just plain-Python replacements for existing CPython functionality. @@ -32,7 +32,7 @@ **Description of work** -- Coordinate with WP01_ which parts of CPython should be implemented in Python +- Coordinate with WP21_ which parts of CPython should be implemented in Python - Implementation of all C modules and C objects - Implement the Python bytecode compiler, which definately should become a regular Python program, instead of being built Deleted: /pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt Mon Sep 22 19:10:37 2003 +++ (empty file) @@ -1,55 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Specific Analysis and Redesign -.. |wp| replace:: WP04 -.. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Putting interpreter (WP01_, WP02_) and translator (WP31_) together. -- Enhancing the PyPy Design. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Part a): Specific Analysis and Redesign** - -- Investigate the particular problems that the - global translation of PyPy is going to raise. -- Enhance and redesign parts of PyPy as needed - to give best support to the translation of RPython. -- Re-iterate this until we agree to have reached - an optimum solution. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp05_bootstrap_runtime.txt Mon Sep 22 19:10:37 2003 +++ (empty file) @@ -1,60 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Low Level PyPy Runtime -.. |wp| replace:: WP05 -.. |start| replace:: WP31_ -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Providing a Runtime Library for PyPy - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -In order to give a working environment to the translated RPython -program, we need to build the low-level-specific **run-time** -components of PyPy. - -- Design the object layout and implement it. -- Decide about memory management and implement it. -- Consider threading support and implement it. Make sure - that the other components are thread-aware. -- Design and implement an efficient multimethod dispatch. - -The target language is not yet decided on. We might go different -paths at the same time. -If producing C code is a target, important -parts can be directly re-used from CPython. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -DL??? Runtime Library for PyPy -DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt (original) +++ pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt Mon Sep 22 19:10:37 2003 @@ -23,7 +23,7 @@ **Objectives** - Ability to translate arbitrary RPython source code (e.g. - the one produced in WP01_ into low-level code (C, Pyrex, Java, others). + the one produced in WP21_ into low-level code (C, Pyrex, Java, others). - Creation of a stand-alone, not-PyPy-related tool for general optimization of arbitrary but suitably restricted Python application or parts thereof. Copied: pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt (from rev 1392, pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp04_bootstrap_redesign.txt (original) +++ pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt Mon Sep 22 19:10:37 2003 @@ -22,7 +22,7 @@ **Objectives** -- Putting interpreter (WP01_, WP02_) and translator (WP31_) together. +- Putting interpreter (WP21_, WP02_) and translator (WP31_) together. - Enhancing the PyPy Design. .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt (original) +++ pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt Mon Sep 22 19:10:37 2003 @@ -41,14 +41,14 @@ - Implement Stackless partially in RPython, partially in Python, favoring the latter. -- Implement a dedicated run-time component (WP05_) for Stackless. +- Implement a dedicated run-time component (WP33_) for Stackless. - Implement a single-threaded, pre-emptive scheduler with priorities. This is necessary to support small OSes, like WP11_. - Further develop these technologies based on the results gathered in WP43_, e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP42_. + solutions developed in WP33_ and studying several solutions in WP42_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc Modified: pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt (original) +++ pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt Mon Sep 22 19:10:37 2003 @@ -40,11 +40,11 @@ - Implement Psyco partially in RPython, partially in Python, favoring the latter. -- Implement a dedicated run-time component (WP05_) for Psyco. +- Implement a dedicated run-time component (WP33_) for Psyco. - Further develop these technologies based on the results gathered in WP43_, e.g. identify when these technologies would guide specific choices among the - solutions developed in WP05_ and studying several solutions in WP42_. + solutions developed in WP33_ and studying several solutions in WP42_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc Modified: pypy/trunk/doc/funding/B6.wp51_embedded_device.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp51_embedded_device.txt (original) +++ pypy/trunk/doc/funding/B6.wp51_embedded_device.txt Mon Sep 22 19:10:37 2003 @@ -2,7 +2,7 @@ .. |title| replace:: Porting PyPy to an Embedded Device .. |wp| replace:: WP10 -.. |start| replace:: WP05_ +.. |start| replace:: WP33_ .. |p1| replace:: X .. |m1| replace:: 3 .. |p2| replace:: Y Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 19:10:37 2003 @@ -1,14 +1,14 @@ -.. _WP01: B6.wp01_interpreter_completion.html .. _WP02: B6.wp02_cpython_to_python.html -.. _WP04: B6.wp04_bootstrap_redesign.html -.. _WP05: B6.wp05_bootstrap_runtime.html .. _WP11: B6.wp11_small_os.html .. _WP15: .. _WP16: .. _WP17: .. _WP18: .. _WP19: +.. _WP21: B6.wp21_interpreter_completion.html .. _WP31: B6.wp31_translation_of_rpython.html +.. _WP32: B6.wp32_bootstrap_redesign.html +.. _WP33: B6.wp33_bootstrap_runtime.html .. _WP41: B6.wp41_integrate_stackless.html .. _WP42: B6.wp42_several_obj_impl.html .. _WP43: B6.wp43_translator_optimisations.html From arigo at codespeak.net Mon Sep 22 19:59:42 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Sep 2003 19:59:42 +0200 (MEST) Subject: [pypy-svn] rev 1394 - in pypy/trunk/src/pypy: interpreter interpreter/testobjspace/std/attic tool Message-ID: <20030922175942.72ADB5C2E3@thoth.codespeak.net> Author: arigo Date: Mon Sep 22 19:59:41 2003 New Revision: 1394 Modified: pypy/trunk/src/pypy/interpreter/debug.py (props changed) pypy/trunk/src/pypy/interpreter/eval.py (props changed) pypy/trunk/src/pypy/interpreter/extmodule.py (props changed) pypy/trunk/src/pypy/interpreter/function.py (props changed) pypy/trunk/src/pypy/interpreter/gateway.py (props changed) pypy/trunk/src/pypy/interpreter/generator.py (props changed) pypy/trunk/src/pypy/interpreter/module.py (props changed) pypy/trunk/src/pypy/interpreter/nestedscope.py (props changed) pypy/trunk/src/pypy/interpreter/pycode.py (props changed) pypy/trunk/src/pypy/interpreter/test/test_eval.py (props changed) pypy/trunk/src/pypy/interpreter/test/test_extmodule.py (props changed) pypy/trunk/src/pypy/interpreter/test/test_function.py (props changed) pypy/trunk/src/pypy/interpreter/test/test_module.py (props changed) pypy/trunk/src/pypy/objspace/std/attic/ (props changed) pypy/trunk/src/pypy/tool/ppdb.py (props changed) pypy/trunk/src/pypy/tool/testpm.py (props changed) Log: fixeol From arigo at codespeak.net Mon Sep 22 20:01:19 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Sep 2003 20:01:19 +0200 (MEST) Subject: [pypy-svn] rev 1395 - in pypy/trunk/src/pypy/interpreter: . test Message-ID: <20030922180119.B79665C2E3@thoth.codespeak.net> Author: arigo Date: Mon Sep 22 20:01:19 2003 New Revision: 1395 Added: pypy/trunk/src/pypy/interpreter/test/test_gateway.py (contents, props changed) Modified: pypy/trunk/src/pypy/interpreter/extmodule.py pypy/trunk/src/pypy/interpreter/gateway.py pypy/trunk/src/pypy/interpreter/test/test_function.py pypy/trunk/src/pypy/interpreter/test/test_objspace.py Log: More tests for the interpreter, plus a fix to built-in functions: now a built-in function must have a signature like 'f(self, w_x, w_y, *args_w)' and will show at app-level with signature 'f(x, y, *args)'. Modified: pypy/trunk/src/pypy/interpreter/extmodule.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/extmodule.py (original) +++ pypy/trunk/src/pypy/interpreter/extmodule.py Mon Sep 22 20:01:19 2003 @@ -30,7 +30,7 @@ continue # ignore CPython functions # ignore tricky class-attrs we can't send from interp to app-level - if name in ('__metaclass__', '__init__', '__new__', ): + if name in ('__metaclass__',): continue space.call_method(self.w_dict, 'setdefault', space.wrap(name), space.wrap(value)) Modified: pypy/trunk/src/pypy/interpreter/gateway.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/gateway.py (original) +++ pypy/trunk/src/pypy/interpreter/gateway.py Mon Sep 22 20:01:19 2003 @@ -31,8 +31,40 @@ # extract the signature from the (CPython-level) code object tmp = pycode.PyCode(None) tmp._from_code(func.func_code) - self.sig = tmp.signature() - self.ismethod = False + # signature-based hacks: renaming arguments from w_xyz to xyz. + # Currently we enforce the following signature tricks: + # * the first arg must be either 'self' or 'space' + # * 'w_' prefixes for the rest + # * '_w' suffixes on * and ** + # Not exactly a clean approach XXX. + argnames, varargname, kwargname = tmp.signature() + argnames = list(argnames) + if argnames[0] == 'self': + self.ismethod = True + elif argnames[0] == 'space': + self.ismethod = False + del argnames[0] + else: + raise AssertionError, ( + "first argument must be called 'self' in methods " + "and 'space' in global functions") + for i in range(self.ismethod, len(argnames)): + a = argnames[i] + assert a.startswith('w_'), ( + "argument %s of built-in function %r should start with 'w_'" % + (a, func)) + argnames[i] = a[2:] + if varargname is not None: + assert varargname.endswith('_w'), ( + "argument *%s of built-in function %r should end in '_w'" % + (varargname, func)) + varargname = varargname[:-2] + if kwargname is not None: + assert kwargname.endswith('_w'), ( + "argument **%s of built-in function %r should end in '_w'" % + (kwargname, func)) + kwargname = kwargname[:-2] + self.sig = argnames, varargname, kwargname def create_frame(self, space, w_globals, closure=None): return BuiltinFrame(space, self, w_globals) @@ -113,8 +145,11 @@ return self # Gateways as unbound methods not implemented else: # the object space is implicitely fetched out of the instance + if isinstance(self.code, BuiltinCode): + assert self.code.ismethod, ( + 'global built-in function %r used as method' % + self.code.func) fn = self.get_function(obj.space) - self.code.ismethod = True # that's a hack all right return fn.__get__(obj, cls) def get_function(self, space): @@ -193,11 +228,18 @@ def exportall(d): """Publish every function from a dict.""" for name, obj in d.items(): - # ignore names in '_xyz' - if not name.startswith('_') or name.endswith('_'): - if isinstance(obj, types.FunctionType): - if 'app_'+name not in d: - d['app_'+name] = interp2app(obj, name) + if isinstance(obj, types.FunctionType): + # names starting in 'app_' are supposedly already app-level + if name.startswith('app_'): + continue + # ignore tricky functions with another interp-level meaning + if name in ('__init__', '__new__'): + continue + # ignore names in '_xyz' + if name.startswith('_') and not name.endswith('_'): + continue + if 'app_'+name not in d: + d['app_'+name] = interp2app(obj, name) def importall(d): """Import all app_-level functions as Gateways into a dict.""" Modified: pypy/trunk/src/pypy/interpreter/test/test_function.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/test/test_function.py (original) +++ pypy/trunk/src/pypy/interpreter/test/test_function.py Mon Sep 22 20:01:19 2003 @@ -2,8 +2,11 @@ import autopath from pypy.tool import test import unittest +from pypy.interpreter.function import Function, Method +from pypy.interpreter.pycode import PyCode -class ArgParseTest(test.AppTestCase): + +class AppTestFunction(test.AppTestCase): def test_simple_call(self): def func(arg1, arg2): return arg1, arg2 @@ -80,27 +83,34 @@ return arg1, kw self.assertRaises(TypeError, func, 42, **{'arg1': 23}) -class ModuleMinimalTest(test.IntTestCase): + +class TestMethod(test.IntTestCase): def setUp(self): self.space = test.objspace() + def c(self, bar): + return bar + code = PyCode()._from_code(c.func_code) + self.fn = Function(self.space, code) + + def test_get(self): + class X(object): + fn = self.fn + x = X() + meth = x.fn + self.failUnless(isinstance(meth, Method)) + + def test_call(self): + class X(object): + fn = self.fn + x = X() + self.assertEquals(x.fn(42), 42) + + def test_fail_call(self): + class X(object): + fn = self.fn + x = X() + self.assertRaises_w(self.space.w_TypeError, x.fn, "spam", "egg") - def test_sys_exists(self): - w_sys = self.space.get_builtin_module('sys') - self.assert_(self.space.is_true(w_sys)) - - def test_import_exists(self): - space = self.space - w_builtin = space.get_builtin_module('__builtin__') - self.assert_(space.is_true(w_builtin)) - w_name = space.wrap('__import__') - w_import = self.space.getattr(w_builtin, w_name) - self.assert_(space.is_true(w_import)) - - def test_sys_import(self): - from pypy.interpreter.main import run_string - run_string('import sys', space=self.space) if __name__ == '__main__': test.main() - - Added: pypy/trunk/src/pypy/interpreter/test/test_gateway.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/interpreter/test/test_gateway.py Mon Sep 22 20:01:19 2003 @@ -0,0 +1,84 @@ + +import autopath +from pypy.tool import test +from pypy.interpreter import gateway + + +class TestBuiltinCode(test.IntTestCase): + def setUp(self): + self.space = test.objspace() + + def test_signature(self): + def c(space, w_x, w_y, *args_w, **kw_w): + pass + code = gateway.BuiltinCode(c) + self.assertEqual(code.signature(), (['x', 'y'], 'args', 'kw')) + def d(self, w_boo): + pass + code = gateway.BuiltinCode(d) + self.assertEqual(code.signature(), (['self', 'boo'], None, None)) + + def test_call(self): + def c(space, w_x, w_y, *args_w, **kw_w): + u = space.unwrap + w = space.wrap + return w((u(w_x) - u(w_y) + len(args_w)) * u(kw_w['boo'])) + code = gateway.BuiltinCode(c) + w = self.space.wrap + w_dict = self.space.newdict([ + (w('x'), w(123)), + (w('y'), w(23)), + (w('args'), self.space.newtuple([w(0), w(True)])), + (w('kw'), self.space.newdict([(w('boo'), w(10))])), + ]) + w_result = code.exec_code(self.space, w_dict, w_dict) + self.assertEqual_w(w_result, w(1020)) + + +class TestGateway(test.IntTestCase): + def setUp(self): + self.space = test.objspace() + + def test_app2interp(self): + w = self.space.wrap + def app_g3(a, b): + return a+b + g3 = gateway.app2interp(app_g3) + self.assertEqual_w(g3(self.space, w('foo'), w('bar')), w('foobar')) + + def test_interp2app(self): + w = self.space.wrap + def g3(space, w_a, w_b): + return space.add(w_a, w_b) + app_g3 = gateway.interp2app(g3) + self.assertEqual_w(app_g3(self.space, w('foo'), w('bar')), w('foobar')) + + def test_importall(self): + w = self.space.wrap + g = {} + exec """ +def app_g3(a, b): + return a+b +def app_g1(x): + return g3('foo', x) +""" in g + gateway.importall(g) + g1 = g['g1'] + self.assertEqual_w(g1(self.space, w('bar')), w('foobar')) + + def test_exportall(self): + w = self.space.wrap + g = {} + exec """ +def g3(space, w_a, w_b): + return space.add(w_a, w_b) +def app_g1(x): + return g3('foo', x) +""" in g + gateway.exportall(g) + g1 = gateway.app2interp(g['app_g1']) + self.assertEqual_w(g1(self.space, w('bar')), w('foobar')) + + +if __name__ == '__main__': + test.main() Modified: pypy/trunk/src/pypy/interpreter/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/interpreter/test/test_objspace.py Mon Sep 22 20:01:19 2003 @@ -94,5 +94,25 @@ self.failIf(self.space.exception_match(self.space.w_ValueError, self.space.w_LookupError)) +class ModuleMinimalTest(test.IntTestCase): + def setUp(self): + self.space = test.objspace() + + def test_sys_exists(self): + w_sys = self.space.get_builtin_module('sys') + self.assert_(self.space.is_true(w_sys)) + + def test_import_exists(self): + space = self.space + w_builtin = space.get_builtin_module('__builtin__') + self.assert_(space.is_true(w_builtin)) + w_name = space.wrap('__import__') + w_import = self.space.getattr(w_builtin, w_name) + self.assert_(space.is_true(w_import)) + + def test_sys_import(self): + from pypy.interpreter.main import run_string + run_string('import sys', space=self.space) + if __name__ == '__main__': test.main() From arigo at codespeak.net Mon Sep 22 20:16:51 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Sep 2003 20:16:51 +0200 (MEST) Subject: [pypy-svn] rev 1396 - in pypy/trunk/src/pypy: interpreter/test module Message-ID: <20030922181651.9622B5C2E3@thoth.codespeak.net> Author: arigo Date: Mon Sep 22 20:16:50 2003 New Revision: 1396 Added: pypy/trunk/src/pypy/interpreter/test/test_generator.py (contents, props changed) - copied, changed from rev 1394, pypy/trunk/src/pypy/interpreter/test/test_interpreter.py pypy/trunk/src/pypy/interpreter/test/test_nestedscope.py (contents, props changed) - copied, changed from rev 1394, pypy/trunk/src/pypy/interpreter/test/test_interpreter.py pypy/trunk/src/pypy/interpreter/test/test_pyframe.py (contents, props changed) Modified: pypy/trunk/src/pypy/interpreter/test/test_interpreter.py pypy/trunk/src/pypy/module/sysmodule.py Log: A few more tests, and test refactoring. Preliminary sys._getframe() support. Copied: pypy/trunk/src/pypy/interpreter/test/test_generator.py (from rev 1394, pypy/trunk/src/pypy/interpreter/test/test_interpreter.py) ============================================================================== --- pypy/trunk/src/pypy/interpreter/test/test_interpreter.py (original) +++ pypy/trunk/src/pypy/interpreter/test/test_generator.py Mon Sep 22 20:16:50 2003 @@ -2,247 +2,7 @@ import autopath from pypy.tool import test -class TestInterpreter(test.TestCase): - - def codetest(self, source, functionname, args): - """Compile and run the given code string, and then call its function - named by 'functionname' with arguments 'args'.""" - from pypy.interpreter import baseobjspace, executioncontext - from pypy.interpreter import pyframe, gateway, module - space = self.space - - compile = space.builtin.compile - w = space.wrap - w_code = compile(w(source), w(''), w('exec'), w(0), w(0)) - - ec = executioncontext.ExecutionContext(space) - - tempmodule = module.Module(space, w("__temp__")) - w_glob = tempmodule.w_dict - space.setitem(w_glob, w("__builtins__"), space.w_builtins) - - code = space.unwrap(w_code) - code.exec_code(space, w_glob, w_glob) - - wrappedargs = w(args) - wrappedfunc = space.getitem(w_glob, w(functionname)) - wrappedkwds = space.newdict([]) - try: - w_output = space.call(wrappedfunc, wrappedargs, wrappedkwds) - except baseobjspace.OperationError, e: - #e.print_detailed_traceback(space) - return '<<<%s>>>' % e.errorstr(space) - else: - return space.unwrap(w_output) - - def setUp(self): - self.space = test.objspace() - - def test_exception_trivial(self): - x = self.codetest(''' -def f(): - try: - raise Exception() - except Exception, e: - return 1 - return 2 -''', 'f', []) - self.assertEquals(x, 1) - - def test_exception(self): - x = self.codetest(''' -def f(): - try: - raise Exception, 1 - except Exception, e: - return e.args[0] -''', 'f', []) - self.assertEquals(x, 1) - - def test_finally(self): - code = ''' -def f(a): - try: - if a: - raise Exception - a = -12 - finally: - return a -''' - self.assertEquals(self.codetest(code, 'f', [0]), -12) - self.assertEquals(self.codetest(code, 'f', [1]), 1) - -## def test_raise(self): -## x = self.codetest(''' -## def f(): -## raise 1 -## ''', 'f', []) -## self.assertEquals(x, '<<>>') - - def test_except2(self): - x = self.codetest(''' -def f(): - try: - z = 0 - try: - "x"+1 - except TypeError, e: - z = 5 - raise e - except TypeError: - return z -''', 'f', []) - self.assertEquals(x, 5) - - def test_except3(self): - code = ''' -def f(v): - z = 0 - try: - z = 1//v - except ZeroDivisionError, e: - z = "infinite result" - return z -''' - self.assertEquals(self.codetest(code, 'f', [2]), 0) - self.assertEquals(self.codetest(code, 'f', [0]), "infinite result") - ess = "TypeError: unsupported operand type" - res = self.codetest(code, 'f', ['x']) - self.failUnless(res.find(ess) >= 0) - # the following (original) test was a bit too strict...: - # self.assertEquals(self.codetest(code, 'f', ['x']), "<<>>") - - def test_break(self): - code = ''' -def f(n): - total = 0 - for i in range(n): - try: - if i == 4: - break - finally: - total += i - return total -''' - self.assertEquals(self.codetest(code, 'f', [4]), 1+2+3) - self.assertEquals(self.codetest(code, 'f', [9]), 1+2+3+4) - - def test_continue(self): - code = ''' -def f(n): - total = 0 - for i in range(n): - try: - if i == 4: - continue - finally: - total += 100 - total += i - return total -''' - self.assertEquals(self.codetest(code, 'f', [4]), 1+2+3+400) - self.assertEquals(self.codetest(code, 'f', [9]), - 1+2+3 + 5+6+7+8+900) - -class AppTestInterpreter(test.AppTestCase): - def test_exception(self): - try: - raise Exception, 1 - except Exception, e: - self.assertEquals(e.args[0], 1) - - def test_trivial(self): - x = 42 - self.assertEquals(x, 42) - - def test_raise(self): - def f(): - raise Exception - self.assertRaises(Exception, f) - - def test_exception(self): - try: - raise Exception - self.fail("exception failed to raise") - except: - pass - else: - self.fail("exception executing else clause!") - - def test_raise2(self): - def f(r): - try: - raise r - except LookupError: - return 1 - self.assertRaises(Exception, f, Exception) - self.assertEquals(f(IndexError), 1) - - def test_raise3(self): - try: - raise 1 - except TypeError: - pass - else: - self.fail("shouldn't be able to raise 1") - - def test_trivial_call(self): - def f(): return 42 - self.assertEquals(f(), 42) - - def test_trivial_call2(self): - def f(): return 1 + 1 - self.assertEquals(f(), 2) - - def test_print(self): - import sys - save = sys.stdout - class Out: - def __init__(self): - self.args = [] - def write(self, *args): - self.args.extend(args) - out = Out() - try: - sys.stdout = out - print 10 - self.assertEquals(out.args, ['10','\n']) - finally: - sys.stdout = save - - def test_identity(self): - def f(x): return x - self.assertEquals(f(666), 666) - - def test_nested_scope(self): - x = 42 - def f(): return x - self.assertEquals(f(), 42) - - def test_nested_scope2(self): - x = 42 - y = 3 - def f(): return x - self.assertEquals(f(), 42) - - def test_nested_scope3(self): - x = 42 - def f(): - def g(): - return x - return g - self.assertEquals(f()(), 42) - - def test_nested_scope4(self): - def f(): - x = 3 - def g(): - return x - a = g() - x = 4 - b = g() - return (a, b) - self.assertEquals(f(), (3, 4)) +class AppTestGenerator(test.AppTestCase): def test_generator(self): def f(): Modified: pypy/trunk/src/pypy/interpreter/test/test_interpreter.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/test/test_interpreter.py (original) +++ pypy/trunk/src/pypy/interpreter/test/test_interpreter.py Mon Sep 22 20:16:50 2003 @@ -1,4 +1,3 @@ -from __future__ import generators import autopath from pypy.tool import test @@ -214,67 +213,6 @@ def f(x): return x self.assertEquals(f(666), 666) - def test_nested_scope(self): - x = 42 - def f(): return x - self.assertEquals(f(), 42) - - def test_nested_scope2(self): - x = 42 - y = 3 - def f(): return x - self.assertEquals(f(), 42) - - def test_nested_scope3(self): - x = 42 - def f(): - def g(): - return x - return g - self.assertEquals(f()(), 42) - - def test_nested_scope4(self): - def f(): - x = 3 - def g(): - return x - a = g() - x = 4 - b = g() - return (a, b) - self.assertEquals(f(), (3, 4)) - - def test_generator(self): - def f(): - yield 1 - self.assertEquals(f().next(), 1) - - def test_generator2(self): - def f(): - yield 1 - g = f() - self.assertEquals(g.next(), 1) - self.assertRaises(StopIteration, g.next) - - def test_generator3(self): - def f(): - yield 1 - g = f() - self.assertEquals(list(g), [1]) - - def test_generator4(self): - def f(): - yield 1 - g = f() - self.assertEquals([x for x in g], [1]) - - def test_generator_restart(self): - def g(): - i = me.next() - yield i - me = g() - self.assertRaises(ValueError, me.next) - if __name__ == '__main__': test.main() Copied: pypy/trunk/src/pypy/interpreter/test/test_nestedscope.py (from rev 1394, pypy/trunk/src/pypy/interpreter/test/test_interpreter.py) ============================================================================== --- pypy/trunk/src/pypy/interpreter/test/test_interpreter.py (original) +++ pypy/trunk/src/pypy/interpreter/test/test_nestedscope.py Mon Sep 22 20:16:50 2003 @@ -1,218 +1,8 @@ -from __future__ import generators import autopath from pypy.tool import test -class TestInterpreter(test.TestCase): - def codetest(self, source, functionname, args): - """Compile and run the given code string, and then call its function - named by 'functionname' with arguments 'args'.""" - from pypy.interpreter import baseobjspace, executioncontext - from pypy.interpreter import pyframe, gateway, module - space = self.space - - compile = space.builtin.compile - w = space.wrap - w_code = compile(w(source), w(''), w('exec'), w(0), w(0)) - - ec = executioncontext.ExecutionContext(space) - - tempmodule = module.Module(space, w("__temp__")) - w_glob = tempmodule.w_dict - space.setitem(w_glob, w("__builtins__"), space.w_builtins) - - code = space.unwrap(w_code) - code.exec_code(space, w_glob, w_glob) - - wrappedargs = w(args) - wrappedfunc = space.getitem(w_glob, w(functionname)) - wrappedkwds = space.newdict([]) - try: - w_output = space.call(wrappedfunc, wrappedargs, wrappedkwds) - except baseobjspace.OperationError, e: - #e.print_detailed_traceback(space) - return '<<<%s>>>' % e.errorstr(space) - else: - return space.unwrap(w_output) - - def setUp(self): - self.space = test.objspace() - - def test_exception_trivial(self): - x = self.codetest(''' -def f(): - try: - raise Exception() - except Exception, e: - return 1 - return 2 -''', 'f', []) - self.assertEquals(x, 1) - - def test_exception(self): - x = self.codetest(''' -def f(): - try: - raise Exception, 1 - except Exception, e: - return e.args[0] -''', 'f', []) - self.assertEquals(x, 1) - - def test_finally(self): - code = ''' -def f(a): - try: - if a: - raise Exception - a = -12 - finally: - return a -''' - self.assertEquals(self.codetest(code, 'f', [0]), -12) - self.assertEquals(self.codetest(code, 'f', [1]), 1) - -## def test_raise(self): -## x = self.codetest(''' -## def f(): -## raise 1 -## ''', 'f', []) -## self.assertEquals(x, '<<>>') - - def test_except2(self): - x = self.codetest(''' -def f(): - try: - z = 0 - try: - "x"+1 - except TypeError, e: - z = 5 - raise e - except TypeError: - return z -''', 'f', []) - self.assertEquals(x, 5) - - def test_except3(self): - code = ''' -def f(v): - z = 0 - try: - z = 1//v - except ZeroDivisionError, e: - z = "infinite result" - return z -''' - self.assertEquals(self.codetest(code, 'f', [2]), 0) - self.assertEquals(self.codetest(code, 'f', [0]), "infinite result") - ess = "TypeError: unsupported operand type" - res = self.codetest(code, 'f', ['x']) - self.failUnless(res.find(ess) >= 0) - # the following (original) test was a bit too strict...: - # self.assertEquals(self.codetest(code, 'f', ['x']), "<<>>") - - def test_break(self): - code = ''' -def f(n): - total = 0 - for i in range(n): - try: - if i == 4: - break - finally: - total += i - return total -''' - self.assertEquals(self.codetest(code, 'f', [4]), 1+2+3) - self.assertEquals(self.codetest(code, 'f', [9]), 1+2+3+4) - - def test_continue(self): - code = ''' -def f(n): - total = 0 - for i in range(n): - try: - if i == 4: - continue - finally: - total += 100 - total += i - return total -''' - self.assertEquals(self.codetest(code, 'f', [4]), 1+2+3+400) - self.assertEquals(self.codetest(code, 'f', [9]), - 1+2+3 + 5+6+7+8+900) - -class AppTestInterpreter(test.AppTestCase): - def test_exception(self): - try: - raise Exception, 1 - except Exception, e: - self.assertEquals(e.args[0], 1) - - def test_trivial(self): - x = 42 - self.assertEquals(x, 42) - - def test_raise(self): - def f(): - raise Exception - self.assertRaises(Exception, f) - - def test_exception(self): - try: - raise Exception - self.fail("exception failed to raise") - except: - pass - else: - self.fail("exception executing else clause!") - - def test_raise2(self): - def f(r): - try: - raise r - except LookupError: - return 1 - self.assertRaises(Exception, f, Exception) - self.assertEquals(f(IndexError), 1) - - def test_raise3(self): - try: - raise 1 - except TypeError: - pass - else: - self.fail("shouldn't be able to raise 1") - - def test_trivial_call(self): - def f(): return 42 - self.assertEquals(f(), 42) - - def test_trivial_call2(self): - def f(): return 1 + 1 - self.assertEquals(f(), 2) - - def test_print(self): - import sys - save = sys.stdout - class Out: - def __init__(self): - self.args = [] - def write(self, *args): - self.args.extend(args) - out = Out() - try: - sys.stdout = out - print 10 - self.assertEquals(out.args, ['10','\n']) - finally: - sys.stdout = save - - def test_identity(self): - def f(x): return x - self.assertEquals(f(666), 666) +class AppTestNestedScope(test.AppTestCase): def test_nested_scope(self): x = 42 @@ -244,37 +34,6 @@ return (a, b) self.assertEquals(f(), (3, 4)) - def test_generator(self): - def f(): - yield 1 - self.assertEquals(f().next(), 1) - - def test_generator2(self): - def f(): - yield 1 - g = f() - self.assertEquals(g.next(), 1) - self.assertRaises(StopIteration, g.next) - - def test_generator3(self): - def f(): - yield 1 - g = f() - self.assertEquals(list(g), [1]) - - def test_generator4(self): - def f(): - yield 1 - g = f() - self.assertEquals([x for x in g], [1]) - - def test_generator_restart(self): - def g(): - i = me.next() - yield i - me = g() - self.assertRaises(ValueError, me.next) - if __name__ == '__main__': test.main() Added: pypy/trunk/src/pypy/interpreter/test/test_pyframe.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/interpreter/test/test_pyframe.py Mon Sep 22 20:16:50 2003 @@ -0,0 +1,33 @@ +import autopath +from pypy.tool import test + + +class AppTestPyFrame(test.AppTestCase): + + # test for the presence of the attributes, not functionality + + def test_f_locals(self): + import sys + f = sys._getframe() + self.failUnless(f.f_locals is locals()) + + def test_f_globals(self): + import sys + f = sys._getframe() + self.failUnless(f.f_globals is globals()) + + def test_f_builtins(self): + import sys, __builtin__ + f = sys._getframe() + self.failUnless(f.f_builtins is __builtin__.__dict__) + + def test_f_code(self): + def g(): + import sys + f = sys._getframe() + return f.f_code + self.failUnless(g() is g.func_code) + + +if __name__ == '__main__': + test.main() Modified: pypy/trunk/src/pypy/module/sysmodule.py ============================================================================== --- pypy/trunk/src/pypy/module/sysmodule.py (original) +++ pypy/trunk/src/pypy/module/sysmodule.py Mon Sep 22 20:16:50 2003 @@ -1,5 +1,6 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.extmodule import ExtModule +from pypy.interpreter.gateway import interp2app import os, pypy import sys as cpy_sys @@ -38,3 +39,9 @@ except OperationError: print "! could not print", w_x space.setitem(space.w_builtins, w('_'), w_x) + + def _getframe(self): + # XXX No Argument Accepted Yet + f = self.space.getexecutioncontext().framestack.items[-1] + return self.space.wrap(f) + app__getframe = interp2app(_getframe) From tismer at codespeak.net Mon Sep 22 20:18:17 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 20:18:17 +0200 (MEST) Subject: [pypy-svn] rev 1397 - pypy/trunk/doc/funding Message-ID: <20030922181817.B98695C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 20:18:16 2003 New Revision: 1397 Added: pypy/trunk/doc/funding/B6.wp22_cpython_to_python.txt (contents, props changed) - copied, changed from rev 1393, pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt pypy/trunk/doc/funding/B6.wp52_small_os.txt (contents, props changed) - copied, changed from rev 1391, pypy/trunk/doc/funding/B6.wp11_small_os.txt Removed: pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt pypy/trunk/doc/funding/B6.wp11_small_os.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt pypy/trunk/doc/funding/B6.wp33_bootstrap_runtime.txt pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt pypy/trunk/doc/funding/B6.wp51_embedded_device.txt pypy/trunk/doc/funding/crossreferences.asc Log: enhanced estimates, inserted milstones Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 20:18:16 2003 @@ -4,8 +4,8 @@ =============================== The PyPy project addresses a number of topics, which -can be categorized into the following groups. -Each group is further broken down into working packages. +can be categorized into the following working packages. +Each group is further broken down into tasks as necessary. - development of PyPy itself, as a Python written in Python. This includes code generation for various targets, re-implementation @@ -57,13 +57,13 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The PyPy interpreter itself should be developed and completed as a regular Python/RPython program. This package includes all -parts of CPython that we don't want to move to WP02_. +parts of CPython that we don't want to move to WP22_. Further investigation is needed concerning the multimethod concepts of the standard object space, and how to hook in the bytecode compiler. -WP02_: Translation of CPython into regular Python +WP22_: Translation of CPython into regular Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Translate all other parts of CPython which we don't want to implement in WP21_ into regular Python libraries. @@ -98,7 +98,7 @@ _`Bootstrapping` PyPy ------------------------------ -The goal is to put interpreter (WP21_, WP02_) and translator +The goal is to put interpreter (WP21_, WP22_) and translator (WP31_) together. The interpreter is written as an RPython program. The translator @@ -247,7 +247,7 @@ the target platform. -WP11_: A small Operating System for an Embedded Device +WP52_: A small Operating System for an Embedded Device ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *we are thinking of a printer here, but don't have facts yet* @@ -361,4 +361,4 @@ do look for existing solutions which allow to automate this effort at least partially. -WP53_ WP54_ WP55_ WP15_ WP16_ WP17_ WP18_ WP19_ +WP53_ WP54_ WP55_ WP10_ WP60_ WP70_ WP18_ WP80_ Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 20:18:16 2003 @@ -3,44 +3,49 @@ Workpackage List ==================== -=============== ===================================================== ================== ============= =========== ========= ============== -Work-package No Workpackage title Lead contractor No Person-months Start month End month Deliverable No -=============== ===================================================== ================== ============= =========== ========= ============== -WP21_ Development and Completion of the PyPy Interpreter ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP02_ Translation of CPython into regular Python ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP31_ Translation of RPython ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP32_ Specific Analysis and Redesign ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP33_ Low Level PyPy Runtime ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP42_ Several Object Implementations ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP43_ Translator Optimisations ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP44_ Integration of Psyco ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP41_ Integration of Stackless ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP51_ Porting PyPy to an Embedded Device ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP11_ A small Operating System for an Embedded Device ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP53_ Code Generator for SIMD Architecture ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP54_ Enhanced Numerical Package ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP55_ Load Balancing in a Multi-Processor environment ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP15_ Coordination and Management ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP16_ Project Documentation ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP17_ Maintenance of Tools ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP18_ Quality Control ---------------- ----------------------------------------------------- ------------------ ------------- ----------- --------- -------------- -WP19_ Synchronisation with Standard Python -=============== ===================================================== ================== ============= =========== ========= ============== +.. |wpn| replace:: Work-package No +.. |lcn| replace:: Lead contractor No +.. |pm| replace:: Person-months +.. |sm| replace:: Start month +.. |em| replace:: End month +.. |dn| replace:: Deliverable No + +===== ===================================================== ===== ==== ==== ==== ==== +|wpn| Workpackage title |lcn| |pm| |sm| |em| |dn| +===== ===================================================== ===== ==== ==== ==== ==== +WP10_ Coordination and Management 18 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP21_ Development and Completion of the PyPy Interpreter 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP22_ Translation of CPython into regular Python 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP31_ Translation of RPython 8 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP32_ Specific Analysis and Redesign 8 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP33_ Low Level PyPy Runtime 6 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP41_ Integration of Stackless 9 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP42_ Several Object Implementations 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP43_ Translator Optimisations 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP44_ Integration of Psyco 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP51_ Porting PyPy to an Embedded Device 9 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP52_ A small Operating System for an Embedded Device 14 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP53_ Code Generator for SIMD Architecture 9 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP54_ Enhanced Numerical Package 6 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP55_ Load Balancing in a Multi-Processor environment 9 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP60_ Project Documentation 25 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP70_ Maintenance of Tools 10 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP80_ Synchronisation with Standard Python 6 +===== ===================================================== ===== ==== ==== ==== ==== Deleted: /pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt Mon Sep 22 20:18:16 2003 +++ (empty file) @@ -1,56 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Translation of CPython into regular Python -.. |wp| replace:: WP02 -.. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Z -.. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Translate all parts of CPython which we don't want to -implement in WP21_ into regular Python libraries. -Note: These ones should also work without PyPy, being just plain-Python -replacements for existing CPython functionality. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Coordinate with WP21_ which parts of CPython should be implemented in Python -- Implementation of all C modules and C objects -- Implement the Python bytecode compiler, which definately should - become a regular Python program, instead of being built - into the interpreter. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL01: First complete Python Implementation -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- Initial Prototype - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp11_small_os.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp11_small_os.txt Mon Sep 22 20:18:16 2003 +++ (empty file) @@ -1,72 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: A small Operating System for an Embedded Device -.. |wp| replace:: WP11 -.. |start| replace:: WP51_ -.. |p1| replace:: CT -.. |m1| replace:: 6 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: Y -.. |m3| replace:: 4 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Building all parts of a small Operating System - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Analyse the existing device drivers for the embedded device. - Figure out which drivers should be interfaced to, which - parts of the BIOS must be interfaced, and what has to be - recoded in Python. At least all drivers which have - blocking calls must be recoded in in Python. - -- Check whether we need to implement a TCP/IP stack and if this - work is doable in reasonable time. If not, then cancel this - WP in favor of another application WP to be done instead. - -- Recode the according device drivers in Python. Design tests - which ensure functional compatibility with the original ones, - as far as these exist. - -- Extend the simulator of WP51_ accordingly. - -- Implement the OS kernel as a set of prioritized tasklets - (using the support from WP41_) - -- Implement a simple file system which is accessible by - Python, only. - -- Implement a few basic protocols like FTP in order to provide - basic access to the internal data of the embedded device. - -- Test the implementation both on the simulator and on the real hardware. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? Python implements the whole OS of the embedded device. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt (original) +++ pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt Mon Sep 22 20:18:16 2003 @@ -1,7 +1,7 @@ .. include:: crossreferences.asc .. |title| replace:: Development and Completion of the PyPy Interpreter -.. |wp| replace:: WP01 +.. |wp| replace:: WP21 .. |start| replace:: 0 .. |p1| replace:: X .. |m1| replace:: 3 @@ -9,8 +9,8 @@ .. |m2| replace:: 3 .. |p3| replace:: Z .. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| +.. |p4| replace:: T +.. |m4| replace:: 3 .. |p5| replace:: |e| .. |m5| replace:: |e| .. |p6| replace:: |e| @@ -24,14 +24,14 @@ The PyPy interpreter itself should be developed and completed as a regular Python program. This package includes all -parts of CPython that we don't want to move to WP02_. +parts of CPython that we don't want to move to WP22_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -- Coordinate with WP02_ which parts of CPython +- Coordinate with WP22_ which parts of CPython should be implemented in RPython. - Implementation of all C modules and C objects - Further investigation of the unorthodox multimethod @@ -50,6 +50,6 @@ **Milestones and Expected Result** -- Initial Prototype +- M1 with WP22_ .. include:: wp-tableend.asc Copied: pypy/trunk/doc/funding/B6.wp22_cpython_to_python.txt (from rev 1393, pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp02_cpython_to_python.txt (original) +++ pypy/trunk/doc/funding/B6.wp22_cpython_to_python.txt Mon Sep 22 20:18:16 2003 @@ -1,7 +1,7 @@ .. include:: crossreferences.asc .. |title| replace:: Translation of CPython into regular Python -.. |wp| replace:: WP02 +.. |wp| replace:: WP22 .. |start| replace:: 0 .. |p1| replace:: X .. |m1| replace:: 3 @@ -9,8 +9,8 @@ .. |m2| replace:: 3 .. |p3| replace:: Z .. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| +.. |p4| replace:: T +.. |m4| replace:: 3 .. |p5| replace:: |e| .. |m5| replace:: |e| .. |p6| replace:: |e| @@ -51,6 +51,6 @@ **Milestones and Expected Result** -- Initial Prototype +- M1 with WP21_ .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt (original) +++ pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt Mon Sep 22 20:18:16 2003 @@ -1,12 +1,12 @@ .. include:: crossreferences.asc .. |title| replace:: Translation of RPython -.. |wp| replace:: WP03 -.. |start| replace:: 0 +.. |wp| replace:: WP31 +.. |start| replace:: M1 .. |p1| replace:: X -.. |m1| replace:: 3 +.. |m1| replace:: 4 .. |p2| replace:: Y -.. |m2| replace:: 3 +.. |m2| replace:: 4 .. |p3| replace:: |e| .. |m3| replace:: |e| .. |p4| replace:: |e| @@ -52,14 +52,16 @@ - Provide some measure for the efficiency of the generated code, in comparison to Standard Python. +Note that this task can actually start earlier than M1, but +finally becomes dependent from it. + .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Deliverables** -- DL02: Stand-alone Python Interpreter -- DL03: Tool for Analysis of restricted Python Programs -- DL??? Passing all applicable elements of the standard test suite. +- DL???: Tool for Analysis of restricted Python Programs +- DL???: Passing all applicable elements of the standard test suite. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc Modified: pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt (original) +++ pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt Mon Sep 22 20:18:16 2003 @@ -1,12 +1,12 @@ .. include:: crossreferences.asc .. |title| replace:: Specific Analysis and Redesign -.. |wp| replace:: WP04 -.. |start| replace:: 0 +.. |wp| replace:: WP32 +.. |start| replace:: WP31_ .. |p1| replace:: X -.. |m1| replace:: 3 +.. |m1| replace:: 4 .. |p2| replace:: Y -.. |m2| replace:: 3 +.. |m2| replace:: 4 .. |p3| replace:: |e| .. |m3| replace:: |e| .. |p4| replace:: |e| @@ -22,7 +22,7 @@ **Objectives** -- Putting interpreter (WP21_, WP02_) and translator (WP31_) together. +- Putting interpreter (WP21_, WP22_) and translator (WP31_) together. - Enhancing the PyPy Design. .. include:: wp-tableend.asc @@ -52,4 +52,6 @@ **Milestones and Expected Result** +- M2 with WP33_ + .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp33_bootstrap_runtime.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp33_bootstrap_runtime.txt (original) +++ pypy/trunk/doc/funding/B6.wp33_bootstrap_runtime.txt Mon Sep 22 20:18:16 2003 @@ -1,7 +1,7 @@ .. include:: crossreferences.asc .. |title| replace:: Low Level PyPy Runtime -.. |wp| replace:: WP05 +.. |wp| replace:: WP33 .. |start| replace:: WP31_ .. |p1| replace:: X .. |m1| replace:: 3 @@ -57,4 +57,6 @@ **Milestones and Expected Result** +- M2 with WP32_ + .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt (original) +++ pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt Mon Sep 22 20:18:16 2003 @@ -1,14 +1,14 @@ .. include:: crossreferences.asc .. |title| replace:: Integration of Stackless -.. |wp| replace:: WP09 -.. |start| replace:: WP43_ +.. |wp| replace:: WP41 +.. |start| replace:: M2 .. |p1| replace:: CT -.. |m1| replace:: 6 +.. |m1| replace:: 5 .. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: Y -.. |m3| replace:: 4 +.. |m2| replace:: 2 +.. |p3| replace:: Z +.. |m3| replace:: 2 .. |p4| replace:: |e| .. |m4| replace:: |e| .. |p5| replace:: |e| @@ -44,7 +44,7 @@ - Implement a dedicated run-time component (WP33_) for Stackless. - Implement a single-threaded, pre-emptive scheduler with - priorities. This is necessary to support small OSes, like WP11_. + priorities. This is necessary to support small OSes, like WP52_. - Further develop these technologies based on the results gathered in WP43_, e.g. identify when these technologies would guide specific choices among the @@ -60,4 +60,6 @@ **Milestones and Expected Result** +- M3 + .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt (original) +++ pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt Mon Sep 22 20:18:16 2003 @@ -1,14 +1,14 @@ .. include:: crossreferences.asc .. |title| replace:: Several Object Implementations -.. |wp| replace:: WP06 -.. |start| replace:: WP31_ +.. |wp| replace:: WP42 +.. |start| replace:: M2 .. |p1| replace:: X -.. |m1| replace:: 2 +.. |m1| replace:: 4 .. |p2| replace:: Y -.. |m2| replace:: 2 -.. |p3| replace:: |e| -.. |m3| replace:: |e| +.. |m2| replace:: 4 +.. |p3| replace:: Z +.. |m3| replace:: 4 .. |p4| replace:: |e| .. |m4| replace:: |e| .. |p5| replace:: |e| @@ -68,4 +68,6 @@ **Milestones and Expected Result** +- M4 with M3 WP43_ WP44_ + .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt (original) +++ pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt Mon Sep 22 20:18:16 2003 @@ -1,12 +1,12 @@ .. include:: crossreferences.asc .. |title| replace:: Translator Optimisations -.. |wp| replace:: WP07 -.. |start| replace:: WP31_ +.. |wp| replace:: WP43 +.. |start| replace:: M2 .. |p1| replace:: X -.. |m1| replace:: 4 +.. |m1| replace:: 6 .. |p2| replace:: Y -.. |m2| replace:: 4 +.. |m2| replace:: 6 .. |p3| replace:: |e| .. |m3| replace:: |e| .. |p4| replace:: |e| @@ -74,4 +74,6 @@ **Milestones and Expected Result** +- M4 with M3 WP42_ WP44_ + .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt (original) +++ pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt Mon Sep 22 20:18:16 2003 @@ -1,16 +1,16 @@ .. include:: crossreferences.asc .. |title| replace:: Integration of Psyco -.. |wp| replace:: WP08 -.. |start| replace:: WP43_ +.. |wp| replace:: WP44 +.. |start| replace:: M2 .. |p1| replace:: AR .. |m1| replace:: 6 .. |p2| replace:: Y -.. |m2| replace:: 4 +.. |m2| replace:: 2 .. |p3| replace:: Y -.. |m3| replace:: 4 -.. |p4| replace:: |e| -.. |m4| replace:: |e| +.. |m3| replace:: 2 +.. |p4| replace:: Z +.. |m4| replace:: 2 .. |p5| replace:: |e| .. |m5| replace:: |e| .. |p6| replace:: |e| @@ -59,4 +59,6 @@ **Milestones and Expected Result** +- M4 with M3 WP42_ WP43_ + .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp51_embedded_device.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp51_embedded_device.txt (original) +++ pypy/trunk/doc/funding/B6.wp51_embedded_device.txt Mon Sep 22 20:18:16 2003 @@ -1,8 +1,8 @@ .. include:: crossreferences.asc .. |title| replace:: Porting PyPy to an Embedded Device -.. |wp| replace:: WP10 -.. |start| replace:: WP33_ +.. |wp| replace:: WP51 +.. |start| replace:: M2 .. |p1| replace:: X .. |m1| replace:: 3 .. |p2| replace:: Y @@ -63,4 +63,6 @@ **Milestones and Expected Result** +- M5 + .. include:: wp-tableend.asc Copied: pypy/trunk/doc/funding/B6.wp52_small_os.txt (from rev 1391, pypy/trunk/doc/funding/B6.wp11_small_os.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp11_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp52_small_os.txt Mon Sep 22 20:18:16 2003 @@ -2,7 +2,7 @@ .. |title| replace:: A small Operating System for an Embedded Device .. |wp| replace:: WP11 -.. |start| replace:: WP51_ +.. |start| replace:: M5 .. |p1| replace:: CT .. |m1| replace:: 6 .. |p2| replace:: Y @@ -24,6 +24,9 @@ - Building all parts of a small Operating System + +**FIXME: Split this one into two and insert M6 between!** + .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -69,4 +72,6 @@ **Milestones and Expected Result** +- M7 + .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 20:18:16 2003 @@ -1,11 +1,7 @@ -.. _WP02: B6.wp02_cpython_to_python.html -.. _WP11: B6.wp11_small_os.html -.. _WP15: -.. _WP16: -.. _WP17: +.. _WP10: .. _WP18: -.. _WP19: .. _WP21: B6.wp21_interpreter_completion.html +.. _WP22: B6.wp22_cpython_to_python.html .. _WP31: B6.wp31_translation_of_rpython.html .. _WP32: B6.wp32_bootstrap_redesign.html .. _WP33: B6.wp33_bootstrap_runtime.html @@ -14,9 +10,13 @@ .. _WP43: B6.wp43_translator_optimisations.html .. _WP44: B6.wp44_integrate_psyco.html .. _WP51: B6.wp51_embedded_device.html +.. _WP52: B6.wp52_small_os.html .. _WP53: .. _WP54: .. _WP55: -.. attention:: **8 files missing: WP15 WP16 WP17 WP18 WP19 WP53 WP54 WP55** +.. _WP60: +.. _WP70: +.. _WP80: +.. attention:: **8 files missing: WP10 WP18 WP53 WP54 WP55 WP60 WP70 WP80** .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file From tismer at codespeak.net Mon Sep 22 20:21:53 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 20:21:53 +0200 (MEST) Subject: [pypy-svn] rev 1398 - pypy/trunk/doc/funding Message-ID: <20030922182153.D80F85C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 20:21:52 2003 New Revision: 1398 Added: pypy/trunk/doc/funding/B6.wp53_small_os.txt (props changed) - copied unchanged from rev 1397, pypy/trunk/doc/funding/B6.wp52_small_os.txt Removed: pypy/trunk/doc/funding/B6.wp52_small_os.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt pypy/trunk/doc/funding/crossreferences.asc Log: splitting the small os Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 20:21:52 2003 @@ -247,7 +247,7 @@ the target platform. -WP52_: A small Operating System for an Embedded Device +WP53_: A small Operating System for an Embedded Device ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *we are thinking of a printer here, but don't have facts yet* @@ -361,4 +361,4 @@ do look for existing solutions which allow to automate this effort at least partially. -WP53_ WP54_ WP55_ WP10_ WP60_ WP70_ WP18_ WP80_ +WP54_ WP55_ WP56_ WP10_ WP60_ WP70_ WP18_ WP80_ Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 20:21:52 2003 @@ -35,13 +35,13 @@ ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP51_ Porting PyPy to an Embedded Device 9 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- -WP52_ A small Operating System for an Embedded Device 14 +WP53_ A small Operating System for an Embedded Device 14 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- -WP53_ Code Generator for SIMD Architecture 9 +WP54_ Code Generator for SIMD Architecture 9 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- -WP54_ Enhanced Numerical Package 6 +WP55_ Enhanced Numerical Package 6 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- -WP55_ Load Balancing in a Multi-Processor environment 9 +WP56_ Load Balancing in a Multi-Processor environment 9 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP60_ Project Documentation 25 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- Modified: pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt (original) +++ pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt Mon Sep 22 20:21:52 2003 @@ -44,7 +44,7 @@ - Implement a dedicated run-time component (WP33_) for Stackless. - Implement a single-threaded, pre-emptive scheduler with - priorities. This is necessary to support small OSes, like WP52_. + priorities. This is necessary to support small OSes, like WP53_. - Further develop these technologies based on the results gathered in WP43_, e.g. identify when these technologies would guide specific choices among the Deleted: /pypy/trunk/doc/funding/B6.wp52_small_os.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp52_small_os.txt Mon Sep 22 20:21:52 2003 +++ (empty file) @@ -1,77 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: A small Operating System for an Embedded Device -.. |wp| replace:: WP11 -.. |start| replace:: M5 -.. |p1| replace:: CT -.. |m1| replace:: 6 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: Y -.. |m3| replace:: 4 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Building all parts of a small Operating System - - -**FIXME: Split this one into two and insert M6 between!** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Analyse the existing device drivers for the embedded device. - Figure out which drivers should be interfaced to, which - parts of the BIOS must be interfaced, and what has to be - recoded in Python. At least all drivers which have - blocking calls must be recoded in in Python. - -- Check whether we need to implement a TCP/IP stack and if this - work is doable in reasonable time. If not, then cancel this - WP in favor of another application WP to be done instead. - -- Recode the according device drivers in Python. Design tests - which ensure functional compatibility with the original ones, - as far as these exist. - -- Extend the simulator of WP51_ accordingly. - -- Implement the OS kernel as a set of prioritized tasklets - (using the support from WP41_) - -- Implement a simple file system which is accessible by - Python, only. - -- Implement a few basic protocols like FTP in order to provide - basic access to the internal data of the embedded device. - -- Test the implementation both on the simulator and on the real hardware. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? Python implements the whole OS of the embedded device. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M7 - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 20:21:52 2003 @@ -10,13 +10,13 @@ .. _WP43: B6.wp43_translator_optimisations.html .. _WP44: B6.wp44_integrate_psyco.html .. _WP51: B6.wp51_embedded_device.html -.. _WP52: B6.wp52_small_os.html -.. _WP53: +.. _WP53: B6.wp53_small_os.html .. _WP54: .. _WP55: +.. _WP56: .. _WP60: .. _WP70: .. _WP80: -.. attention:: **8 files missing: WP10 WP18 WP53 WP54 WP55 WP60 WP70 WP80** +.. attention:: **8 files missing: WP10 WP18 WP54 WP55 WP56 WP60 WP70 WP80** .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file From tismer at codespeak.net Mon Sep 22 20:23:37 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 20:23:37 +0200 (MEST) Subject: [pypy-svn] rev 1399 - pypy/trunk/doc/funding Message-ID: <20030922182337.DB32B5C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 20:23:37 2003 New Revision: 1399 Added: pypy/trunk/doc/funding/B6.wp52_small_os_pre.txt (props changed) - copied unchanged from rev 1398, pypy/trunk/doc/funding/B6.wp53_small_os.txt Modified: pypy/trunk/doc/funding/crossreferences.asc Log: splitting small os Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 20:23:37 2003 @@ -10,6 +10,7 @@ .. _WP43: B6.wp43_translator_optimisations.html .. _WP44: B6.wp44_integrate_psyco.html .. _WP51: B6.wp51_embedded_device.html +.. _WP52: B6.wp52_small_os_pre.html .. _WP53: B6.wp53_small_os.html .. _WP54: .. _WP55: From tismer at codespeak.net Mon Sep 22 20:32:12 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 20:32:12 +0200 (MEST) Subject: [pypy-svn] rev 1400 - pypy/trunk/doc/funding Message-ID: <20030922183212.C9D275C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 20:32:12 2003 New Revision: 1400 Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/B6.wp52_small_os_pre.txt pypy/trunk/doc/funding/B6.wp53_small_os.txt Log: end of big renaming, so far. Some WPs have yet to be added as files. Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 20:32:12 2003 @@ -35,7 +35,9 @@ ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP51_ Porting PyPy to an Embedded Device 9 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- -WP53_ A small Operating System for an Embedded Device 14 +WP52_ Evaluation whether to do a small OS in PyPy 3 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP53_ A small Operating System for an Embedded Device 11 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP54_ Code Generator for SIMD Architecture 9 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- Modified: pypy/trunk/doc/funding/B6.wp52_small_os_pre.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp52_small_os_pre.txt (original) +++ pypy/trunk/doc/funding/B6.wp52_small_os_pre.txt Mon Sep 22 20:32:12 2003 @@ -1,14 +1,14 @@ .. include:: crossreferences.asc -.. |title| replace:: A small Operating System for an Embedded Device -.. |wp| replace:: WP11 +.. |title| replace:: Evaluation whether to do a small OS in PyPy +.. |wp| replace:: WP52 .. |start| replace:: M5 .. |p1| replace:: CT -.. |m1| replace:: 6 +.. |m1| replace:: 1 .. |p2| replace:: Y -.. |m2| replace:: 4 +.. |m2| replace:: 1 .. |p3| replace:: Y -.. |m3| replace:: 4 +.. |m3| replace:: 1 .. |p4| replace:: |e| .. |m4| replace:: |e| .. |p5| replace:: |e| @@ -22,10 +22,7 @@ **Objectives** -- Building all parts of a small Operating System - - -**FIXME: Split this one into two and insert M6 between!** +- Deciding whether to build a small Operating System .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -39,39 +36,19 @@ blocking calls must be recoded in in Python. - Check whether we need to implement a TCP/IP stack and if this - work is doable in reasonable time. If not, then cancel this - WP in favor of another application WP to be done instead. - -- Recode the according device drivers in Python. Design tests - which ensure functional compatibility with the original ones, - as far as these exist. - -- Extend the simulator of WP51_ accordingly. - -- Implement the OS kernel as a set of prioritized tasklets - (using the support from WP41_) + work is doable in reasonable time. If not, then cancel the + next WP in favor of another validation WP to be done instead. -- Implement a simple file system which is accessible by - Python, only. - -- Implement a few basic protocols like FTP in order to provide - basic access to the internal data of the embedded device. - -- Test the implementation both on the simulator and on the real hardware. - .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Deliverables** -- DL??? Python implements the whole OS of the embedded device. -- DL??? Passing all applicable elements of the standard test suite. - .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Milestones and Expected Result** -- M7 +- M6 .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/B6.wp53_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp53_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp53_small_os.txt Mon Sep 22 20:32:12 2003 @@ -1,14 +1,14 @@ .. include:: crossreferences.asc .. |title| replace:: A small Operating System for an Embedded Device -.. |wp| replace:: WP11 -.. |start| replace:: M5 +.. |wp| replace:: WP53 +.. |start| replace:: M6 .. |p1| replace:: CT -.. |m1| replace:: 6 +.. |m1| replace:: 5 .. |p2| replace:: Y -.. |m2| replace:: 4 +.. |m2| replace:: 3 .. |p3| replace:: Y -.. |m3| replace:: 4 +.. |m3| replace:: 3 .. |p4| replace:: |e| .. |m4| replace:: |e| .. |p5| replace:: |e| @@ -32,19 +32,9 @@ **Description of work** -- Analyse the existing device drivers for the embedded device. - Figure out which drivers should be interfaced to, which - parts of the BIOS must be interfaced, and what has to be - recoded in Python. At least all drivers which have - blocking calls must be recoded in in Python. - -- Check whether we need to implement a TCP/IP stack and if this - work is doable in reasonable time. If not, then cancel this - WP in favor of another application WP to be done instead. - -- Recode the according device drivers in Python. Design tests - which ensure functional compatibility with the original ones, - as far as these exist. +- Recode device drivers in Python, according to the results of WP52_. + Design tests which ensure functional compatibility with the original + ones, as far as these exist. - Extend the simulator of WP51_ accordingly. From tismer at codespeak.net Mon Sep 22 20:39:52 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 20:39:52 +0200 (MEST) Subject: [pypy-svn] rev 1401 - pypy/trunk/doc/funding Message-ID: <20030922183952.381905C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 20:39:51 2003 New Revision: 1401 Modified: pypy/trunk/doc/funding/B6.wp53_small_os.txt Log: small correction Modified: pypy/trunk/doc/funding/B6.wp53_small_os.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp53_small_os.txt (original) +++ pypy/trunk/doc/funding/B6.wp53_small_os.txt Mon Sep 22 20:39:51 2003 @@ -24,9 +24,6 @@ - Building all parts of a small Operating System - -**FIXME: Split this one into two and insert M6 between!** - .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc From tismer at codespeak.net Mon Sep 22 21:05:11 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 21:05:11 +0200 (MEST) Subject: [pypy-svn] rev 1402 - pypy/trunk/doc/funding Message-ID: <20030922190511.203495C2E3@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 21:05:10 2003 New Revision: 1402 Added: pypy/trunk/doc/funding/B6.wp10_management.txt (contents, props changed) - copied, changed from rev 1397, pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/crossreferences.asc Log: added management. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 21:05:10 2003 @@ -7,7 +7,7 @@ can be categorized into the following working packages. Each group is further broken down into tasks as necessary. -- development of PyPy itself, as a Python written in Python. +- Development of PyPy itself, as a Python written in Python. This includes code generation for various targets, re-implementation of all builtin Python objects and some extension modules, and the development of several object spaces, testing and documentation. @@ -20,8 +20,8 @@ - High-performance PyPy-Python -- application level code which shows the results of PyPy as a usability - study. Generating down-sized code for embedded systems, load balancing +- Validation of PyPy in real applications. + Generating down-sized code for embedded systems, load balancing in a distributed network, code generators optimized for number crunching on some processor architectures and re-writing numerical Python packages should be able to prove that PyPy is more than an @@ -38,16 +38,24 @@ - Coordination and Management - - Project Documentation + - Project Documentation and Dissemination - Maintenance of Tools - - - Quality Control - + - Synchronisation with Standard Python + +WP10_: Coordination and Management +------------------------------------- +WP1 continues throughout the duration of the project and is carried out +by the project coordinator. It involves the collection and monitoring of +monthly status reports, reporting to the EU, organising meetings +and maintaining an internal web site. The website will make heavy +use of collaborative tools like Wiki pages, a content management system, +and issue trackers. + -The PyPy Interpreter +WP20_: The PyPy Interpreter --------------------------- The goal is to make a complete Python interpreter that runs under any @@ -55,48 +63,47 @@ WP21_: Development and Completion of the PyPy Interpreter ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The PyPy interpreter itself should be developed and completed -as a regular Python/RPython program. This package includes all -parts of CPython that we don't want to move to WP22_. -Further investigation is needed concerning the -multimethod concepts of the standard object space, -and how to hook in the bytecode compiler. + The PyPy interpreter itself should be developed and completed + as a regular Python/RPython program. This package includes all + parts of CPython that we don't want to move to WP22_. + Further investigation is needed concerning the + multimethod concepts of the standard object space, + and how to hook in the bytecode compiler. WP22_: Translation of CPython into regular Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Translate all other parts of CPython which we don't want to -implement in WP21_ into regular Python libraries. -These ones should also work without PyPy, being just plain-Python -replacements for existing CPython functionality. -This includes the bytecode compiler, which definitely should -become a regular Python program, instead of being built -into the interpreter. + Translate all other parts of CPython which we don't want to + implement in WP21_ into regular Python libraries. + These ones should also work without PyPy, being just plain-Python + replacements for existing CPython functionality. + This includes the bytecode compiler, which definitely should + become a regular Python program, instead of being built + into the interpreter. -The PyPy Compiler +WP30_: The PyPy Compiler --------------------------- - WP31_: Translation of RPython ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -RPython is a restricted version of Python, which is much -more statically defined than standard Python. It allows -to write code that is rather easily translated into something -else, like C source, Pyrex code, or even passed to -an assembly code generator. -The Definition of RPython is slightly in flux and should -be adjusted during WP31_. - -The goal is to be able to translate arbitrary RPython source code (e.g. -the one produced in WP21_ into low-level code (C, Pyrex, Java, others). -This includes making a stand-alone, not-PyPy-related tool for general -optimisation of arbitrary but suitably restricted Python application or -parts thereof. + RPython is a restricted version of Python, which is much + more statically defined than standard Python. It allows + to write code that is rather easily translated into something + else, like C source, Pyrex code, or even passed to + an assembly code generator. + The Definition of RPython is slightly in flux and should + be adjusted during WP31_. + + The goal is to be able to translate arbitrary RPython source code (e.g. + the one produced in WP21_ into low-level code (C, Pyrex, Java, others). + This includes making a stand-alone, not-PyPy-related tool for general + optimisation of arbitrary but suitably restricted Python application or + parts thereof. -_`Bootstrapping` PyPy ------------------------------- +Bootstrapping PyPy +~~~~~~~~~~~~~~~~~~~ The goal is to put interpreter (WP21_, WP22_) and translator (WP31_) together. @@ -113,31 +120,31 @@ WP32_: Specific Analysis and Redesign ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The global translation of PyPy is going to raise particular -problems, other than more general RPython programs do. -Since translation of RPython is the core idea of the -bootstrap process and the main target of the translator, -we need to investigate and isolate the -particular problems, and redesign specific parts of PyPy -to support translation, code generation and optimisation -better. This will also include a re-iteration of the interface -design between application level and interpreter level -until we reach overall convergence. - + The global translation of PyPy is going to raise particular + problems, other than more general RPython programs do. + Since translation of RPython is the core idea of the + bootstrap process and the main target of the translator, + we need to investigate and isolate the + particular problems, and redesign specific parts of PyPy + to support translation, code generation and optimisation + better. This will also include a re-iteration of the interface + design between application level and interpreter level + until we reach overall convergence. + WP33_: Low Level PyPy Runtime ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In order to give a working environment to the translated RPython -program, we need to build the low-level-specific **run-time** -components of PyPy. -Most notably are the object layout, the memory management, -and possibly threading support, as well as an -efficient multimethod dispatch. -The target language is not yet decided on. We might go different -paths at the same time. -If producing C code is a target, important -parts can be directly re-used from CPython. + In order to give a working environment to the translated RPython + program, we need to build the low-level-specific **run-time** + components of PyPy. + Most notably are the object layout, the memory management, + and possibly threading support, as well as an + efficient multimethod dispatch. + The target language is not yet decided on. We might go different + paths at the same time. + If producing C code is a target, important + parts can be directly re-used from CPython. High-performance PyPy-Python @@ -306,21 +313,9 @@ Infrastructure ------------------ -*XXX: Numbering these beginning with 100 for the moment -until we know the exact number of packages that come before* -[[WP101]]: Coordination and Management -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -WP1 continues throughout the duration of the project and is carried out -by the project coordinator. It involves the collection and monitoring of -monthly status reports, reporting to the EU, organising meetings -and maintaining an internal web site. The website will make heavy -use of collaborative tools like Wiki pages, a content management system, -and issue trackers. - - -[[WP102]]: Project Documentation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +WP60_: Project Documentation and Dissemination +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ During the whole project, a set of documents will be maintained that documents the current status of the project, results of @@ -329,7 +324,7 @@ various congresses. -[[WP103]]: Maintenance of Tools +WP70_: Maintenance of Tools ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ PyPy's own development needs an infrastructure that must continuously @@ -339,19 +334,7 @@ documentation extraction, and change notification via mailing lists. -[[WP104]]: Quality Control -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The PyPy project has included an exhaustive set of tests for -all software modules. This has to be extended and maintained -throughout the whole project. The intent is to be able to -run all of our internal tests frequently, and to be able to -run the standard Python test suite, as far as applicable. - -XXX: How do we write this, to make clear that a cetain amount -of time is needed for testing, in every milestone? - -[[WP105]]: Synchronisation with Standard Python +WP80_: Synchronisation with Standard Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Since Python is being further developed all the time, there is Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 21:05:10 2003 @@ -45,7 +45,7 @@ ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP56_ Load Balancing in a Multi-Processor environment 9 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- -WP60_ Project Documentation 25 +WP60_ Project Documentation and Dissemination 25 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP70_ Maintenance of Tools 10 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- Copied: pypy/trunk/doc/funding/B6.wp10_management.txt (from rev 1397, pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt (original) +++ pypy/trunk/doc/funding/B6.wp10_management.txt Mon Sep 22 21:05:10 2003 @@ -1,16 +1,16 @@ .. include:: crossreferences.asc -.. |title| replace:: Development and Completion of the PyPy Interpreter -.. |wp| replace:: WP21 +.. |title| replace:: Coordination and Management +.. |wp| replace:: WP10 .. |start| replace:: 0 .. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Z -.. |m3| replace:: 3 -.. |p4| replace:: T -.. |m4| replace:: 3 +.. |m1| replace:: 18 +.. |p2| replace:: |e| +.. |m2| replace:: |e| +.. |p3| replace:: |e| +.. |m3| replace:: |e| +.. |p4| replace:: |e| +.. |m4| replace:: |e| .. |p5| replace:: |e| .. |m5| replace:: |e| .. |p6| replace:: |e| @@ -22,34 +22,28 @@ **Objectives** -The PyPy interpreter itself should be developed and completed -as a regular Python program. This package includes all -parts of CPython that we don't want to move to WP22_. +Coordination and Management of the PyPy Project. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -- Coordinate with WP22_ which parts of CPython - should be implemented in RPython. -- Implementation of all C modules and C objects -- Further investigation of the unorthodox multimethod - concepts that the standard object space is based on, - and how to hook in the bytecode compiler. +- collection and monitoring of monthly status reports, +- reporting to the EU +- organising meetings +- maintaining an internal web site. The website will make heavy + use of collaborative tools like Wiki pages, a content management system, + and issue trackers. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Deliverables** -- DL01: First complete Python Implementation - .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Milestones and Expected Result** -- M1 with WP22_ - .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Mon Sep 22 21:05:10 2003 @@ -1,7 +1,9 @@ -.. _WP10: +.. _WP10: B6.wp10_management.html .. _WP18: +.. _WP20: .. _WP21: B6.wp21_interpreter_completion.html .. _WP22: B6.wp22_cpython_to_python.html +.. _WP30: .. _WP31: B6.wp31_translation_of_rpython.html .. _WP32: B6.wp32_bootstrap_redesign.html .. _WP33: B6.wp33_bootstrap_runtime.html @@ -18,6 +20,6 @@ .. _WP60: .. _WP70: .. _WP80: -.. attention:: **8 files missing: WP10 WP18 WP54 WP55 WP56 WP60 WP70 WP80** +.. attention:: **9 files missing: WP18 WP20 WP30 WP54 WP55 WP56 WP60 WP70 WP80** .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file From tismer at codespeak.net Mon Sep 22 21:10:44 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 21:10:44 +0200 (MEST) Subject: [pypy-svn] rev 1403 - pypy/trunk/doc/funding Message-ID: <20030922191044.8B3B05C2E4@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 21:10:43 2003 New Revision: 1403 Added: pypy/trunk/doc/funding/B6.wp20_interpreter.txt (contents, props changed) - copied, changed from rev 1402, pypy/trunk/doc/funding/B6.wp10_management.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt Log: adding more packages which are top-level Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 22 21:10:43 2003 @@ -344,4 +344,4 @@ do look for existing solutions which allow to automate this effort at least partially. -WP54_ WP55_ WP56_ WP10_ WP60_ WP70_ WP18_ WP80_ +WP54_ WP55_ WP56_ WP10_ WP60_ WP70_ WP80_ Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 22 21:10:43 2003 @@ -15,10 +15,14 @@ ===== ===================================================== ===== ==== ==== ==== ==== WP10_ Coordination and Management 18 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP20_ The PyPy Interpreter +----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP21_ Development and Completion of the PyPy Interpreter 12 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP22_ Translation of CPython into regular Python 12 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP20_ The PyPy Compiler +----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP31_ Translation of RPython 8 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP32_ Specific Analysis and Redesign 8 Copied: pypy/trunk/doc/funding/B6.wp20_interpreter.txt (from rev 1402, pypy/trunk/doc/funding/B6.wp10_management.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp10_management.txt (original) +++ pypy/trunk/doc/funding/B6.wp20_interpreter.txt Mon Sep 22 21:10:43 2003 @@ -1,10 +1,10 @@ .. include:: crossreferences.asc -.. |title| replace:: Coordination and Management -.. |wp| replace:: WP10 +.. |title| replace:: The PyPy Interpreter +.. |wp| replace:: WP20 .. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 18 +.. |p1| replace:: |e| +.. |m1| replace:: |e| .. |p2| replace:: |e| .. |m2| replace:: |e| .. |p3| replace:: |e| @@ -22,19 +22,14 @@ **Objectives** -Coordination and Management of the PyPy Project. +Building a complete PyPy interpreter. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -- collection and monitoring of monthly status reports, -- reporting to the EU -- organising meetings -- maintaining an internal web site. The website will make heavy - use of collaborative tools like Wiki pages, a content management system, - and issue trackers. +This working package consists of the tasks WP21_ WP22_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -46,4 +41,6 @@ **Milestones and Expected Result** +- M1 + .. include:: wp-tableend.asc From tismer at codespeak.net Mon Sep 22 21:15:15 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 22 Sep 2003 21:15:15 +0200 (MEST) Subject: [pypy-svn] rev 1404 - pypy/trunk/doc/funding Message-ID: <20030922191515.C45125C2E5@thoth.codespeak.net> Author: tismer Date: Mon Sep 22 21:15:15 2003 New Revision: 1404 Added: pypy/trunk/doc/funding/B6.wp30_compiler.txt (contents, props changed) - copied, changed from rev 1403, pypy/trunk/doc/funding/B6.wp20_interpreter.txt Log: just a stop for dinner, continuing from home... Copied: pypy/trunk/doc/funding/B6.wp30_compiler.txt (from rev 1403, pypy/trunk/doc/funding/B6.wp20_interpreter.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp20_interpreter.txt (original) +++ pypy/trunk/doc/funding/B6.wp30_compiler.txt Mon Sep 22 21:15:15 2003 @@ -1,7 +1,7 @@ .. include:: crossreferences.asc -.. |title| replace:: The PyPy Interpreter -.. |wp| replace:: WP20 +.. |title| replace:: The PyPy Compiler +.. |wp| replace:: WP30 .. |start| replace:: 0 .. |p1| replace:: |e| .. |m1| replace:: |e| @@ -22,14 +22,14 @@ **Objectives** -Building a complete PyPy interpreter. +Building the translator and Bootstrapping PyPy. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -This working package consists of the tasks WP21_ WP22_. +This working package consists of the tasks WP31_ WP32_ WP33_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -41,6 +41,6 @@ **Milestones and Expected Result** -- M1 +- M2 .. include:: wp-tableend.asc From tismer at codespeak.net Tue Sep 23 02:18:35 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 23 Sep 2003 02:18:35 +0200 (MEST) Subject: [pypy-svn] rev 1405 - pypy/trunk/doc/funding Message-ID: <20030923001835.D7FC35ACAE@thoth.codespeak.net> Author: tismer Date: Tue Sep 23 02:18:34 2003 New Revision: 1405 Added: pypy/trunk/doc/funding/B6.wp40_high_performance.txt (contents, props changed) - copied, changed from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt pypy/trunk/doc/funding/B6.wp50_validations.txt (contents, props changed) - copied, changed from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/crossreferences.asc Log: various changes, new WPs (on top of existing ones), just a few left. Tried to get more structure into the introduction. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 23 02:18:34 2003 @@ -147,7 +147,7 @@ parts can be directly re-used from CPython. -High-performance PyPy-Python +WP40_: High-performance PyPy-Python ----------------------------------- The goal is to optimize Bootstrapping in possibly various ways, @@ -159,81 +159,85 @@ executed directly, but goes though a code generator which produces the actual machine code. This layer is highly configurable. +Integration of Existing Technology +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + There are existing projects, notably Psyco_ and Stackless_, + which have been traditionally dependent on closely + following CPython's code base. + + Both will be rewritten as a meta-component that hooks into the + translator plus a dedicated run-time component (WP33_). + + As a side effect, after successful completion of the + PyPy project, they no longer need to exist as stand-alone + projects. + + +WP41_: Integration of _`Stackless` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Stackless Python has implemented high speed multitasking in + a single thread for C Python in two different ways: + Continuation passing and stack switching. Both ways of + task switching can be integrated into PyPy. Furthermore, + pickling of running programs has been implemented in + Stackless Python and should enable PyPy for load-balancing + between different machines. + WP42_: Several Object Implementations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Since the standard object space allows coexistence of different -implementations of the same type, we can develop -**several object implementations** . -We will develop heuristics to switch between different -implementations during execution. -The goal is to study the efficiency of different approaches, -with the possibility to change the default implementation -if favor of a different one, not known in CPython. -Some object layouts will further exist in parallel, if their -efficiency is highly application dependant.. + Since the standard object space allows coexistence of different + implementations of the same type, we can develop + **several object implementations** . + We will develop heuristics to switch between different + implementations during execution. + The goal is to study the efficiency of different approaches, + with the possibility to change the default implementation + if favor of a different one, not known in CPython. + Some object layouts will further exist in parallel, if their + efficiency is highly application dependant.. WP43_: Translator Optimisations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -It should be identified, which **optimisations** would benefit -from support from the translator. These are the optimisations -not easily available to CPython because they would require -large-scale code rewrites. -This includes design-considerations including the decision whether -to use reference counting together with garbage collection, -or whether to go for garbage collection, only. - -The meta-programming underlying WP43_ --- namely the work on the translator instead of on the resulting code -- is -what gives us the possibility of actually implementing several very -different schemes. The outcome of this effort is of course not unique, -but depends on the specific target of the optimisation. -There will be at least two efforts at the same time: - -- optimisation towards maximum performance of an integrated PyPy system + It should be identified, which **optimisations** would benefit + from support from the translator. These are the optimisations + not easily available to CPython because they would require + large-scale code rewrites. + This includes design-considerations including the decision whether + to use reference counting together with garbage collection, + or whether to go for garbage collection, only. + + The meta-programming underlying WP43_ + -- namely the work on the translator instead of on the resulting code -- is + what gives us the possibility of actually implementing several very + different schemes. The outcome of this effort is of course not unique, + but depends on the specific target of the optimisation. + There will be at least two efforts at the same time: + + - optimisation towards maximum performance of an integrated PyPy system + + - optimisation towards minimal code size for embedded systems. + -- optimisation towards minimal code size for embedded systems. - - -Integration of Existing Technology ------------------------------------ - -There are existing projects, notably Psyco and Stackless, -which have been traditionally dependent on closely -following CPython's code base. - -Both will be rewritten as a meta-component that hooks into the -translator plus a dedicated run-time component (WP33_). - -As a side effect, after successful completion of the -PyPy project, they no longer need to exist as stand-alone -projects. - -WP44_: Integration of Psyco +WP44_: Integration of _`Psyco` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Psyco provides techniques to increase the performance of C -Python by generating specialized machine code at run-time. -Developing C Psyco has proved that more flexibility would be of -paramount importance to the project. All prior knowledge of the -Psyco project will thus be integrated into PyPy, as Python and -RPython code, enabling yet more efficient optimisations and -allowing new processor architectures to be targeted. - -WP41_: Integration of Stackless -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Stackless Python has implemented high speed multitasking in -a single thread for C Python in two different ways: -Continuation passing and stack switching. Both ways of -task switching can be integrated into PyPy. Furthermore, -pickling of running programs has been implemented in -Stackless Python and should enable PyPy for load-balancing -between different machines. + Psyco provides techniques to increase the performance of C + Python by generating specialized machine code at run-time. + Developing C Psyco has proved that more flexibility would be of + paramount importance to the project. All prior knowledge of the + Psyco project will thus be integrated into PyPy, as Python and + RPython code, enabling yet more efficient optimisations and + allowing new processor architectures to be targeted. + +WP50_: Validation of PyPy +--------------------------- Supporting Embedded Devices ------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Embedded devices are often limited in processor speed and memory size, which either limits the power of software that @@ -247,27 +251,36 @@ WP51_: Porting PyPy to an Embedded Device ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Dependant from the actual processor architecture, PyPy's -code generator needs an extra platform specific support -module. Interfaces to necessary device drivers are needed -as well as it makes sense to develop a PyPy simulator for -the target platform. - + Dependant from the actual processor architecture, PyPy's + code generator needs an extra platform specific support + module. Interfaces to necessary device drivers are needed + as well as it makes sense to develop a PyPy simulator for + the target platform. + +WP52_: Evaluation whether to do a small OS in PyPy +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Implementing a small operating system in PyPy raises + some new questions and opens a new category of problems. + One of them is the possible need to write an IP stack + in PyPy, and a number of device drivers as well. + In this short task, we will figure out whether it is + feasible to carry on with the next task. WP53_: A small Operating System for an Embedded Device ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -*we are thinking of a printer here, but don't have facts yet* - -In extent to WP51_, it makes sense to write the whole -operating system in PyPy. We need a low-level extension -to the code generator that allows to access I/O ports. -Interrupts need to be supported as well, and primitive -access to persistant storage. -Ideally, we can create a single threaded PyPy OS with -a prioritized scheduler that runs the OS parts together -with multiple Python application. The latter depends on -WP41_ very much. + *we are thinking of a printer here, but don't have facts yet* + + In extent to WP51_, it makes sense to write the whole + operating system in PyPy. We need a low-level extension + to the code generator that allows to access I/O ports. + Interrupts need to be supported as well, and primitive + access to persistant storage. + Ideally, we can create a single threaded PyPy OS with + a prioritized scheduler that runs the OS parts together + with multiple Python application. The latter depends on + WP41_ very much. Numerical Applications @@ -344,4 +357,4 @@ do look for existing solutions which allow to automate this effort at least partially. -WP54_ WP55_ WP56_ WP10_ WP60_ WP70_ WP80_ +WP54_ WP55_ WP56_ Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Tue Sep 23 02:18:34 2003 @@ -21,7 +21,7 @@ ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP22_ Translation of CPython into regular Python 12 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- -WP20_ The PyPy Compiler +WP30_ The PyPy Compiler ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP31_ Translation of RPython 8 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- @@ -29,6 +29,8 @@ ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP33_ Low Level PyPy Runtime 6 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP40_ High-performance PyPy-Python +----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP41_ Integration of Stackless 9 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP42_ Several Object Implementations 12 @@ -37,6 +39,8 @@ ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP44_ Integration of Psyco 12 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP50_ Validation of PyPy +----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP51_ Porting PyPy to an Embedded Device 9 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP52_ Evaluation whether to do a small OS in PyPy 3 Copied: pypy/trunk/doc/funding/B6.wp40_high_performance.txt (from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp30_compiler.txt (original) +++ pypy/trunk/doc/funding/B6.wp40_high_performance.txt Tue Sep 23 02:18:34 2003 @@ -1,7 +1,7 @@ .. include:: crossreferences.asc -.. |title| replace:: The PyPy Compiler -.. |wp| replace:: WP30 +.. |title| replace:: High-performance PyPy-Python +.. |wp| replace:: M2 .. |start| replace:: 0 .. |p1| replace:: |e| .. |m1| replace:: |e| @@ -22,14 +22,18 @@ **Objectives** -Building the translator and Bootstrapping PyPy. +Optimizing the Bootstrapping in various ways. + +Exploring several Object Implementations. + +Folding Psyco and Stackless into PyPy. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -This working package consists of the tasks WP31_ WP32_ WP33_. +This working package consists of the tasks WP41_ WP42_ WP43_ WP44_. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -41,6 +45,6 @@ **Milestones and Expected Result** -- M2 +- M4 .. include:: wp-tableend.asc Copied: pypy/trunk/doc/funding/B6.wp50_validations.txt (from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp30_compiler.txt (original) +++ pypy/trunk/doc/funding/B6.wp50_validations.txt Tue Sep 23 02:18:34 2003 @@ -1,8 +1,8 @@ .. include:: crossreferences.asc -.. |title| replace:: The PyPy Compiler -.. |wp| replace:: WP30 -.. |start| replace:: 0 +.. |title| replace:: Validation of PyPy +.. |wp| replace:: WP50 +.. |start| replace:: M4/M3 .. |p1| replace:: |e| .. |m1| replace:: |e| .. |p2| replace:: |e| @@ -22,14 +22,16 @@ **Objectives** -Building the translator and Bootstrapping PyPy. +Validation of the PyPy concept by proving its applicability +in real applications. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -This working package consists of the tasks WP31_ WP32_ WP33_. +This working package consists of the tasks WP51_ WP52_ WP53_ WP54_ WP55_ WP56_ +or a subset thereof, dependent of feasability. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -41,6 +43,6 @@ **Milestones and Expected Result** -- M2 +- M6 M7 M8 M9 .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Tue Sep 23 02:18:34 2003 @@ -1,16 +1,17 @@ .. _WP10: B6.wp10_management.html -.. _WP18: -.. _WP20: +.. _WP20: B6.wp20_interpreter.html .. _WP21: B6.wp21_interpreter_completion.html .. _WP22: B6.wp22_cpython_to_python.html -.. _WP30: +.. _WP30: B6.wp30_compiler.html .. _WP31: B6.wp31_translation_of_rpython.html .. _WP32: B6.wp32_bootstrap_redesign.html .. _WP33: B6.wp33_bootstrap_runtime.html +.. _WP40: B6.wp40_high_performance.html .. _WP41: B6.wp41_integrate_stackless.html .. _WP42: B6.wp42_several_obj_impl.html .. _WP43: B6.wp43_translator_optimisations.html .. _WP44: B6.wp44_integrate_psyco.html +.. _WP50: B6.wp50_validations.html .. _WP51: B6.wp51_embedded_device.html .. _WP52: B6.wp52_small_os_pre.html .. _WP53: B6.wp53_small_os.html @@ -20,6 +21,6 @@ .. _WP60: .. _WP70: .. _WP80: -.. attention:: **9 files missing: WP18 WP20 WP30 WP54 WP55 WP56 WP60 WP70 WP80** +.. attention:: **6 files missing: WP54 WP55 WP56 WP60 WP70 WP80** .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file From tismer at codespeak.net Tue Sep 23 02:20:09 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 23 Sep 2003 02:20:09 +0200 (MEST) Subject: [pypy-svn] rev 1406 - pypy/trunk/doc/funding Message-ID: <20030923002009.48F3C5ACAE@thoth.codespeak.net> Author: tismer Date: Tue Sep 23 02:20:08 2003 New Revision: 1406 Modified: pypy/trunk/doc/funding/B6.wp50_validations.txt Log: typo Modified: pypy/trunk/doc/funding/B6.wp50_validations.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp50_validations.txt (original) +++ pypy/trunk/doc/funding/B6.wp50_validations.txt Tue Sep 23 02:20:08 2003 @@ -31,7 +31,7 @@ **Description of work** This working package consists of the tasks WP51_ WP52_ WP53_ WP54_ WP55_ WP56_ -or a subset thereof, dependent of feasability. +or a subset thereof, dependent of feasibility. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc From lac at codespeak.net Tue Sep 23 11:40:30 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Tue, 23 Sep 2003 11:40:30 +0200 (MEST) Subject: [pypy-svn] rev 1407 - pypy/trunk/doc/funding Message-ID: <20030923094030.EFC605ACAE@thoth.codespeak.net> Author: lac Date: Tue Sep 23 11:40:30 2003 New Revision: 1407 Added: pypy/trunk/doc/funding/STREP-template.txt Log: Here is a text version of the STREP template. I thought I had checked this in before. Sorry about that. Added: pypy/trunk/doc/funding/STREP-template.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/STREP-template.txt Tue Sep 23 11:40:30 2003 @@ -0,0 +1,1084 @@ +Please note that these documents and this Template is aimed +specifically at IST Call 2 fixed deadline. It is of particular +importance to understand the Evaluation Criteria to be used on a STREP +so I reproduce them here (Note overall threshold of 21 points out of +30 for STREP) ? + + +0.1 Relevance + +(Threshold: 3 out of 5) Weight 1 + +What is the extent to which the proposal addresses the scientific, +technical, socio-economic and policy objectives of the Workprogramme +in the areas open in the call ? + + +0.2 Potential impact +(Threshold: 3 out of 5) Weight 1 + +The extent to which + +? the proposed project is likely to have an impact in terms of + strategic impact on reinforcing competitiveness or solving societal + problems + +? exploitation and dissemination plans are adequate to ensure optimum use of + project results + +? the proposal demonstrates a clear added-value in carrying out the + work at a European level and takes account of research activities + at a national level and under European initiatives (e.g. Eureka) ? + + +0.3 Scientific and technical excellence +(Threshold: 4 out of 5) Weight 1 + +The extent to which +? the proposal has clearly defined and well focused objectives + +? the objectives represent clear progress on the state-of-the-art + +? the proposed S&T approach is likely to enable the project to achieve + its objectives in research and innovation + + +0.4 Quality of the consortium +(Threshold: 3 out of 5) Weight 1 + +The extent to which + +? the participants collectively constitute a consortium of high + quality + +? the participants are well suited and committed to their tasks + assigned to them + +? there is good complementarity between the participants + +? the real opportunity to involve SMEs been adequately addressed + +0.5 Quality of the management +(Threshold: 3 out of 5) Weight 1 + +The extent to which + +? the project management of high quality + +? there is a satisfactory plan for the management of knowledge, IPR + and innovation-related activities + +0.6 Mobilisation of resources +(Threshold: 3 out of 5) Weight 1 + +The extent to which + +? the project forsees the resources (personnel, equipment, finance...) + necessary for success + +? the resources are convincingly integrated to form a coherent project + +? the overall financial plan for the project adequate + + +0.7 Horizontal Issues (comment if applicable) + +? If there gender issues associated with the subject of the proposal, + have they been adequately taken into account? + +? Have the applicants identified the potential ethical and/or safety + aspects of the proposed research regarding its objectives, the + methodology and the possible implications of the results? + +? To what extent does the proposal demonstrate a readiness to engage + with actors beyond the research community and the public as a + whole, to help spread awareness and knowledge and to explore + the wider societal implications of the proposed work? + +? Have the synergies with education at all levels been clearly set out? + +? If third country participation is envisaged in the proposal, is it + well justified and the participation well integrated in the activities? + + +TEMPLATE STARTS HERE: + +Proposal full title: + + + + + +Proposal acronym: + + +Date of preparation: + + +Type of instrument: Specific Targeted Research Project + + +List of participants (Co-ordinator first): + + +Participant no. Participant name Participant short name +1 (coordinator) +2 +3 +4 +etc + + + + +Co-ordinator Organisation name: +Point of Contact name: +Point of Contact email: +Point of Contact telephone: + + +Contents + + +Contents + +Proposal summary page +B.1 Scientific & technological objectives of the project & state of the art +B.1.1 Problem to be solved +B.1.2 Quantified specific objectives +B.1.3 Current State of the Art +B.1.4 Beyond the State of the Art +B.2 Relevance to the objectives of the IST Priority +B.3 Potential impact +B.3.1 Contributions to Standards +B.3.2 Strategic impact +B.3.2.1 Potential Impact on Industrial/Research Sector +B.3.2.2 Balance of Trade +B.3.2.3 European Dimension and AddedValue +B.3.3 Innovation Related Activities +B.3.3.1 Management of Knowledge Produced +B.3.3.2 Protection of Intellectual Property +B.3.4 Dissemination +B.3.5 Exploitation +B.4 The consortium and project resources +Description of the consortium (1 page) +B.4.1 Sub-contracting +B.4.2 Other countries +B.4.3 Description of the participants +B.4.4 Quality of partnership, involvement of users and SMEs +B.4.5 Resources to be deployed +B.4.6 Overall FinancialPlan +B.4.7 STREP Project Effort Form +B.5 Project management +B.5.1 Project Manager +B.5.2 Management Structure +B.5.3 Co-ordinator +B.5.4 Project Meetings +B.5.5 Quality procedures +B.5.6 Communication and Reporting +B.5.7 Consortium Agreement +B.5.8 Management of Knowledge and Intellectual Property +B.6 Detailed Implementation Plan +B.6.1 Introduction +B.6.2 Research and Technological Aspects and Options +B.6.3 Risks in the Project and Steps to Minimise +B.6.4 Project planning and time table; (Gantt chart) +B.6.5 Graphical presentation of the project's components; (PERT diagram) +B.6.6 Deliverables list (full duration of project) +B.6.7 Workpackage description (full duration of project) +B.7 Other issues +B.7.1 Ethical Considerations +B.7.2 Gender Issues +B.7.3 Safety Issues +B.7.4 Conservation Regulations +B.7.5 Other Policy related Issues + + + Proposal summary page + + + Proposal full title: + + + Proposal acronym: + + Strategic Objectives: + + List here the Strategic Objectives in this call that are + addressed by this proposal in priority order with the most + important one first if there is more than one. Remember the + proposal must be specifically directed at this first one and will + normally be evaluated by that team. + + + Proposal abstract: + + Proposal abstract copied from Part A1 to be added here (Maximum + 2,000 characters) + + +B.1 Scientific & technological objectives of the project & state of the art + +Describe in detail the proposed project's S&T objectives. The +objectives should be those achievable within the project, not through +subsequent development, and should be stated in a measurable and +verifiable form. The progress of the project work will be measured +against these goals in later reviews and assessments. Describe the +state-of-the-art in the area concerned and how the proposed project +will enhance the state-of- the-art in that area. + +(Recommended length ? three pages) + + +This section describes the scientific/technological objectives of the +proposal in a measurable and verifiable form. The progress of the +project work will be measured against these objectives in later +reviews and assessments. + +Although this section talks about objectives, I strongly recommend you +only have a single high level objective to ensure project focus. You +could identify spin off benefits or subsidiary objectives but try to +ensure you start off by identifying a single objective. + +B.1.1 Problem to be solved + +It is also a good idea to clearly identify the problem to be solved + + +B.1.2 Quantified specific objectives + +Try to quantify statements whereever made, especially technical targets. + +B.1.3 Current State of the Art + +Demonstrate awareness of the current state-of-the-art and differences +between this proposal and any similar existing or previous projects. + +B.1.4 Beyond the State of the Art + +Show the degree of originality, innovation and promise of progress +beyond it. Strike an appropriate balance in the level of risk +associated with the project compared to its potential benefits - high +risk may be acceptable in return for high benefits. Avoid very large +or unacceptable levels of risk. + +B.2 Relevance to the objectives of the IST Priority + +Describe the manner in which the proposed project's goals address the +scientific, technical, wider societal and policy objectives of the IST +Priority in the areas concerned. (Recommended length ? three pages) + +Information for this section comes from several main sources - + + + 1. Each Workprogram and the Commission specific program documents + identify and address the policy needs to a certain extent. + The introductory sections of the Workprogram for IST 2003/4 + contains good reference material. + + + 2. Via the Europa web site, http://europa.eu.int there is + information on all EU policies and they can be identified and + downloaded from there. For example we have the following ? + Policies - Access by subject to legal instruments in force, + legislative activity in progress, implementation of common + policies, EU grants and loans, statistics and publications. + + + 3. There is also good material under eEurope initiatives and at + the ISPO (information Society Project Office) site. + + +You must also address where appropriate ERA related issues such as +relationships to any Eureka activities, (such as commonality of partners) +or relationships to national research programs. + +B.3 Potential impact + +Describe the strategic impact of the proposed project, for example in +reinforcing competitiveness or on solving societal problems. Describe +the innovation-related activities. Describe the exploitation and/or +dissemination plans which are foreseen to ensure use of the project +results. Describe the added-value in carrying out the work at a +European level. Indicate what account is taken of other national or +international research activities. (Recommended length ? four pages, +including one for "Contribution to Standards") + +This section should include the description of plans for the +dissemination and/or exploitation of the results for the consortium as +a whole and for the individual participants in concrete terms, for +example by describing the dissemination and/or exploitation +strategies, the user groups to be involved and how they will be +involved, the tools and/or means to be used to disseminate the results +and the strategic impact of the proposed project in terms of +improvement of competitiveness or creation of market opportunities for +the participants. + + +B.3.1 Contributions to Standards + +Describe contributions to national or international standards which +may be made by the project, if any. Identify specific standards +bodies and committees with an emphasis on European bodies such as +CEN/CENELEC, ETSI, ECMA etc and International bodies such as ISO JTC +as well as Industrial Informal Standards Bodies. It is better to avoid +purely US bodies such as ANSI if possible. + +B.3.2 Strategic impact + +Show that the project will have a significant strategic impact and not +merely satisfy intellectual curiosity. + +B.3.2.1 Potential Impact on Industrial/Research Sector + +Convincingly describe the impact it will have on its +industry/commerce/research sector and how it will improve European +competitiveness and assist in market development where +applicable. Demonstrate a clear view of the market segment(s) and +market needs, which it addresses. + +B.3.2.2 Balance of Trade + +Showing how it will contribute to an improvement in the balance of +trade is also an important aspect. + +B.3.2.3 European Dimension and Added Value + +Does the proposal address European issues or merely address a national +issue? Assess the extent to which the project is required by the EU as +a whole? Does the proposal identify and describe interdependencies or +links with other national or international activities? + +What are the European/international dimension in the execution of the +work, for example is there a need to establish a critical mass in +human or financial terms, or does adequate resources and expertise not +exist in individual countries? Will the impact of carrying out the +work at the European level be greater than the sum of the impacts of +national projects? + +B.3.3 Innovation Related Activities + +Please note that "innovation" as used here is not technological +innovation. The commission in their wisdom have redefined the term to +mean the following (within a STREP) ? "Specific targeted projects +should also include innovation-related activities, in particular with +respect to the management of the knowledge produced and the protection +of intellectual property." + +Note that management aspects of this are addressed under B.5.8. + +B.3.3.1 Management of Knowledge Produced + +Describe this activity. + +This is the ongoing identification, tracking and registration of +knowledge as it is produced within the project. It is particularly +concerned with the deciisons on ownership of IP and should be covered +in the Consortium Agreement. IP, not forseen or falling outside of the +agreement will require special treatment and may even require +modification to the agreement. The process should be identified and +covered in B.5.8. + + +B.3.3.2 Protection of Intellectual Property + +Describe this activity. + +The owner of knowledge should provide adequate and effective +protection for knowledge that is capable of industrial or commercial +application. + +The Commission may adopt protective measures when it considers it +necessary to protect knowledge in a particular country, and when such +protection has not been applied for or has been waived. + +Participants may publish information on the knowledge acquired under +the project, provided this does not affect the protection of that +knowledge. + +Basically this section should define how the project will protect the +IP produced by it, identifying the process and responsibility. It +should be cross referenced from the management section in B.5.8. + +B.3.4 Dissemination + +The project is not funded merely to benefit the participating +organisations. Show that results will be adequately disseminated so +as to support general European scientific or technological progress. +Define specific plans for dissemination, with explicit commitments by +participants. i.e. Papers at European conferences, web site, +publication of papers etc. + +B.3.5 Exploitation + +Emphasise the usefulness and range of applications, which might arise +from the project. Explain the partners' capability to exploit the +results of the project and detail how you foresee doing this in a +credible way. Refer to the draft Consortium Agreement with respect to +exploitation rights within the consortium. This is particularly +important. Be specific and quantify things such as accessible market +etc. + +B.4 The consortium and project resources + +Describe the role of the participants and the specific skills of each +of them. Show how the participants are suited and committed to the +tasks assigned to them; show the complementarity between participants. +Describe how the opportunity of involving SMEs has been addressed. +Describe the resources, human and material, that will be deployed for +the implementation of the project. Include a STREP Project Effort +Form, as shown below, covering the full duration of the project. +Demonstrate how the project will mobilise the critical mass of +resources (personnel, equipment, finance...) necessary for success; +and show that the overall financial plan for the project is +adequate. (Recommended length ? seven pages, including one for B.4.1 +and one for B.4.2) + + +Description of the consortium (1 page) + +Short description of one page of the consortium stating who the +participants are, what their roles and functions in the consortium +are, and how they complement each other. + + + Participant Country Role Function Note + +Either add in here or put under Notes column, Complementarity as appropriate. + + + B.4.1 Sub-contracting + +If any part of the work is foreseen to be sub-contracted by the +participant responsible for it, describe the work involved and +explain why a sub-contract approach has been chosen for it. Do not +sub-contract R&D. Remember if a company sub-contracts some work +they will normally have to pay 100% of their costs (potentially +with profit) and will normally only get 50% back. Do not +subcontract project management. + +B.4.2 Other countries + +If one or more of the participants is based outside of the EU Member +and Associated states, explain in terms of the project's objectives +why this/these participants have been included, describe the level of +importance of their contribution to the project. + +Where a non-EU/Associated State participation is involved, demonstrate +it is in conformity with the interest of the Community, and it is +of substantial added value for implementing all or part of the +specific programme. + +B.4.3 Description of the participants + +Short description of the participating organisations including: + +? The expertise and experience of the organisation, + +? Short CVs of the key persons to be involved indicating relevant + experience, expertise and involvement in other EC projects. (Each + CV no more than 10 lines) Remember that you cannot contractually + commit to these specific persons being assigned so you should state + that these named people or their equivalent will be assigned + +The CV of the nominated Project Manager is of particular importance. +You have to show that he has experience of successful international +project management. Emphasise this aspect. + + +B.4.4 Quality of partnership, involvement of users and SMEs + +Show that the organisations involved in the consortium are capable of +doing the tasks allotted to them. Ensure there is no unnecessary +redundancy and duplication in the make-up of the consortium. Ensure +the consortium does not lack an obvious participant with some +essential skill or resource. + +Describe how involvement of SMEs has been addressed. + +B.4.5 Resources to be deployed + +Show how the project will mobilise the critical mass of resources +(personnel, equipment, finance...) necessary for success. + + +B.4.6 Overall Financial Plan + +Show that the overall financial plan for the project is adequate. + + +In B4.7 (below) do not identify any activities as "demonstration" +i.e. leave that section blank. + +B.4.7 STREP Project Effort Form + +Full duration of project (insert person-months for activities in which +partners are involved) + +Project acronym - + + + Short Names + Partner 1 + Partner 2 + Partner 3 + Partner 4 + Partner 5 + etc + TOTAL PARTNERS + + + WP name + WP name + WP name + etc + Total research/innovation + + + + Demonstration activities + WP name + WP name + WP name + etc + Total demonstration + + + + Consortium management activities + WP name + WP name + WP name + etc + Total management + + + TOTAL ACTIVITIES + + + +B.5 Project management + +Describe the organisation, management and decision making structures +of the project. Describe the plan for the management of knowledge, of +intellectual property and of other innovation-related activities +arising in the project. (Recommended length ? three pages) + +This section should describe how the proposed project will be managed, +the decision making structures to be applied, the communication flow +within the consortium and the quality assurance measures which will be +implemented, and how legal and ethical obligations will be met. + +Quality of the management + +Make it clear how progress will be monitored and how an effective +management structure will be put in place, with agreed lines of +communication and responsibility. Describe how corrective actions +will be initiated and how conflicts will be resolved. + + +B.5.1 Project Manager + +Every project must have a Project Manager. He will be responsible for +the Management of the Project and execution of the contract. He +is appointed by the Co-ordinator and chairs the Management +Meetings. He approves all outputs and reports, is the prime +external interface and also may be the Technical Director (if one +is deemed necessary). + + +There is some confusion as to the role of the Project Manager. This is +not an administrative chore. A Project Manager will require some +administrative support, but that is far from the essence of the job. +The administrative functions such as status tracking, financial +reporting, change control and project library maintenance are really a +minor part of the overall job. + +Project management activities + +Specific targeted research projects will also include an overall +management structure. Over and above the technical management of +individual work packages, an appropriate management framework linking +together all the project components and maintaining communications +with the Commission will be needed. + +Project management will include: + +? co-ordination of the technical activities of the project; + +? the overall legal, contractual, ethical, financial and administrative + management of the project; + +? preparing, updating and managing the consortium agreement between + the participants; + +? co-ordination of knowledge management and other innovation-related + activities; + +? overseeing the promotion of gender equality in the project; + +? overseeing science and society issues, related to the research + activities conducted within the project; + +? obtaining audit certificates (as and when required) by each of the + participants; + +? bank guarantees for SMEs (if applicable). + +Successful Project Management of a FrameWorkprogram Project +requires various skills and knowledge. In my view it requires a +person with the following attributes ? + + ? Good appreciation of the relevant business area + + ? Participation in a previous Framework project + + ? Knowledge of Framework procedures + + ? Good interpersonal skills + + ? Well organised and systematic in own work + + ? Good knowledge of ISO 9001 + + ? Good knowledge of English + + ? Some knowledge of project technical area + + ? Some knowledge of financial management + +Project Management is a combination of all of the above skills. Extra +strength in some areas could compensate for weakness in others. +Remember this function includes legal responsibility aspects and thus +keeping of good records is essential. Any telephone calls and +agreements, especially with the Project Officer should be minuted +and/or confirmed in writing, at least by email. + + + B.5.2 Management Structure + +As this type of project is essentially the same as the previous RTD +project, I would maintain the traditional structure as follows - + +(see the PDF) + +For smaller projects and depending on the technical abilities of the +company representatives, it is sometimes possible and more effective +to combine the Management and Technical Boards although they must +continue to deal with both aspects. + + + B.5.3 Co-ordinator + +This is the principal interface to the Commission - both during +proposal and project stages and is responsible for submitting the +proposal; also conducts the contract negotiation. During project +appoints the Project Manager, submits all reports, normally handles +the financial statements and payments, chairs the Project Management +Board and has overall control of the project. + +Note that the Coordinator normally appoints the project manager. If, +under rare circumstances it is decided to split the management into +Managerial and Technical parts, then a Technical Director is also +appointed, although he is not necessarily from the Coordinating +Organisation. + + +A distinction between Financial Coordinator and Scientific +Coordinator is no longer recognised in the contract. Any +distinctions of role between the partners must be embodied in +the Consortium Agreement. + +It is also normal for the Technical Director to chair a Technical +Board that would consist of the Key Technical staff - one per partner +normally or Work package leaders. Such meetings are normally held +adjacent to Management Board meetings. + + +The Technical Director would also sit on the Management Board ex +officio and the Project Manager on the Technical Board. + +It is normal to have a structure as shown above under B.5.2 (note the +Technical Board could be combined into the Management board). + +I am trying here to give a flavour of the type of words and content to +use. It is not exhaustive. Do not copy them verbatim. + + +A Management Board will be created that will be responsible for the +successful completion of the project and the exploition of its +result. It will be chaired by the appointed Project Manager and will +consist of a senior representative of each partner. + +Decisions regarding the project will be made by vote with each partner +having a single vote. In cases of a tie, the project manager will +have a casting vote. + + The role of the Management Board will include the following - + + ? Management of resources in order to meet schedules and goals + ? To ensure the quality management of the project + ? Tracking of costs related to budget + ? Resolution of conflicts + ? Creation of technology implementation plan and its updating + ? Ensuring compliance with legal and ethical obligations + +Explain the role/responsibilities of Technical Board if constituted +and state that it reports to the Management Board. + +B.5.4 Project Meetings + +The Management Board will meet at the start of the project and (2/3/4) +times per year or on an ad hoc basis as requested. The meetings will +normally be scheduled to rotate between the principal contractors +home base. + +Add info on Technical meetings as required. + + +B.5.5 Quality procedures + +The project manager will circulate a draft Quality Management plan +forthe project prior to first Project Meeting and and then present it +for approval at the first Meeting. + + It will contain as a minimum, procedures for - + + ? Document procedures, standards and control + ? Issue control for documents + ? Reporting procedures, frequency and format + ? Communication procedures + ? Corrective actions + ? Exception control + ? Conflict resolution + ? Meeting draft agenda + ? Format of meeting minutes + ? Tracking system for actions + ? Specific responsibilities within the project + +B.5.6 Communication and Reporting. + +Amplify here specific policies on this subject ie use of email or +communication via web site management page, telephones, video +conferencing, frequency etc + +B.5.7 Consortium Agreement + +A Consortium Agreement between the partners is now mandatory and must +be signed before any participant start work on the project. The +Project Manager must be responsible for this activity. + + +B.5.8 Management of Knowledge and Intellectual Property + +The rules regarding the protection dissemination and use of knowledge +have been simplified and a larger flexibility is granted to the +participants: + + ? rules are identical for all participants; + ? rules concentrate on the principles and provisions considered + necessary for an efficient cooperation and the appropriate use and + dissemination of the results; + ? participants may define among themselves the arrangements that + fit them the best within the framework provided in the model + contract. + + + See also comments under B.3.3.1 and B.3.3.2 and address here. + +B.6 Detailed Implementation Plan + +This section describes in detail the work planned to achieve the +objectives for the full duration of the of the proposed project. The +recommended length, excluding the forms specified below, is up to 15 +pages. An introduction should explain the structure of this workplan +plan and how the plan will lead the participants to achieve the +objectives. The workplan should be broken down according to types of +activities: Research, technological development and innovation related +activities, demonstration activities and project management +activities. + +I strongly suggest you do not identify any "demonstration activities". +Do not use the word "demonstration" anywhere. Anything you consider +may fall under this heading refer to as "trial", "validation" +or"system test" or something similar. This simplifies the proposal +and more importantly avoids, or at least may mitigate against, being +reduced to 35% funding. + +It should identify significant risks, and contingency plans for +these. The plan must for each type of activity be broken down into +workpackages (WPs) which should follow the logical phases of the +project, and include management of the project and assessment of +progress and results. + +Essential elements of the plan are: +a) Detailed Implementation plan introduction ? explaining the structure + of this plan and the overall methodology used to achieve the objectives. + +b) Work planning, showing the timing of the different WPs and their + components (Gantt chart or similar). Ensure that the workplan is + appropriate, clear, consistent, and efficient without serious omissions. + Ensure a clear working schedule is laid out, with clearly identified + review points. + +c) Graphical presentation of the components showing their + interdependencies (Pert diagram or similar) + +d) Detailed work description broken down into workpackages: + + Workpackage list (use Workpackage list form below); + Deliverables list (use Deliverables list form below); + Description of each workpackage (use Workpackage description + form below, one per workpackage): + +Note: The number of workpackages used must be appropriate to the +complexity of the work and the overall value of the proposed project. +Each workpackage should be a major sub-division of the proposed +project and should also have a verifiable end-point (normally a +deliverable or an important milestone in the overall project). The +planning should be sufficiently detailed to justify the proposed +effort and allow progress monitoring by the Commission ? the +day-to-day management of the project by the consortium may require a +more detailed plan. + +Ensure the manpower effort for each partner and workpackage is +credible, without seriously under/over estimating. Ensure the other +resources required are also credible. Make sure there are no +resources required which appear not to be foreseen. + +It is normal to assign WP1 to Project Management. This would include +all general activities such as Board Meetings etc. It is normal for +the Coordinator to have the majority of this with small amounts for +each of the partners to cover their participation in the general +meetings. + +A useful metric is that Project Management is usually expected to +average around 10% of the project effort. Any significant deviation +should be justified. + +Allow the evaluators to make an overall assessment of the quality of +the research proposed to be carried out, from a scientific and +technical point of view. + +Dont forget to have a Workpackage or Task related to disseminating the +results. + +B.6.1 Introduction + +Explain the structure of the workplan and the overall methodology used +to achieve the objectives + +B.6.2 Research and Technological Aspects and Options + +Explain the adequacy of the chosen approach, methodology and work plan +for achieving the objective(s). As appropriate, you may describe +overall systems design. Schematics can help to illustrate this +section. + +B.6.3 Risks in the Project and Steps to Minimise + +Be frank about potential risks. They may be technical, organisational, +business related etc. For each risk say how you will monitor it, minimise +it or even what the contingency or backup plan is. + +B.6.4 Project planning and time table; (Gantt chart) + +It is normal to identify key events on the Gantt chart such as Board +Meetings (Bx) and /or other specific events (Evx, MTR = Mid term +Report, etc) + +B.6.5 Graphical presentation of the project's components; (PERT diagram) + +Broken down into workpackages and showing constraints and events with +the critical path identified: + +It is usually a good idea to identify major milestones on this PERT +diagram as appropriate. + +B.6.4 Workpackage list (full duration of project) + B.6.7 Workpackage description (full duration of project) + + One page description of each workpackage (use form below): + + Note: The number of workpackages used must be appropriate to the + complexity of the work and the overall value of the proposed + project. Each workpackage should be a major sub- division of + the proposed project and should also have a verifiable end-point + (normally a deliverable or an important milestone in the overall + workplan). + + + In medium to large projects it is also good practice to further + divide each Workpackage into Tasks. Each with a leader and each + ending in deliverable. + + + The level of detail expected under the Fifth Framework is + unclear. It is hoped to have CEC advice shortly. + + + Do not plan long running activities (ie more than a year) + without an interim deliverable. + + + Numbering scheme: + + In the past it was normal to adopt the following type of scheme + (note this is an illustration only) - + + + WP Task Deliverables + + + WP1 + T1.1 D1.1.1, D1.1.2 + T1.2 D1.2.1 + WP2 + T2.1 D2.1.1 + T2.2 D2.2.1, D2.2.2 + T2.3 D2.3.1 + WP3 + T3.1 D3.1.1, D3.1.2, D3.1.3 + WP4 + T4.1 D4.1.1 + T4.2 D4.2.1, D4.2.2 + T4.3 D4.3.1 + T4.4 D4.4.1, D4.4.2, D4.4.3, D4.4.4 + + + This type of numbering allows deliverables to be related to + Workpackages and Tasks and thus permits simpler tracking. + + + Project Management + + Workpackage number WP1 Start date or starting event: + Participant id + Person-months per participant: + + + Objectives + + + Description of work + + Deliverables + + Milestones and expected result + + + + + + Milestones are control points at which decisions are + needed; for example concerning which of several + technologies will be adopted as the basis for the next + phase of the project. + + + Workpackage number WP2 Start date or starting event: + Participant id + Person-months per participant: + + Objectives + + + Description of work + + + Deliverables + + + Milestones and expected result + +B.7 Other issues + +If there are ethical or gender issues associated with the subject of +the proposal, show they have been adequately taken into account - +indicate which national and international regulations are applicable +and explain how they will be respected. Explore potential ethical +aspects of the implementation of project results. Include the Ethical +issues form given below. See Annexes 3 and four of Proposers Guide +for more information on Ethical Rules and Gender Dimension. + +Are there other EC-policy related issues, and are they taken into +account? Demonstrate a readiness to engage with actors beyond the +research to help spread awareness and knowledge and to explore the +wider societal implications of the proposed work; if relevant set out +synergies with education at all levels. + +(No recommended length ? depends on the number of such other issues which the project involves). + +B.7.1 Ethical Considerations + + + A. Proposers are requested to fill in the following table + + Does your proposed research raise sensitive ethical YES NO + questions related to: + + Human beings + + + Human biological samples + + + Personal data (whether identified by name or not) + + + Genetic information + + + Animals + + + B. Proposers are requested to confirm that the proposed research does + not involve: + + + Research activity aimed at human cloning for reproductive purposes, + + Research activity intended to modify the genetic heritage of + human beings which could make such changes heritable12 + + Research activity intended to create human embryos solely for + the purpose of research or for the purpose of stem cell + procurement, including by means of somatic cell nuclear + transfer; + + Research involving the use of human embryos or embryonic stem + cells with the exception of banked or isolated human embryonic + stem cells in culture13 + + + + 12 Research relating to cancer treatment of the gonads can be financed + + 13 Applicants should note that the Council and the Commission + have agreed that detailed implementing provisions concerning + research activities involving the use of human embryos and + human embryonic stem cells which may be funded under the 6th + Framework Programme shall be established by 31 December 2003. + The Commission has stated that, during that + Confirmation : the proposed research involves none + of the issues listed in section B + + + + Further information on ethics requirements and rules are given + at the science and ethics website at + http://europa.eu.int/comm/research/science-society/ethics/ethics_en.html. + + + + B.7.2 Gender Issues + + + B.7.3 Safety Issues + + + B.7.4 Conservation Regulations + + + + B.7.5 Other Policy related Issues From tismer at codespeak.net Tue Sep 23 11:52:58 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 23 Sep 2003 11:52:58 +0200 (MEST) Subject: [pypy-svn] rev 1408 - pypy/trunk/doc/funding Message-ID: <20030923095258.1415D5B658@thoth.codespeak.net> Author: tismer Date: Tue Sep 23 11:52:57 2003 New Revision: 1408 Added: pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt (contents, props changed) - copied, changed from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt pypy/trunk/doc/funding/B6.wp55_numerical_package.txt (contents, props changed) - copied, changed from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt pypy/trunk/doc/funding/B6.wp56_load_balancing.txt (contents, props changed) - copied, changed from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt pypy/trunk/doc/funding/B6.wp60_documentation.txt (contents, props changed) - copied, changed from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt pypy/trunk/doc/funding/B6.wp70_maintenance.txt (contents, props changed) - copied, changed from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt pypy/trunk/doc/funding/B6.wp80_synchronisation.txt (contents, props changed) - copied, changed from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp10_management.txt pypy/trunk/doc/funding/crossreferences.asc Log: All working packages so far exist. Some text needs to be added to 54-56 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 23 11:52:57 2003 @@ -236,6 +236,9 @@ WP50_: Validation of PyPy --------------------------- +The goal is to provide a validation of the advancements of +the state of the art and the real world applicability of these. + Supporting Embedded Devices ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -336,6 +339,9 @@ as well as the preparation of papers for presentation on various congresses. +In extent, there is an ongoing information flow to external +communities, like the Python Business Forum (PBF) and the Python +developers list (python-dev at python.org). WP70_: Maintenance of Tools ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -357,4 +363,4 @@ do look for existing solutions which allow to automate this effort at least partially. -WP54_ WP55_ WP56_ + Modified: pypy/trunk/doc/funding/B6.wp10_management.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp10_management.txt (original) +++ pypy/trunk/doc/funding/B6.wp10_management.txt Tue Sep 23 11:52:57 2003 @@ -41,6 +41,11 @@ **Deliverables** +- Collection and monitoring of reports +- Report to EU +- Organise meetings +- Create and maintain internal web site + .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc Copied: pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt (from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp30_compiler.txt (original) +++ pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt Tue Sep 23 11:52:57 2003 @@ -1,14 +1,14 @@ .. include:: crossreferences.asc -.. |title| replace:: The PyPy Compiler -.. |wp| replace:: WP30 -.. |start| replace:: 0 -.. |p1| replace:: |e| -.. |m1| replace:: |e| -.. |p2| replace:: |e| -.. |m2| replace:: |e| -.. |p3| replace:: |e| -.. |m3| replace:: |e| +.. |title| replace:: Code Generator for SIMD Architecture +.. |wp| replace:: WP54 +.. |start| replace:: M4 +.. |p1| replace:: X +.. |m1| replace:: 3 +.. |p2| replace:: Y +.. |m2| replace:: 3 +.. |p3| replace:: Z +.. |m3| replace:: 3 .. |p4| replace:: |e| .. |m4| replace:: |e| .. |p5| replace:: |e| @@ -22,14 +22,16 @@ **Objectives** -Building the translator and Bootstrapping PyPy. +Extending PyPy towards general support of Single Instruction Multiple +Data (SIMD) architectures. +Building a concrete implementation for one choosen SIMD architecture. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -This working package consists of the tasks WP31_ WP32_ WP33_. +*FIXME: write that* .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -41,6 +43,6 @@ **Milestones and Expected Result** -- M2 +- Extended optimisation towards parallel numerics. .. include:: wp-tableend.asc Copied: pypy/trunk/doc/funding/B6.wp55_numerical_package.txt (from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp30_compiler.txt (original) +++ pypy/trunk/doc/funding/B6.wp55_numerical_package.txt Tue Sep 23 11:52:57 2003 @@ -1,12 +1,12 @@ .. include:: crossreferences.asc -.. |title| replace:: The PyPy Compiler -.. |wp| replace:: WP30 -.. |start| replace:: 0 -.. |p1| replace:: |e| -.. |m1| replace:: |e| -.. |p2| replace:: |e| -.. |m2| replace:: |e| +.. |title| replace:: Enhanced Numerical Package +.. |wp| replace:: WP55 +.. |start| replace:: WP54 +.. |p1| replace:: X +.. |m1| replace:: 3 +.. |p2| replace:: Y +.. |m2| replace:: 3 .. |p3| replace:: |e| .. |m3| replace:: |e| .. |p4| replace:: |e| @@ -22,25 +22,31 @@ **Objectives** -Building the translator and Bootstrapping PyPy. +Building an enhanced numerical package on top of the SIMD extensions of PyPy. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -This working package consists of the tasks WP31_ WP32_ WP33_. +*FIXME: write this* .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Deliverables** +- Enhanced numerical package + .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Milestones and Expected Result** -- M2 +- M8 + +- We expect to at least reach the efficiency of the existing numerical + packages. The target is to outperform these and to get a peformance that + is comparable to the efficiency of optimised parallel C libraries. .. include:: wp-tableend.asc Copied: pypy/trunk/doc/funding/B6.wp56_load_balancing.txt (from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp30_compiler.txt (original) +++ pypy/trunk/doc/funding/B6.wp56_load_balancing.txt Tue Sep 23 11:52:57 2003 @@ -1,14 +1,14 @@ .. include:: crossreferences.asc -.. |title| replace:: The PyPy Compiler -.. |wp| replace:: WP30 -.. |start| replace:: 0 -.. |p1| replace:: |e| -.. |m1| replace:: |e| -.. |p2| replace:: |e| -.. |m2| replace:: |e| -.. |p3| replace:: |e| -.. |m3| replace:: |e| +.. |title| replace:: Load Balancing in a Multi-Processor environment +.. |wp| replace:: WP56 +.. |start| replace:: M3, M4 +.. |p1| replace:: X +.. |m1| replace:: 3 +.. |p2| replace:: Y +.. |m2| replace:: 3 +.. |p3| replace:: Z +.. |m3| replace:: 3 .. |p4| replace:: |e| .. |m4| replace:: |e| .. |p5| replace:: |e| @@ -22,14 +22,20 @@ **Objectives** -Building the translator and Bootstrapping PyPy. +Implement load balancing for distributed applications. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -This working package consists of the tasks WP31_ WP32_ WP33_. +*Part a):* + + - work with stackless results *FIXME: write this* + +*Part b):* + + - use the whole optimized thing *FIXME: write this* .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -41,6 +47,6 @@ **Milestones and Expected Result** -- M2 +- M9 .. include:: wp-tableend.asc Copied: pypy/trunk/doc/funding/B6.wp60_documentation.txt (from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp30_compiler.txt (original) +++ pypy/trunk/doc/funding/B6.wp60_documentation.txt Tue Sep 23 11:52:57 2003 @@ -1,14 +1,14 @@ .. include:: crossreferences.asc -.. |title| replace:: The PyPy Compiler -.. |wp| replace:: WP30 +.. |title| replace:: Project Documentation and Dissemination +.. |wp| replace:: WP60 .. |start| replace:: 0 -.. |p1| replace:: |e| -.. |m1| replace:: |e| -.. |p2| replace:: |e| -.. |m2| replace:: |e| -.. |p3| replace:: |e| -.. |m3| replace:: |e| +.. |p1| replace:: AM +.. |m1| replace:: 19 +.. |p2| replace:: JH +.. |m2| replace:: 3 +.. |p3| replace:: MWH +.. |m3| replace:: 3 .. |p4| replace:: |e| .. |m4| replace:: |e| .. |p5| replace:: |e| @@ -22,14 +22,24 @@ **Objectives** -Building the translator and Bootstrapping PyPy. +Providing ongoing documentation throughout the whole project. + +Keeping external groups informed of the ongoings in PyPy. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -This working package consists of the tasks WP31_ WP32_ WP33_. +- During the whole project, maintain and extend a set of documents + represends the current status of the project, results of + discussions, the planning of new sprints and their status reports, + as well as the preparation of papers for presentation on + various congresses. + +- Send reports of ongoings in the project to the Python Business + Forum (PBF) and the Python + developers list (python-dev at python.org). .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -41,6 +51,6 @@ **Milestones and Expected Result** -- M2 +- Good documentation and dissemination .. include:: wp-tableend.asc Copied: pypy/trunk/doc/funding/B6.wp70_maintenance.txt (from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp30_compiler.txt (original) +++ pypy/trunk/doc/funding/B6.wp70_maintenance.txt Tue Sep 23 11:52:57 2003 @@ -1,12 +1,12 @@ .. include:: crossreferences.asc -.. |title| replace:: The PyPy Compiler -.. |wp| replace:: WP30 +.. |title| replace:: Maintenance of Tools +.. |wp| replace:: WP70 .. |start| replace:: 0 -.. |p1| replace:: |e| -.. |m1| replace:: |e| -.. |p2| replace:: |e| -.. |m2| replace:: |e| +.. |p1| replace:: X +.. |m1| replace:: 5 +.. |p2| replace:: Y +.. |m2| replace:: 5 .. |p3| replace:: |e| .. |m3| replace:: |e| .. |p4| replace:: |e| @@ -22,14 +22,28 @@ **Objectives** -Building the translator and Bootstrapping PyPy. +Supporting the PyPy project by maintaining and enhancing the tools. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -This working package consists of the tasks WP31_ WP32_ WP33_. +- Keep the infrastructure up-to-date and develop it further. + + - maintain the Subversion package and track its ongoing development + + - maintain the Subversion repository + + - build extensions for automatic document extraction + + - support email notification via mailing lists + +- Provide a mirror repository which is kept up-to-date and which + can be used in case of failure of the main repository. + +- Take care of regular repository backups. Design a strict backup + policy suitable for the project, and adhere to it. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc @@ -41,6 +55,4 @@ **Milestones and Expected Result** -- M2 - .. include:: wp-tableend.asc Copied: pypy/trunk/doc/funding/B6.wp80_synchronisation.txt (from rev 1404, pypy/trunk/doc/funding/B6.wp30_compiler.txt) ============================================================================== --- pypy/trunk/doc/funding/B6.wp30_compiler.txt (original) +++ pypy/trunk/doc/funding/B6.wp80_synchronisation.txt Tue Sep 23 11:52:57 2003 @@ -1,10 +1,10 @@ .. include:: crossreferences.asc -.. |title| replace:: The PyPy Compiler -.. |wp| replace:: WP30 +.. |title| replace:: Synchronisation with Standard Python +.. |wp| replace:: WP80 .. |start| replace:: 0 -.. |p1| replace:: |e| -.. |m1| replace:: |e| +.. |p1| replace:: X +.. |m1| replace:: 6 .. |p2| replace:: |e| .. |m2| replace:: |e| .. |p3| replace:: |e| @@ -22,25 +22,39 @@ **Objectives** -Building the translator and Bootstrapping PyPy. +Keeping PyPy in sync with the ongoing changes to Standard Python. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Description of work** -This working package consists of the tasks WP31_ WP32_ WP33_. +- Monitor the development of Standard Python for its ongoing changes. + +- Check the relevance of the changes concerning the structure of + PyPy. + +- Embed trivial changes with proper testing, notifying the other + concurrent tasks of these changes. + +- Changes which impose larger structural changes must influence + the overall design and be discussed with the whole group. + It is the responsibility of this task to gather such information + as early as possible. + +- Also investigate whether there are tools which could automate + integration of simple changes, partially. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Deliverables** +- Running all tests of PyPy and Stadard Python after every update. + .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Milestones and Expected Result** -- M2 - .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Tue Sep 23 11:52:57 2003 @@ -15,12 +15,11 @@ .. _WP51: B6.wp51_embedded_device.html .. _WP52: B6.wp52_small_os_pre.html .. _WP53: B6.wp53_small_os.html -.. _WP54: -.. _WP55: -.. _WP56: -.. _WP60: -.. _WP70: -.. _WP80: -.. attention:: **6 files missing: WP54 WP55 WP56 WP60 WP70 WP80** +.. _WP54: B6.wp54_simd_architecture.html +.. _WP55: B6.wp55_numerical_package.html +.. _WP56: B6.wp56_load_balancing.html +.. _WP60: B6.wp60_documentation.html +.. _WP70: B6.wp70_maintenance.html +.. _WP80: B6.wp80_synchronisation.html .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file From lac at codespeak.net Tue Sep 23 11:53:05 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Tue, 23 Sep 2003 11:53:05 +0200 (MEST) Subject: [pypy-svn] rev 1409 - pypy/trunk/doc/funding Message-ID: <20030923095305.BECE35C2E2@thoth.codespeak.net> Author: lac Date: Tue Sep 23 11:53:04 2003 New Revision: 1409 Added: pypy/trunk/doc/funding/Z1_background.txt (props changed) - copied unchanged from rev 1383, pypy/trunk/doc/funding/B2.0.0_background.txt Removed: pypy/trunk/doc/funding/B2.0.0_background.txt Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt Log: Reordering. The background was to go, so now is Z1. I am now Spelling Appendix 'Zappendix' so they will sort to the end of the directory. Deleted: /pypy/trunk/doc/funding/B2.0.0_background.txt ============================================================================== --- /pypy/trunk/doc/funding/B2.0.0_background.txt Tue Sep 23 11:53:04 2003 +++ (empty file) @@ -1,288 +0,0 @@ -B2.0.0 background -================= - -THIS WE NEED TO MAKE APPENDIX A. We can canabalise as we like for -other sections. But the Message from Stockholm is, 'don't give them -a background like this. The evaluators get bored' - -The relevance of Python ------------------------ - -Python is a portable, interpreted, object-oriented Very-High Level -Language (VHLL). Its development started in 1990 at CWI, Centrum voor -Wiskunde en Informatica, the National Research Institute for -Mathematics and Computer Science in the Netherlands. Its principal -author is Guido van Rossum, a Dutch citizen currently living in the -United States. It is an Free/Open Source language. The most recent -version of the language is Python 2.3, released under the Python -Software Foundation License, which is approved by both the Open Source -Initiative, and the Free Software Foundation. - -FIXME: Include this as a reference. -http://www.opensource.org/licenses/PythonSoftFoundation.php - -Python has tens, perhaps hundreds of thousands of active developers -worldwide, which makes it one of the top ten most popular programming -languages in the world. [FIXME Footnote -- see section XXX and -Appendix XXX for support for this assertion]. Some other languages, -specifically C, C++, Java, Perl, and Visual Basic, have even larger -user bases but they are either proprietary or rather low-level -languages. On the other hand, the languages which most excite the -Computer Science Research community -- Self, Lisp, Haskell, Limbo, ML, -and so on -- are nowhere on this list, yet they are the targets of -most European academic research and innovation. Thus European -economic competitiveness suffers. The innovative research lives in -academia, trapped in languages that are rarely used for commercial -development. - -Of those more popular languages, two, Java and Visual Basic are -proprietary. Sun Microsystems owns Java, and Microsoft owns Visual -Basic. Any company which writes its software in Java or Visual Basic -is at the mercy of these large American companies. And this is a -real, not theoretical, threat. In 2002, Microsoft announced that it -would no longer be supporting Visual Basic 6.0 after the year 2005. -All Visual Basic Developers have been told to convert their code to -run under Microsoft's new .NET framework. In 2001 Microsoft -immediately stopped supporting its Visual J++ language, meant to be a -direct competitor with Java, after settling a lawsuit with Sun -Microsystems. Microsoft is making these decisions because they make -business sense for Microsoft, regardless of the effects on businesses -who develop software using Microsoft proprietary software. - - -European SMEs are moving to Free/Open Source platforms ------------------------------------------------------- - -In the face of these threats to the very survival of their businesses, -European SMEs are moving to Free/Open Source languages such as Python. -In the year 2002, a group of SME's who rely on the Python programming -language came together to form the Python Business Forum -(www.python-in-business.org), at EuroPython, the European Python -Community Conference (www.europython.org). - -FIXME! include the bylaws of the PBF, its Board of Directors and the -like as an Appendix. - -Advancing the Python platform ------------------------------ - -While each SME member of the Python Business Forum has sufficient -faith in the Python programming language to use it for the development -of its own projects, it was agreed that there are defects in the -current implementation of the language. The two most often cited was -that the Python was too large for embedded applications and -applications designed for handhelds, and that the interpreter itself -ran too slowly. - -The developers of embedded systems intend to run on tiny machines -would like a language with a 'smaller footprint'. They would like to -strip out everything which they do not need from the language and run -with the bare-bones minimum. This is hard to do in any language, and -Python was not implemented with this goal in mind. - -Another goal that was not paramount in Python's design and -implementation was execution speed. Python is a dynamically-typed, -late-binding, interpreted language. While this proved to provide -extremely productive development environments, execution speed -sometimes is not fast enough. Today, optimisation of high-level -languages must be done at run-time, and is notoriously more difficult -to optimise than statically typed, early-binding compiled languages -such as C or C++. - -Now a number of people and factors played together to start what is -now one of the most promising high-level-language projects. - -Some high-profile research --------------------------- - -Independently some researchers who worked with Python were pondering -writing an implementation of Python in Python itself. This group -included Armin Rigo, author of Psyco -http://psyco.sourceforge.net/introduction.html, a specialising -Just-In-Time compiler for Python. He is intimately familiar with both -Python internals and advanced research in compilers and runtime -systems, and saw a Python implementation in Python itself as a chance -to put the two fields together. It is useful to quote from his webpage -which states the goals of his Psyco in full: - - My goal in programming Psyco is to contribute to reduce the - following wide gap between academic computer science and - industrial programming tools. - - While the former develops a number of programming languages - with very cool semantics and features, the latter stick with - low-level languages principally for performance reasons, on - the ground that the higher the level of a language, the slower - it is. Although clearly justified in practise, this belief is - theoretically false, and even completely inverted --- for - large, evolving systems like a whole operating system and its - applications, high-level programming can deliver much higher - performances. - - The new class of languages called 'dynamic scripting - languages', of which Python is an example, is semantically - close to long-studied languages like Lisp. The constraints - behind their designs are however different: some high-level - languages can be relatively well statically compiled, we can - do some type inference, and so on, whereas with Python it is - much harder --- the design goal was different. We now have - powerful enough machines to stick with interpretation for a - number of applications. This, of course, contributes to the - common belief that high-level languages are terribly slow. - - Psyco is both an academic and an industrial project. It is an - academic experiment testing some new techniques in the field - of on-line specialization. It develops an industrially useful - performance benefit for Python. And first of all it is a - modest step towards: - - High-level languages are faster than low-level ones! - -FIXME Footnote to: http://psyco.sourceforge.net/introduction.html - -Although Armin Rigo proved with his 'Psyco' project that higher level -languages can actually be optimized to be as fast or faster than C, he -was very limited by the fact that Python is itself implemented in C -like almost all other languages in industrial use today. There was no -real-life project who tried to go all 'optimize high-level down to -machine-code' way. Christian Tismer with his 'Stackless' project had -already come to a similar conclusion from a more industrial viewpoint -in that it is difficult to advance language technology while relying -on a large C-code base. - -FIXME Footnote to: http://www.stackless.com/ - -Some mailing list discussions ------------------------------ - -On the German Python mailing list an independent discussion evolved -where developers and academics were pondering about the possibility of -developing a 'minimal' Python implemented in Python itself. Most -noticeably Christian Tismer author of an industrial-use extension -('Stackless') of the Python language noted in a postscript to one of -his mails that having a minimized version of Python could provide a -new base to advance the language. Nevertheless, his extensions to the -language proved to be useful for companies who needed a way to have -millions of active objects and he had a branch of CPython to make this -possible. - -Some organizational experience ------------------------------- - -Meanwhile Holger Krekel had joined the Python community in 2002. For -some years he had designed the architecture for platforms and -consulted for CEO's of some large banking centers in Europe. While -participating in two 'coding sprints' of the Zope3 web-platform (the -to-be successor of the successful Zope web-platform) he realized that -agile 'sprints' in combination with the rapid development language -Python provide an extremely productive way of communicating about and -coding complex projects where traditional, slow-moving methods often -fail. - -At a Sprint a group of people assemble together to write code and -practice Agile software methodological techniques, such as -Pair-Programming. Not only is this a lot of fun, but it is a way to -transmit knowledge and enthusiasm throughout the community. - -ASK_STOCKHOLM DO we need to discuss Sprints more? - -Holger Krekel, seeing the opportunity to launch the PyPy project with -Armin Rigo and Christian Tismer offered to organize the first one-week -meeting, the 'Sprint towards a minimal Python'. Soon many interested -developers joined and intense academical and practical planning -ensued. Just a few weeks later the Sprint took place at -'Trillke-Gut', a castle-like building in Germany, bringing together -some key developers, among them Michael Hudson, the release manager of -version 2.2.1 of Python. - -Here is the mail that started this now rapidly evolving project -http://starship.python.net/pipermail/python-de/2003q1/002925.html - -Some promising open-source development tools --------------------------------------------- - -From the beginning, the PyPy developers were committed using and -integrating the most promising open-source technologies. Jens-Uwe -Mager, the retired CTO from Helios (http://www.helios.de) attended the -Sprint and helped set up a state-of-the-art open-source development -environment. With his 12-year experience of setting up and leading a -SME-company which is one of the worldwide leaders in -print-preprocessing technology he helped organise the development and -net-connectivity for the various web services needed by the PyPy -developers. - -Research, pragmatism and industry experience combined ------------------------------------------------------ - -This combination of research, applicability and industrial experience -was precisely what the Python Business Forum members had been looking -for. Academics, developers and practitioners conversant in the latest -in language and platform architecture who were interested in producing -the Python interpreter which they needed. Laura Creighton and Jacob -Hall?n from the PBF attended the sprint and began participating in the -project. - -The group decided soon to test their proof of concept by developing a -working prototype and to test their ability to work together. They -would meet for Sprints in their own private time, and work on the -prototype. - -After the first Sprint at Trillke-Gut in Hildesheim, Jacob Hall?n and -Laura Creighton organized the next Sprint at AB Strakt in Gothenburg, -Sweden. At this Sprint, Samuele Pedroni, the lead developer of Jython -(a tight industrial-use integration of Java and Python), joined the -project not only because such a project was relevant to his own -computer science research interests, but also to see if PyPy would -help with his own java-integration project. Prototyping went quickly, -with the work of all these experienced people, and by the end of the -week you could already run simple Python programs within PyPy. PyPy -had gone from being 'a nice idea' to 'something we knew we could do'. - - - - - -The third sprint was organized by interested developers in Belgium at -the University in Louvain-La-Neuve, Belgium and held June 20-24. We -invited Guido van Rossum, the inventor of Python to attend. He not -only attended the Belgium sprint but announced a few days later at the -EuroPython conference that PyPy had a high priority on his list of -'dreams he hoped would come true' and he enjoyed sprinting with us a -lot. - -By the end of the third one-week sprint at the University in -Louvain-La-Neuve, the PyPy project had already produced a fully -working prototype. There existed a working interpreter and standard -implementation of 90% of the python types, as well as advanced -language features such as nested scopes, generators and metaclasses. -Armin Rigo and Guido van Rossum had started work on a type-inference -engine which is the foundation for the final missing step: generating -a native machine-level version of Python from its high-level -PyPy-implementation. Perhaps most importantly, we had the -enthusiastic support of the Python community at the EuroPython -conference. - -No work had been done on actually optimising the code so it ran around -30,000 times slower than the existing CPython-implementation but this -was expected from the start. Nevertheless, for a proof of concept, -approximately four weeks work total for a group of about a dozen -people, it was clearly a success. - -It was time to look for funding. - -< -While some people suggested to apply to DARPA for funding we realized -that most of us were rooted in Europe and it would make more sense to -look for European possibilities. - -I don't think that this is wise. Why remind them that we could get -somebody else to fund it, maybe? Besides, everybody on the list of -people they are getting are in Europe, not just most ... -> - -On June 17th, the 2nd Call of the Information Society Technologies -[IST] Priority went out. Included in it was IST-2002-2.3.2.3 - Open -development platforms for software and services. - -We believe that what we intend to do is a perfect fit for the goals of -this call. Modified: pypy/trunk/doc/funding/B2.0.0_relevance.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_relevance.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_relevance.txt Tue Sep 23 11:53:04 2003 @@ -6,6 +6,10 @@ (Recommended length ? three pages) ------------------------------------------- +This is the stuff I wrote for B2. It belongs in B1, if at all. +I sort of like how I made the PyPy objective match the Call objective. +That might be good to clip for B1. +--------------------------------------------------------------- PyPy is a precise match for IST-2002-2.3.2.3 - Open development platforms for software and services. Its stated objective is as From lac at codespeak.net Tue Sep 23 11:54:10 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Tue, 23 Sep 2003 11:54:10 +0200 (MEST) Subject: [pypy-svn] rev 1410 - pypy/trunk/doc/funding Message-ID: <20030923095410.CE0A05B658@thoth.codespeak.net> Author: lac Date: Tue Sep 23 11:54:10 2003 New Revision: 1410 Added: pypy/trunk/doc/funding/B1._obj_from_badB2.txt (props changed) - copied unchanged from rev 1409, pypy/trunk/doc/funding/B2.0.0_relevance.txt Removed: pypy/trunk/doc/funding/B2.0.0_relevance.txt Log: Now rename the B2 that was wrong to a B1 name. Deleted: /pypy/trunk/doc/funding/B2.0.0_relevance.txt ============================================================================== --- /pypy/trunk/doc/funding/B2.0.0_relevance.txt Tue Sep 23 11:54:10 2003 +++ (empty file) @@ -1,151 +0,0 @@ -B.2 Relevance to the objectives of the IST Priority - -Describe the manner in which the proposed project's goals address the -scientific, technical, wider societal and policy objectives of the IST -Priority in the areas concerned. - -(Recommended length ? three pages) -------------------------------------------- -This is the stuff I wrote for B2. It belongs in B1, if at all. -I sort of like how I made the PyPy objective match the Call objective. -That might be good to clip for B1. ---------------------------------------------------------------- - -PyPy is a precise match for IST-2002-2.3.2.3 - Open development -platforms for software and services. Its stated objective is as -follows: - - To build open development and run-time environments for software - and services providing the next generation of methodologies, - interoperable middleware and tools to support developers - through - all phases of the software life-cycle, from requirements analysis - until deployment and maintenance - in the production of networked - and distributed software systems and services, embedded software - and value-added user services. This will enable the development of - future software engineering methods and tools. - -The PyPy project aims to build an flexible and fast implementation of -the Open Source programming language Python written in itself. Python -is a Very High Level Language, a modern tool for developing software. -While Python is already suitable for developing software of all sizes, -this new implementation of Python would be particularly suitable for -the development of networked, distributed software systems, and -embedded software. It will facilitate the development of future -software engineering methods and tools. - -Priority is to be given to projects in which - - 'strong industrial users join forces with software and service - suppliers in building common platforms with support of - academic research partners'. - -This project is a collaboration between academic researchers, and -software and service providers to produce an outcome desired by all -industrial users. ASK_STOCKHOLM 'what's an Industrial User?' Is AB -Strakt one? Large companies, who are not software providers, such as -Bang and Bang & Olufsen, the Danish maker of high-end audio equipment, -http://www.bang-olufsen.com/ have expressed interest in using PyPy -once it is developed. - -Moreover, our project is relevant to every focus objective. - -Focus is on: - - a) High level methods and concepts (esp. at requirements and - architectural level) for system design, development and - integration, addressing non-functional aspects, complexity, - autonomy and composability. - -PyPy will advance the 'state-of-the-art' in high level -implementation of languages, introducing high level methods -and concepts in the domain, producing a development environment -and methods with as-yet-unmatched flexibility, -with a new architecture for object libraries that provide for radical -improvements in (the management of) complexity, autonomy -(self-hosting), and composability (seamless modularity). - -ASK_STOCKHOLM DO I HAVE TO EXPLAIN THIS MORE? - - b) Open and modular development environments, enabling - flexibility and extensibility with new or sector-specific - tools (e.g. intelligent distributed decision support), - supporting different adaptable development processes and - methodologies and ensuring consistency and traceability - across the development lifecycle. - -Flexibility is one of the primary goals of PyPy. Our new architecture -will be the basis of a host of more flexible and extensible ways to -create software products. PyPy will, of course, be released as -Free/Open Source software, under the MIT license. - -FOOTNOTE: Stick MIT License in here. - -ASK_STOCKHOLM -- Am I correct? They want the thing we made to be Open -and Modular? Or are they asking that our own way of working uses Open -and Modular techniques? We do this as well .... - - c) Light/agile methodologies and adaptive workflow - providing a dynamic and adaptive environment, suitable - for co-operative and distributed development. - -Each member of the group is committed to Agile Methodologies. Some of -us have experience with Crystal and others with XP, [FIXME add URLS -and see if we have any more trained people on the team]two of the more -popular ones. Moreover, since we have been active for many years in -the Open Source community, we are already used to co-operating with -others in a distributed development process. - -ASK_STOCKHOLM -- is this what they want, or more like this? - -AB Strakt is a young Swedish software company whose main product, CAPS -is an adaptive workflow framework. A faster Python would seamlessly -improve this product, and those applications which are based upon it. - -[FIXME: if we want this bit, we want to refer to the AB Strakt - Appendix, that says that we are a cool company, and that we do - workflow and procurement, and that we are responsible -- blah blah - blah. I'll boast about Strakt later, or see if I can steal some - boasts from marketing.] - - d) Open platforms, middleware and languages supporting - standards for interoperability, composability and - integration. (incl. e.g. P2P, GRID, autonomy, agents, - dynamic adaptability and evolvability, context - awareness, customer profiles). Open source middleware - layers can facilitate rapid and broad adoption. - -PyPy is an Open Language. ASK_STOCKHOLM. So they have just asked us -to make them a language? Or they just want us to use a language to -make what we are making? - - In addition, related foundational research, to be implemented - by Specific Targeted Research Projects and Coordination - Actions, should focus on fundamental design concepts, - systematisation of domain specifications, concurrency, - distribution and timing, formal and quantitative analysis and - testing tools, and future database and information system - concepts. - -ASK_STOCKHOLM PyPy is Applied Research. When they say 'foundational -research' do they want 'foundational applied research', in which case -we can all cheer and say, here we are with a STREP, new foundational -research in the field of specializing compilers, just what you asked -for? Or by 'foundational research' do they mean the sort of basic -research that has no direct applications, but goes under the heading -'it is nice to learn stuff'? - - FIXME - Emphasis "quantitative analysis" as per WP06 and WP07 where we - describe PyPy as a research and comparison platform - - Work should where appropriate, enhance and complement work - implemented under EUREKA/ITEA and in software initiatives at - member and associated state level. The IST programme will seek - active co-operation with ITEA in software intensive systems. - -ASK_STOCKHOLM I don't know how to find out which work implemented -under EUREKA/ITEA was implemented using Python. I do know of 2 prior -IST projects. FIXME: add URL for ASWAD. A faster Python will enhance -any project done in Python. Also, if this means that EUREKA/ITEA has -more pots of money that they would like to hand us, we'd love to take -it. From lac at codespeak.net Tue Sep 23 12:00:05 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Tue, 23 Sep 2003 12:00:05 +0200 (MEST) Subject: [pypy-svn] rev 1411 - pypy/trunk/doc/funding Message-ID: <20030923100005.839E25B658@thoth.codespeak.net> Author: lac Date: Tue Sep 23 12:00:04 2003 New Revision: 1411 Added: pypy/trunk/doc/funding/B2.0.0_new_relevance.txt Log: Start over, now that I know what I should have been writing Added: pypy/trunk/doc/funding/B2.0.0_new_relevance.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/B2.0.0_new_relevance.txt Tue Sep 23 12:00:04 2003 @@ -0,0 +1,30 @@ +B.2 Relevance to the objectives of the IST Priority + +Describe the manner in which the proposed project's goals address the +scientific, technical, wider societal and policy objectives of the IST +Priority in the areas concerned. (Recommended length ? three pages) + +Information for this section comes from several main sources - + + + 1. Each Workprogram and the Commission specific program documents + identify and address the policy needs to a certain extent. + The introductory sections of the Workprogram for IST 2003/4 + contains good reference material. + + + 2. Via the Europa web site, http://europa.eu.int there is + information on all EU policies and they can be identified and + downloaded from there. For example we have the following ? + Policies - Access by subject to legal instruments in force, + legislative activity in progress, implementation of common + policies, EU grants and loans, statistics and publications. + + + 3. There is also good material under eEurope initiatives and at + the ISPO (information Society Project Office) site. + + +You must also address where appropriate ERA related issues such as +relationships to any Eureka activities, (such as commonality of partners) +or relationships to national research programs. From tismer at codespeak.net Tue Sep 23 15:10:43 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 23 Sep 2003 15:10:43 +0200 (MEST) Subject: [pypy-svn] rev 1412 - pypy/trunk/doc/funding Message-ID: <20030923131043.4F34B5ACAE@thoth.codespeak.net> Author: tismer Date: Tue Sep 23 15:10:42 2003 New Revision: 1412 Added: pypy/trunk/doc/funding/pypy.mpp (contents, props changed) Log: added Microsoft project file for PyPy Added: pypy/trunk/doc/funding/pypy.mpp ============================================================================== Binary file. No diff available. From arigo at codespeak.net Tue Sep 23 15:32:58 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Sep 2003 15:32:58 +0200 (MEST) Subject: [pypy-svn] rev 1413 - pypy/trunk/doc/objspace Message-ID: <20030923133258.617D35ACAE@thoth.codespeak.net> Author: arigo Date: Tue Sep 23 15:32:57 2003 New Revision: 1413 Modified: pypy/trunk/doc/objspace/objspaceinterface.txt Log: Updated this document Modified: pypy/trunk/doc/objspace/objspaceinterface.txt ============================================================================== --- pypy/trunk/doc/objspace/objspaceinterface.txt (original) +++ pypy/trunk/doc/objspace/objspaceinterface.txt Tue Sep 23 15:32:57 2003 @@ -2,23 +2,11 @@ PyPython ObjectSpaceInterface ================================ -Note this is only a draft version of the ObjectSpace_ interface. The reality of the interface depends on the particular implementation you are using. Consult the code for details. +This is a draft version of the ObjectSpace_ interface. It is still evolving, although the public interface is not evolving as much as the internal interface that subclasses need to know about. This document now generally refers to the public interface; for subclassing you need to poke at the code in detail anyway. -class ObjSpace -================= -Data Members ------------------ - -+ ObjSpace.MethodTable: - List of tuples (method name, symbol, number of arguments) for the regular part of the interface. (Tuples are interpreter level.) - -+ self.w_builtins -+ self.w_modules -+ self.appfile_helpers -+ self.w_None: The ObjectSpace's None -+ self.w_True: The ObjectSpace's True -+ self.w_False: The ObjectSpace's False +class baseobjspace.ObjSpace +=========================== Administrative Functions ---------------------------- @@ -29,9 +17,6 @@ **getexecutioncontext():** Return current active execution context. -**gethelper(applicationfile):** - Get helper for applicationfile. - Operations on Objects in ObjectSpace ----------------------------------------- @@ -45,27 +30,37 @@ ``pos, neg, not_, abs, invert, add, sub, mul, truediv, floordiv, div, mod, divmod, pow, lshift, rshift, and_, or_, xor,`` +``hex, oct, int, float, ord,`` + ``lt, le, eq, ne, gt, ge, contains,`` -``inplace_add, inplace_sub, inplace_mul, inplace_truediv, inplace_floordiv, inplace_div, inplace_mod, inplace_pow, inplace_lshift, inplace_rshift, inplace_and, inplace_or, inplace_xor`` +``inplace_add, inplace_sub, inplace_mul, inplace_truediv, inplace_floordiv, inplace_div, inplace_mod, inplace_pow, inplace_lshift, inplace_rshift, inplace_and, inplace_or, inplace_xor,`` + +``get, set, delete`` **next(w):** - Call the next function for iterator w. + Call the next function for iterator w, or raises a real NoValue. -**call(callable, args, kwds):** +**call(w_callable, w_args, w_kwds):** Call a function with the given args and keywords. +**call_function(w_callable, *args_w, **kw_w):** + Convenience function that collects the arguments in a wrapped tuple and dict and invokes 'call()'. + **is_(w_x, w_y):** - Implements 'w_x is w_y'. + Implements 'w_x is w_y'. (Returns a wrapped result too!) + +**isinstance(w_obj, w_type):** + Implements 'issubtype(type(w_obj), w_type)'. (Returns a wrapped result too!) **exception_match(w_exc_type, w_check_class):** - Checks if the given exception type matches 'w_check_class'. Used in matching the actual exception raised with the list of those to catch in an except clause. Returns a bool. + Checks if the given exception type matches 'w_check_class'. Used in matching the actual exception raised with the list of those to catch in an except clause. (Returns a wrapped result too!) Creation of Application Level objects --------------------------------------- **wrap(x):** - Return ObjectSpace equivalent of x. + Returns a wrapped object that is a reference to the interpreter-level object x. This can be used either on simple immutable objects (integers, strings...) to create a new wrapped object, or on complex mutable objects to obtain an application-level-visible reference to them (e.g. instances of internal interpreter classes). **newbool(b):** Creates a Bool Object from an interpreter level object. @@ -77,19 +72,13 @@ Takes an interpreter level list of wrapped objects. **newdict([..]):** - Takes an interpreter level list of interpreter level pairs of wrapped key:wrapped value entries. + Takes an interpreter level list of interpreter level pairs of wrapped key:wrapped value entries (and NOT an interpreter level dictionary!). **newslice(w_start, w_end, w_step):** Makes a new slice object. -**newfunction(w_code, w_globals, w_defaultarguments, w_closure=None):** - Creates a new function object. - **newstring(asciilist):** - Creates a string from a list of integers. - -**newmodule(w_name):** - Creates a new module with a given name. + Creates a string from a list of wrapped integers. Conversions from Application Level to Interpreter Level ---------------------------------------------------------- @@ -108,4 +97,31 @@ --------------------------- + + +Data Members +----------------- + ++ self.w_builtins ++ self.w_modules ++ self.w_None: The ObjectSpace's None ++ self.w_True: The ObjectSpace's True ++ self.w_False: The ObjectSpace's False ++ self.w_Ellipsis: The ObjectSpace's Ellipsis ++ self.w_NotImplemented: The ObjectSpace's NotImplemented + ++ ObjSpace.MethodTable: + List of tuples (method name, symbol, number of arguments, list of special names) for the regular part of the interface. (Tuples are interpreter level.) + ++ ObjSpace.BuiltinModuleTable: + List of names of built-in modules. + ++ ObjSpace.ConstantTable: + List of names of the constants that the object space should define + ++ ObjSpace.ExceptionTable: + List of names of exception classes. + +--------------------------- + .. _ObjectSpace: objspace.html From arigo at codespeak.net Tue Sep 23 15:34:30 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Sep 2003 15:34:30 +0200 (MEST) Subject: [pypy-svn] rev 1414 - pypy/trunk/doc/objspace Message-ID: <20030923133430.E47C15ACAE@thoth.codespeak.net> Author: arigo Date: Tue Sep 23 15:34:30 2003 New Revision: 1414 Modified: pypy/trunk/doc/objspace/objspaceinterface.txt Log: update Modified: pypy/trunk/doc/objspace/objspaceinterface.txt ============================================================================== --- pypy/trunk/doc/objspace/objspaceinterface.txt (original) +++ pypy/trunk/doc/objspace/objspaceinterface.txt Tue Sep 23 15:34:30 2003 @@ -95,9 +95,6 @@ **unpacktuple(w_tuple, expected_length=None):** Same as unpackiterable(), but only for tuples. ---------------------------- - - Data Members ----------------- From tismer at codespeak.net Tue Sep 23 15:56:44 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 23 Sep 2003 15:56:44 +0200 (MEST) Subject: [pypy-svn] rev 1415 - pypy/trunk/doc/funding Message-ID: <20030923135644.211085ACAE@thoth.codespeak.net> Author: tismer Date: Tue Sep 23 15:56:43 2003 New Revision: 1415 Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt pypy/trunk/doc/funding/B6.wp55_numerical_package.txt Log: all WPs are so far ready but the load balancing. And we wanted to add a "eat your own dog food" one. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 23 15:56:43 2003 @@ -304,21 +304,38 @@ an optimum implementation for the particular hardware configuration during startup time. -Write specialized code generators which -support SIMD instruction sets, like SSE2 and AltiVec. + +WP54_: Code Generator for SIMD Architecture +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + In order to support numerical applications better, + PyPy should be extended to support special hardware + like SIMD (Single Instruction Multiple Data) processors. + We are aiming to pick one SIMD architecture, like + SSE2 or AltiVec, and extend the code generator to support + and optimize for the new instructions. + We don't intend to support a completely new instruction + set, but prefer to choose an extension to an architecture + that we are already supporting. + + The existence of special instructions and other extensions + should be probed using run-time checks. Psyco should be + extended to be aware of parallel instructions, and be + enabled to emit optimized code for that. + + PyPy needs to be extended to support a vectorized + data type from the Python level, as well. + The specification of that is part of this task. + +WP55_: Enhanced Numerical Package +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Rewrite a numerical Python package like NumPy using RPython. Identify the numerical operations in the package which are candidates for parallelization. Write an interface that allows to implement these operations in either a traditional way or using SIMD instructions, -dependant from the available capabilities of the current +dependent from the available capabilities of the current code generator. -The ability of special instructions and other extensions -should be probed using run-time checks. Psyco should be -extended to be aware of parallel instructions, and be -enabled to emit optimized code for that. - Load Balancing in a Multi-Processor environment --------------------------------------------------- Modified: pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt (original) +++ pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt Tue Sep 23 15:56:43 2003 @@ -31,13 +31,34 @@ **Description of work** -*FIXME: write that* +- Select a processor architecture which has SIMD extensions, + like SSE2 or AltiVec. + Preference should be given to a technology that we are + supporting, already. + +- Examine the necessary design changes/extensions to PyPy for supporting + parallel instructions. + +- Extend the code generator to support + and optimize for the new instructions. + This involves re-iterations of parts of WP32_ and WP43_. + Extending Psyco involves WP44_ as well. + +- Extend PyPy by an uto-probing feature that tries to + examine the availability of certain extensions at runtime. + +- Design and implement the necessary vectorised data types + for Python/RPython. The extra types should be made sufficient to + enable us to write parallel extension modules, and to + write the new testing programs. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc **Deliverables** +- Parallel extension for PyPy + .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc Modified: pypy/trunk/doc/funding/B6.wp55_numerical_package.txt ============================================================================== --- pypy/trunk/doc/funding/B6.wp55_numerical_package.txt (original) +++ pypy/trunk/doc/funding/B6.wp55_numerical_package.txt Tue Sep 23 15:56:43 2003 @@ -29,7 +29,16 @@ **Description of work** -*FIXME: write this* +- Decide which numerical extension package to re-implement in PyPy. + +- Re-implement the numeric package in terms of applying the + new vectorized operations. + +- Make sure that the new implementation passes all tests. + +- Provide backward-compatibility of the package in a way that + allows it to fall back to a simpler architecture when there + is no SIMD extension available. .. include:: wp-tableend.asc .. include:: wp-tablebegin.asc From tismer at codespeak.net Thu Sep 25 11:27:16 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 25 Sep 2003 11:27:16 +0200 (MEST) Subject: [pypy-svn] rev 1416 - pypy/trunk/doc/funding Message-ID: <20030925092716.C66975A271@thoth.codespeak.net> Author: tismer Date: Thu Sep 25 11:27:15 2003 New Revision: 1416 Added: pypy/trunk/doc/funding/B6.7.wp10_management.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp10_management.txt pypy/trunk/doc/funding/B6.7.wp20_interpreter.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp20_interpreter.txt pypy/trunk/doc/funding/B6.7.wp21_interpreter_completion.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt pypy/trunk/doc/funding/B6.7.wp22_cpython_to_python.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp22_cpython_to_python.txt pypy/trunk/doc/funding/B6.7.wp30_compiler.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp30_compiler.txt pypy/trunk/doc/funding/B6.7.wp31_translation_of_rpython.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt pypy/trunk/doc/funding/B6.7.wp32_bootstrap_redesign.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt pypy/trunk/doc/funding/B6.7.wp33_bootstrap_runtime.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp33_bootstrap_runtime.txt pypy/trunk/doc/funding/B6.7.wp40_high_performance.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp40_high_performance.txt pypy/trunk/doc/funding/B6.7.wp41_integrate_stackless.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt pypy/trunk/doc/funding/B6.7.wp42_several_obj_impl.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt pypy/trunk/doc/funding/B6.7.wp43_translator_optimisations.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt pypy/trunk/doc/funding/B6.7.wp44_integrate_psyco.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt pypy/trunk/doc/funding/B6.7.wp50_validations.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp50_validations.txt pypy/trunk/doc/funding/B6.7.wp51_embedded_device.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp51_embedded_device.txt pypy/trunk/doc/funding/B6.7.wp52_small_os_pre.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp52_small_os_pre.txt pypy/trunk/doc/funding/B6.7.wp53_small_os.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp53_small_os.txt pypy/trunk/doc/funding/B6.7.wp54_simd_architecture.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt pypy/trunk/doc/funding/B6.7.wp55_numerical_package.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp55_numerical_package.txt pypy/trunk/doc/funding/B6.7.wp56_load_balancing.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp56_load_balancing.txt pypy/trunk/doc/funding/B6.7.wp60_documentation.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp60_documentation.txt pypy/trunk/doc/funding/B6.7.wp70_maintenance.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp70_maintenance.txt pypy/trunk/doc/funding/B6.7.wp80_synchronisation.txt (props changed) - copied unchanged from rev 1415, pypy/trunk/doc/funding/B6.wp80_synchronisation.txt Removed: pypy/trunk/doc/funding/B6.wp10_management.txt pypy/trunk/doc/funding/B6.wp20_interpreter.txt pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt pypy/trunk/doc/funding/B6.wp22_cpython_to_python.txt pypy/trunk/doc/funding/B6.wp30_compiler.txt pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt pypy/trunk/doc/funding/B6.wp33_bootstrap_runtime.txt pypy/trunk/doc/funding/B6.wp40_high_performance.txt pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt pypy/trunk/doc/funding/B6.wp50_validations.txt pypy/trunk/doc/funding/B6.wp51_embedded_device.txt pypy/trunk/doc/funding/B6.wp52_small_os_pre.txt pypy/trunk/doc/funding/B6.wp53_small_os.txt pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt pypy/trunk/doc/funding/B6.wp55_numerical_package.txt pypy/trunk/doc/funding/B6.wp56_load_balancing.txt pypy/trunk/doc/funding/B6.wp60_documentation.txt pypy/trunk/doc/funding/B6.wp70_maintenance.txt pypy/trunk/doc/funding/B6.wp80_synchronisation.txt pypy/trunk/doc/funding/uncrappify.py Modified: pypy/trunk/doc/funding/B1.0_objectives.txt pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/crossreferences.asc pypy/trunk/doc/funding/mkxref.py Log: renamed all WPs to B6.7.wp... Modified: pypy/trunk/doc/funding/B1.0_objectives.txt ============================================================================== --- pypy/trunk/doc/funding/B1.0_objectives.txt (original) +++ pypy/trunk/doc/funding/B1.0_objectives.txt Thu Sep 25 11:27:15 2003 @@ -48,7 +48,7 @@ impairing the flexibility. -The flexibility goal +The Flexibility Goal -------------------- The flexibility problem is pernicious. It affects almost all aspects of @@ -62,7 +62,7 @@ interpreter globally. These aspects are usually frozen in early design decisions (memory management, object model, multithreading,...). -* optimization drives the interpreter code base, an error-prone and +* optimisation drives the interpreter code base, an error-prone and scope-limiting process. It is impossible to reuse the code base for implementation on new, non-C platforms; you need to start from scratch. @@ -109,7 +109,7 @@ interpreter at no effort. Such a use will be shown below. -The performance goal, phase 1 +The Performance Goal, Phase 1 ----------------------------- Flexibility is often associated with poor performances, although @@ -144,7 +144,7 @@ source, but derived from it by the translator with the support of a small run-time system written in C. It is thus expected that the translator should be a highly configurable tool, with pluggable -components, that will allow us (and third-parties) to produce customized +components, that will allow us (and third-parties) to produce customised C versions of PyPy with tailored trade-offs. Moreover, the exact set of Python source that the translator will be applied to can be varied as well, producing C versions with different capabilities. @@ -154,7 +154,7 @@ 4. to produce several different C versions for different identified usages of Python; -5. to allow third-parties to produce customized C version easily; +5. to allow third-parties to produce customised C version easily; 6. to adapt the translator to target other non-C platforms (Java and/or .NET and/or other). @@ -195,10 +195,10 @@ levels of a hand-crafted JIT. -The performance goal, phase 2 +The Performance Goal, Phase 2 ----------------------------- -Building on phase 1, we aim to add a JIT compiler to PyPy. This will be +Building upon phase 1, we aim to add a JIT compiler to PyPy. This will be done using a different technique than described in the previous paragraph: our goal is to *generate* the JIT compiler from the high-level Python source of PyPy. @@ -213,19 +213,19 @@ uncommon for JITs, but Psyco's good results give ample proof-of-concept. Moreover, this property can be explicitely related to the DynamoRIO project cited above, which instrumentates the interpreter itself. -Indeed, the one-to-one correspondance between parts of C Python and +Indeed, the one-to-one correspondence between parts of C Python and Psyco is as follows: to each expression in C Python corresponds a more complex expression in Psyco, which does instrumentation and -optimizations. The difference with DynamoRIO is that the latter analyses -at run-time the machine code resulting from the compilation of the -interpreter. In Psyco the one-to-one correspondance is with the C source +optimisations. The difference with DynamoRIO is that the latter analyses +the machine code resulting from the compilation of the interpreter +at run-time. In Psyco, the one-to-one correspondance is with the C source instead. It could certainly have been done by automated analysis of the C source of C Python, but this is difficult and error-prone. -In the PyPy project, on the other hand, the source code of the -interpreter will be in Python instead of C. It is much easier to analyse +In the PyPy project, on the other hand, the source code of the interpreter +will be written in Python instead of C. This is much easier to analyse safely. Thus, the plan is to raise the level further and rely on yet -another customization of the translator, to let it emit the complex +another customisation of the translator, to let it emit the complex Psyco-style expressions instead of (or in addition to) the normal C expressions that would be the direct translation. @@ -234,7 +234,7 @@ 7. to provide a flexible Python interpreter running at JIT speed. -The translator +The Translator -------------- Finally, if this were still needed, let us further demonstrate the @@ -252,12 +252,12 @@ to translate; all manipulated objects are abstracted into their type only. Operations between two such "abstract objects" return a new "abstract object", which represents the type of the result.] As we have -seen above, abstract interpretation is easily acheived by plugging a +seen above, abstract interpretation is easily achieved by plugging a custom object space into the PyPy code base. The translator can be built as an extension of this "type inference" object space, emitting translated code as a side-effect along the analysis. Thus the translator is nothing more than PyPy itself with a custom object space: when you -use this special version of PyPy to "interpret" a Python function, it is -in fact translated... +use this special version of PyPy to "interpret" a Python function, it +becomes in fact translated... In summary, to translate PyPy itself to C, we will be using PyPy! Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Thu Sep 25 11:27:15 2003 @@ -328,14 +328,14 @@ WP55_: Enhanced Numerical Package ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Rewrite a numerical Python package like NumPy -using RPython. Identify the numerical operations -in the package which are candidates for parallelization. -Write an interface that allows to implement these operations -in either a traditional way or using SIMD instructions, -dependent from the available capabilities of the current -code generator. + + Rewrite a numerical Python package like NumPy + using RPython. Identify the numerical operations + in the package which are candidates for parallelization. + Write an interface that allows to implement these operations + in either a traditional way or using SIMD instructions, + dependent from the available capabilities of the current + code generator. Load Balancing in a Multi-Processor environment --------------------------------------------------- Deleted: /pypy/trunk/doc/funding/B6.wp10_management.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp10_management.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,54 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Coordination and Management -.. |wp| replace:: WP10 -.. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 18 -.. |p2| replace:: |e| -.. |m2| replace:: |e| -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Coordination and Management of the PyPy Project. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- collection and monitoring of monthly status reports, -- reporting to the EU -- organising meetings -- maintaining an internal web site. The website will make heavy - use of collaborative tools like Wiki pages, a content management system, - and issue trackers. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- Collection and monitoring of reports -- Report to EU -- Organise meetings -- Create and maintain internal web site - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp20_interpreter.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp20_interpreter.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,46 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: The PyPy Interpreter -.. |wp| replace:: WP20 -.. |start| replace:: 0 -.. |p1| replace:: |e| -.. |m1| replace:: |e| -.. |p2| replace:: |e| -.. |m2| replace:: |e| -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Building a complete PyPy interpreter. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -This working package consists of the tasks WP21_ WP22_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M1 - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp21_interpreter_completion.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,55 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Development and Completion of the PyPy Interpreter -.. |wp| replace:: WP21 -.. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Z -.. |m3| replace:: 3 -.. |p4| replace:: T -.. |m4| replace:: 3 -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -The PyPy interpreter itself should be developed and completed -as a regular Python program. This package includes all -parts of CPython that we don't want to move to WP22_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Coordinate with WP22_ which parts of CPython - should be implemented in RPython. -- Implementation of all C modules and C objects -- Further investigation of the unorthodox multimethod - concepts that the standard object space is based on, - and how to hook in the bytecode compiler. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL01: First complete Python Implementation - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M1 with WP22_ - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp22_cpython_to_python.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp22_cpython_to_python.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,56 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Translation of CPython into regular Python -.. |wp| replace:: WP22 -.. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Z -.. |m3| replace:: 3 -.. |p4| replace:: T -.. |m4| replace:: 3 -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Translate all parts of CPython which we don't want to -implement in WP21_ into regular Python libraries. -Note: These ones should also work without PyPy, being just plain-Python -replacements for existing CPython functionality. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Coordinate with WP21_ which parts of CPython should be implemented in Python -- Implementation of all C modules and C objects -- Implement the Python bytecode compiler, which definately should - become a regular Python program, instead of being built - into the interpreter. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL01: First complete Python Implementation -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M1 with WP21_ - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp30_compiler.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp30_compiler.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,46 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: The PyPy Compiler -.. |wp| replace:: WP30 -.. |start| replace:: 0 -.. |p1| replace:: |e| -.. |m1| replace:: |e| -.. |p2| replace:: |e| -.. |m2| replace:: |e| -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Building the translator and Bootstrapping PyPy. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -This working package consists of the tasks WP31_ WP32_ WP33_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M2 - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp31_translation_of_rpython.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,72 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Translation of RPython -.. |wp| replace:: WP31 -.. |start| replace:: M1 -.. |p1| replace:: X -.. |m1| replace:: 4 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Ability to translate arbitrary RPython source code (e.g. - the one produced in WP21_ into low-level code (C, Pyrex, Java, others). -- Creation of a stand-alone, not-PyPy-related tool for general - optimization of arbitrary but suitably restricted Python application or - parts thereof. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -*Part a): Code Analysis* - -- Analyse code to produce the relevant typing information. -- Investigate whether we can use the annotation object space - only or if additional AST-based control flow analysis is needed. -- Give a formal definition of RPython. - -*Part b): Code Production* - -- Compare different low-level environments that - we could target (C, Pyrex, others?). Select at least - one suitable target. -- Produce low-level code out of the data gathered in Part a). -- Again investigate how this is best done (AST-guided translation or - reverse-engeneering of the low-level control flow gathered by the - annotation object space). -- Provide some measure for the efficiency of the generated - code, in comparison to Standard Python. - -Note that this task can actually start earlier than M1, but -finally becomes dependent from it. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL???: Tool for Analysis of restricted Python Programs -- DL???: Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- Initial Complete Self-Contained PyPy Interpreter -- Initial Measures of Efficiency Deleted: /pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp32_bootstrap_redesign.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,57 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Specific Analysis and Redesign -.. |wp| replace:: WP32 -.. |start| replace:: WP31_ -.. |p1| replace:: X -.. |m1| replace:: 4 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Putting interpreter (WP21_, WP22_) and translator (WP31_) together. -- Enhancing the PyPy Design. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Part a): Specific Analysis and Redesign** - -- Investigate the particular problems that the - global translation of PyPy is going to raise. -- Enhance and redesign parts of PyPy as needed - to give best support to the translation of RPython. -- Re-iterate this until we agree to have reached - an optimum solution. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M2 with WP33_ - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp33_bootstrap_runtime.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp33_bootstrap_runtime.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,62 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Low Level PyPy Runtime -.. |wp| replace:: WP33 -.. |start| replace:: WP31_ -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Providing a Runtime Library for PyPy - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -In order to give a working environment to the translated RPython -program, we need to build the low-level-specific **run-time** -components of PyPy. - -- Design the object layout and implement it. -- Decide about memory management and implement it. -- Consider threading support and implement it. Make sure - that the other components are thread-aware. -- Design and implement an efficient multimethod dispatch. - -The target language is not yet decided on. We might go different -paths at the same time. -If producing C code is a target, important -parts can be directly re-used from CPython. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -DL??? Runtime Library for PyPy -DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M2 with WP32_ - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp40_high_performance.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp40_high_performance.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,50 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: High-performance PyPy-Python -.. |wp| replace:: M2 -.. |start| replace:: 0 -.. |p1| replace:: |e| -.. |m1| replace:: |e| -.. |p2| replace:: |e| -.. |m2| replace:: |e| -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Optimizing the Bootstrapping in various ways. - -Exploring several Object Implementations. - -Folding Psyco and Stackless into PyPy. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -This working package consists of the tasks WP41_ WP42_ WP43_ WP44_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M4 - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp41_integrate_stackless.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,65 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Integration of Stackless -.. |wp| replace:: WP41 -.. |start| replace:: M2 -.. |p1| replace:: CT -.. |m1| replace:: 5 -.. |p2| replace:: Y -.. |m2| replace:: 2 -.. |p3| replace:: Z -.. |m3| replace:: 2 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Integration of Stackless's Technology into PyPy. - -- Enable Massive Parallelism in a Single Thread - -- Allow Pickling of a Running Program - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Determine which parts of Stackless need to be written in RPython. - -- Design Stackless as a meta-component - that hooks into the translator (WP31_) - -- Implement Stackless partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP33_) for Stackless. - -- Implement a single-threaded, pre-emptive scheduler with - priorities. This is necessary to support small OSes, like WP53_. - -- Further develop these technologies based on the results gathered in WP43_, - e.g. identify when these technologies would guide specific choices among the - solutions developed in WP33_ and studying several solutions in WP42_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Green threads, pickling. -- DL??? Passing all applicable elements of the standard test suite. - -**Milestones and Expected Result** - -- M3 - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp42_several_obj_impl.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,73 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Several Object Implementations -.. |wp| replace:: WP42 -.. |start| replace:: M2 -.. |p1| replace:: X -.. |m1| replace:: 4 -.. |p2| replace:: Y -.. |m2| replace:: 4 -.. |p3| replace:: Z -.. |m3| replace:: 4 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Providing several object implementations for - - general optimisation - - application specific alternatives - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -The following objects are mostly independent from any particular low-level run-time -environment. - -* Implement dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, - or with string-only or integer-only keys. Dictionaries with specific - support for "on-change" callbacks (useful for Psyco). - -* Implement strings as plain immutable memory buffers vs. immutable but more - complex data structures (see functional languages) vs. internally - mutable data structures (e.g. Psyco's concatenated strings) - -* Implement ints as machine words vs. two machine words vs. internal longs vs. - external bignum library (investigate if completely unifying ints and - longs is possible in the Python language at this stage). - -* Consider further objects and do multiple implementations of them. - -* Run various performance tests on the above, check for an - overall improvement or application dependency. - -* Create reports and merge the results back into the optimization effort. - As of general interest, submit the reports as publication. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Publication. -- DL??? Passing all applicable elements of the standard test - suite for every tested alternative. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M4 with M3 WP43_ WP44_ - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp43_translator_optimisations.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,79 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Translator Optimisations -.. |wp| replace:: WP43 -.. |start| replace:: M2 -.. |p1| replace:: X -.. |m1| replace:: 6 -.. |p2| replace:: Y -.. |m2| replace:: 6 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Identification and Implementation of Optimisations - through modifications of the Translator. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -The following alternatives are mostly specific to a particular low-level run-time. - -* Implement and compare different object layout and memory management strategy - or strategies, e.g. reference counting vs. Boehm garbage collection vs. our own. - Includes speed vs. data size trade-offs. - -* Study approaches concerning code size vs. speed trade-offs (e.g. whether the - final interpreter should still include compact precompiled bytecode or be - completely translated into C). - -* Solve the complex issue of threading (global interpreter lock vs. - alternatives). - -* Enhance multimethod dispatching. - -* Implement schemes of pointer tagging, e.g. encoding an integer object as a pointer with a - special value instead of a real pointer to a data structure representing - the integer. - -* Run various performance tests on the above, check for an - overall improvement or application dependency. - -* For each issue, work on **several solutions** when no one is obviously - better than the other ones. - -* Create reports and merge the results back into the optimization effort. - As of general interest, submit the reports as publication. - -Note that compatibility with legacy C extensions can be achieved by choosing, -for each of the following issues, the same one as CPython did. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Publication. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M4 with M3 WP42_ WP44_ - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp44_integrate_psyco.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,64 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Integration of Psyco -.. |wp| replace:: WP44 -.. |start| replace:: M2 -.. |p1| replace:: AR -.. |m1| replace:: 6 -.. |p2| replace:: Y -.. |m2| replace:: 2 -.. |p3| replace:: Y -.. |m3| replace:: 2 -.. |p4| replace:: Z -.. |m4| replace:: 2 -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Integration of Psyco's technology into PyPy. - - - Optimization by Creating Specialized Machine Code at Run-time. - - Adressing new Processor Architectures - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Determine which parts of Psyco need to be written in RPython. - -- Design Psyco as a meta-component - that hooks into the translator (WP31_) - -- Implement Psyco partially in RPython, partially in Python, - favoring the latter. - -- Implement a dedicated run-time component (WP33_) for Psyco. - -- Further develop these technologies based on the results gathered in WP43_, - e.g. identify when these technologies would guide specific choices among the - solutions developed in WP33_ and studying several solutions in WP42_. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? to be defined. Speed boost, architectures. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M4 with M3 WP42_ WP43_ - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp50_validations.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp50_validations.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,48 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Validation of PyPy -.. |wp| replace:: WP50 -.. |start| replace:: M4/M3 -.. |p1| replace:: |e| -.. |m1| replace:: |e| -.. |p2| replace:: |e| -.. |m2| replace:: |e| -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Validation of the PyPy concept by proving its applicability -in real applications. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -This working package consists of the tasks WP51_ WP52_ WP53_ WP54_ WP55_ WP56_ -or a subset thereof, dependent of feasibility. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M6 M7 M8 M9 - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp51_embedded_device.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp51_embedded_device.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,68 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Porting PyPy to an Embedded Device -.. |wp| replace:: WP51 -.. |start| replace:: M2 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Y -.. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Supporting a completely different platform -- Providing an efficient, high-level Language for - embedded Devices -- Proving the flexibility of the PyPy idea - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Analyse the differences between the new architecture and - PyPy's current. Deside whether to optimize for code size - or applicatin speed. - -- Develop the translation process, run-time and those optimisations - that depend on low-level details. - -- Implement a simulator for the embedded device. - -- Implement interfaces to the needed external libraries. - -- Combine different solutions for the different issues discussed in - High-performance (WP42_, WP43_, WP44_). - Gather statistics with real-work Python applications. - Compare the results. This is where the flexibility of the whole - project is vital. Typically, very different trade-offs need to be - made on different environments. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? Python running on the embedded device. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M5 - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp52_small_os_pre.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp52_small_os_pre.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,54 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Evaluation whether to do a small OS in PyPy -.. |wp| replace:: WP52 -.. |start| replace:: M5 -.. |p1| replace:: CT -.. |m1| replace:: 1 -.. |p2| replace:: Y -.. |m2| replace:: 1 -.. |p3| replace:: Y -.. |m3| replace:: 1 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Deciding whether to build a small Operating System - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Analyse the existing device drivers for the embedded device. - Figure out which drivers should be interfaced to, which - parts of the BIOS must be interfaced, and what has to be - recoded in Python. At least all drivers which have - blocking calls must be recoded in in Python. - -- Check whether we need to implement a TCP/IP stack and if this - work is doable in reasonable time. If not, then cancel the - next WP in favor of another validation WP to be done instead. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M6 - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp53_small_os.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp53_small_os.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,64 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: A small Operating System for an Embedded Device -.. |wp| replace:: WP53 -.. |start| replace:: M6 -.. |p1| replace:: CT -.. |m1| replace:: 5 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Y -.. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -- Building all parts of a small Operating System - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Recode device drivers in Python, according to the results of WP52_. - Design tests which ensure functional compatibility with the original - ones, as far as these exist. - -- Extend the simulator of WP51_ accordingly. - -- Implement the OS kernel as a set of prioritized tasklets - (using the support from WP41_) - -- Implement a simple file system which is accessible by - Python, only. - -- Implement a few basic protocols like FTP in order to provide - basic access to the internal data of the embedded device. - -- Test the implementation both on the simulator and on the real hardware. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- DL??? Python implements the whole OS of the embedded device. -- DL??? Passing all applicable elements of the standard test suite. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M7 - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp54_simd_architecture.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,69 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Code Generator for SIMD Architecture -.. |wp| replace:: WP54 -.. |start| replace:: M4 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Z -.. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Extending PyPy towards general support of Single Instruction Multiple -Data (SIMD) architectures. -Building a concrete implementation for one choosen SIMD architecture. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Select a processor architecture which has SIMD extensions, - like SSE2 or AltiVec. - Preference should be given to a technology that we are - supporting, already. - -- Examine the necessary design changes/extensions to PyPy for supporting - parallel instructions. - -- Extend the code generator to support - and optimize for the new instructions. - This involves re-iterations of parts of WP32_ and WP43_. - Extending Psyco involves WP44_ as well. - -- Extend PyPy by an uto-probing feature that tries to - examine the availability of certain extensions at runtime. - -- Design and implement the necessary vectorised data types - for Python/RPython. The extra types should be made sufficient to - enable us to write parallel extension modules, and to - write the new testing programs. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- Parallel extension for PyPy - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- Extended optimisation towards parallel numerics. - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp55_numerical_package.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp55_numerical_package.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,61 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Enhanced Numerical Package -.. |wp| replace:: WP55 -.. |start| replace:: WP54 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Building an enhanced numerical package on top of the SIMD extensions of PyPy. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Decide which numerical extension package to re-implement in PyPy. - -- Re-implement the numeric package in terms of applying the - new vectorized operations. - -- Make sure that the new implementation passes all tests. - -- Provide backward-compatibility of the package in a way that - allows it to fall back to a simpler architecture when there - is no SIMD extension available. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- Enhanced numerical package - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M8 - -- We expect to at least reach the efficiency of the existing numerical - packages. The target is to outperform these and to get a peformance that - is comparable to the efficiency of optimised parallel C libraries. - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp56_load_balancing.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp56_load_balancing.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,52 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Load Balancing in a Multi-Processor environment -.. |wp| replace:: WP56 -.. |start| replace:: M3, M4 -.. |p1| replace:: X -.. |m1| replace:: 3 -.. |p2| replace:: Y -.. |m2| replace:: 3 -.. |p3| replace:: Z -.. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Implement load balancing for distributed applications. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -*Part a):* - - - work with stackless results *FIXME: write this* - -*Part b):* - - - use the whole optimized thing *FIXME: write this* - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- M9 - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp60_documentation.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp60_documentation.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,56 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Project Documentation and Dissemination -.. |wp| replace:: WP60 -.. |start| replace:: 0 -.. |p1| replace:: AM -.. |m1| replace:: 19 -.. |p2| replace:: JH -.. |m2| replace:: 3 -.. |p3| replace:: MWH -.. |m3| replace:: 3 -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Providing ongoing documentation throughout the whole project. - -Keeping external groups informed of the ongoings in PyPy. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- During the whole project, maintain and extend a set of documents - represends the current status of the project, results of - discussions, the planning of new sprints and their status reports, - as well as the preparation of papers for presentation on - various congresses. - -- Send reports of ongoings in the project to the Python Business - Forum (PBF) and the Python - developers list (python-dev at python.org). - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -- Good documentation and dissemination - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp70_maintenance.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp70_maintenance.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,58 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Maintenance of Tools -.. |wp| replace:: WP70 -.. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 5 -.. |p2| replace:: Y -.. |m2| replace:: 5 -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Supporting the PyPy project by maintaining and enhancing the tools. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Keep the infrastructure up-to-date and develop it further. - - - maintain the Subversion package and track its ongoing development - - - maintain the Subversion repository - - - build extensions for automatic document extraction - - - support email notification via mailing lists - -- Provide a mirror repository which is kept up-to-date and which - can be used in case of failure of the main repository. - -- Take care of regular repository backups. Design a strict backup - policy suitable for the project, and adhere to it. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Deleted: /pypy/trunk/doc/funding/B6.wp80_synchronisation.txt ============================================================================== --- /pypy/trunk/doc/funding/B6.wp80_synchronisation.txt Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,60 +0,0 @@ -.. include:: crossreferences.asc - -.. |title| replace:: Synchronisation with Standard Python -.. |wp| replace:: WP80 -.. |start| replace:: 0 -.. |p1| replace:: X -.. |m1| replace:: 6 -.. |p2| replace:: |e| -.. |m2| replace:: |e| -.. |p3| replace:: |e| -.. |m3| replace:: |e| -.. |p4| replace:: |e| -.. |m4| replace:: |e| -.. |p5| replace:: |e| -.. |m5| replace:: |e| -.. |p6| replace:: |e| -.. |m6| replace:: |e| - -.. include:: wp-toptable.asc - -.. include:: wp-tablebegin.asc - -**Objectives** - -Keeping PyPy in sync with the ongoing changes to Standard Python. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Description of work** - -- Monitor the development of Standard Python for its ongoing changes. - -- Check the relevance of the changes concerning the structure of - PyPy. - -- Embed trivial changes with proper testing, notifying the other - concurrent tasks of these changes. - -- Changes which impose larger structural changes must influence - the overall design and be discussed with the whole group. - It is the responsibility of this task to gather such information - as early as possible. - -- Also investigate whether there are tools which could automate - integration of simple changes, partially. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Deliverables** - -- Running all tests of PyPy and Stadard Python after every update. - -.. include:: wp-tableend.asc -.. include:: wp-tablebegin.asc - -**Milestones and Expected Result** - -.. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/crossreferences.asc (original) +++ pypy/trunk/doc/funding/crossreferences.asc Thu Sep 25 11:27:15 2003 @@ -1,25 +1,25 @@ -.. _WP10: B6.wp10_management.html -.. _WP20: B6.wp20_interpreter.html -.. _WP21: B6.wp21_interpreter_completion.html -.. _WP22: B6.wp22_cpython_to_python.html -.. _WP30: B6.wp30_compiler.html -.. _WP31: B6.wp31_translation_of_rpython.html -.. _WP32: B6.wp32_bootstrap_redesign.html -.. _WP33: B6.wp33_bootstrap_runtime.html -.. _WP40: B6.wp40_high_performance.html -.. _WP41: B6.wp41_integrate_stackless.html -.. _WP42: B6.wp42_several_obj_impl.html -.. _WP43: B6.wp43_translator_optimisations.html -.. _WP44: B6.wp44_integrate_psyco.html -.. _WP50: B6.wp50_validations.html -.. _WP51: B6.wp51_embedded_device.html -.. _WP52: B6.wp52_small_os_pre.html -.. _WP53: B6.wp53_small_os.html -.. _WP54: B6.wp54_simd_architecture.html -.. _WP55: B6.wp55_numerical_package.html -.. _WP56: B6.wp56_load_balancing.html -.. _WP60: B6.wp60_documentation.html -.. _WP70: B6.wp70_maintenance.html -.. _WP80: B6.wp80_synchronisation.html +.. _WP10: B6.7.wp10_management.html +.. _WP20: B6.7.wp20_interpreter.html +.. _WP21: B6.7.wp21_interpreter_completion.html +.. _WP22: B6.7.wp22_cpython_to_python.html +.. _WP30: B6.7.wp30_compiler.html +.. _WP31: B6.7.wp31_translation_of_rpython.html +.. _WP32: B6.7.wp32_bootstrap_redesign.html +.. _WP33: B6.7.wp33_bootstrap_runtime.html +.. _WP40: B6.7.wp40_high_performance.html +.. _WP41: B6.7.wp41_integrate_stackless.html +.. _WP42: B6.7.wp42_several_obj_impl.html +.. _WP43: B6.7.wp43_translator_optimisations.html +.. _WP44: B6.7.wp44_integrate_psyco.html +.. _WP50: B6.7.wp50_validations.html +.. _WP51: B6.7.wp51_embedded_device.html +.. _WP52: B6.7.wp52_small_os_pre.html +.. _WP53: B6.7.wp53_small_os.html +.. _WP54: B6.7.wp54_simd_architecture.html +.. _WP55: B6.7.wp55_numerical_package.html +.. _WP56: B6.7.wp56_load_balancing.html +.. _WP60: B6.7.wp60_documentation.html +.. _WP70: B6.7.wp70_maintenance.html +.. _WP80: B6.7.wp80_synchronisation.html .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file Modified: pypy/trunk/doc/funding/mkxref.py ============================================================================== --- pypy/trunk/doc/funding/mkxref.py (original) +++ pypy/trunk/doc/funding/mkxref.py Thu Sep 25 11:27:15 2003 @@ -9,7 +9,7 @@ # find all work package files, like # B6.wp02_cpython_to_python.txt -START_STR = "B6.wp".lower() +START_STR = "B6.7.wp".lower() def get_wps_from_dir(as_html=False): res = {} Deleted: /pypy/trunk/doc/funding/uncrappify.py ============================================================================== --- /pypy/trunk/doc/funding/uncrappify.py Thu Sep 25 11:27:15 2003 +++ (empty file) @@ -1,14 +0,0 @@ -# very simple repair to Armin's mess :-) - -import os, sys - -if sys.platform != "win32": - raise TypeError, "this won't work on %s" % sys.platform - -for fname in os.listdir("."): - pre, ext = os.path.splitext(fname) - if ext.lower() in (".txt", ".py"): - txt = file(fname, "rb").read() - txt = txt.replace("\r\n\n\r\n\n", "\r\n") - file(fname, "wb").write(txt) - \ No newline at end of file From arigo at codespeak.net Thu Sep 25 13:35:08 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Sep 2003 13:35:08 +0200 (MEST) Subject: [pypy-svn] rev 1417 - pypy/trunk/src/pypy/interpreter Message-ID: <20030925113508.A9FD15A271@thoth.codespeak.net> Author: arigo Date: Thu Sep 25 13:35:08 2003 New Revision: 1417 Modified: pypy/trunk/src/pypy/interpreter/error.py Log: automatically enter pdb upon OperationError when the environment variable $PYPY_DEBUG is set Modified: pypy/trunk/src/pypy/interpreter/error.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/error.py (original) +++ pypy/trunk/src/pypy/interpreter/error.py Thu Sep 25 13:35:08 2003 @@ -1,6 +1,6 @@ -import sys +import os, sys -AUTO_DEBUG = 0 +AUTO_DEBUG = os.getenv('PYPY_DEBUG') class PyPyError(Exception): From pedronis at codespeak.net Thu Sep 25 18:50:08 2003 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 25 Sep 2003 18:50:08 +0200 (MEST) Subject: [pypy-svn] rev 1418 - pypy/trunk/src/pypy Message-ID: <20030925165008.A83D45ACAE@thoth.codespeak.net> Author: pedronis Date: Thu Sep 25 18:50:08 2003 New Revision: 1418 Modified: pypy/trunk/src/pypy/test_all.py Log: testing commit, ignore Modified: pypy/trunk/src/pypy/test_all.py ============================================================================== --- pypy/trunk/src/pypy/test_all.py (original) +++ pypy/trunk/src/pypy/test_all.py Thu Sep 25 18:50:08 2003 @@ -1,4 +1,5 @@ import tool.autopath + from pypy.tool import test if __name__ == '__main__': From pedronis at codespeak.net Thu Sep 25 18:51:01 2003 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 25 Sep 2003 18:51:01 +0200 (MEST) Subject: [pypy-svn] rev 1419 - pypy/trunk/src/pypy Message-ID: <20030925165101.84C655ACAE@thoth.codespeak.net> Author: pedronis Date: Thu Sep 25 18:51:01 2003 New Revision: 1419 Modified: pypy/trunk/src/pypy/test_all.py Log: remove spurious line, ignore Modified: pypy/trunk/src/pypy/test_all.py ============================================================================== --- pypy/trunk/src/pypy/test_all.py (original) +++ pypy/trunk/src/pypy/test_all.py Thu Sep 25 18:51:01 2003 @@ -1,5 +1,4 @@ import tool.autopath - from pypy.tool import test if __name__ == '__main__': From lac at codespeak.net Thu Sep 25 23:16:19 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Thu, 25 Sep 2003 23:16:19 +0200 (MEST) Subject: [pypy-svn] rev 1422 - pypy/trunk/doc/funding Message-ID: <20030925211619.35E2C5ACAE@thoth.codespeak.net> Author: lac Date: Thu Sep 25 23:16:18 2003 New Revision: 1422 Modified: pypy/trunk/doc/funding/B0.0_preamble.txt Log: First draft, unfinished, for comment. See if I have the shape about right at any rate. Feel free to change things, but see if you can keep the buzz words from the IST-2002-2.3.2.3 call. Modified: pypy/trunk/doc/funding/B0.0_preamble.txt ============================================================================== --- pypy/trunk/doc/funding/B0.0_preamble.txt (original) +++ pypy/trunk/doc/funding/B0.0_preamble.txt Thu Sep 25 23:16:18 2003 @@ -36,11 +36,22 @@ Part B. -Front page -Proposal full title -Proposal acronym -Date of preparation -Type of instrument in this case: Specific targeted research project +*** Front page -- PAGE 1 *** + + PYPY + + A flexible, modular, self-hosting, next-generation + specialising compiler for the Open Source Programming + Language Python + + +Proposal Part B + +Date of preparation: October 2004 + + A Specific Targeted Research Project for: IST-2002-2.3.2.3 - Open + development platforms for software and services + List of participants Participant no. Participant name Participant short name @@ -55,15 +66,74 @@ Coordinator email Coordinator fax -Contents page *show contents list* +*** END OF PAGE 1 *** + +*** Contents page Page 2 *** +*show contents list* +*** END OF PAGE 2 *** + +*** Proposal Summary Page Page 3 *** -Proposal summary page Proposal full title + + The full title of the Proposal is 'Pypy: A flexible, modular, + self-hosting, next-generation specialising compiler for the + Open Source Programming Language Python'. + Proposal acronym -Strategic objectives addressed - (If more than one objective , indicate their order of importance - to the project). + The proposal acronym is PYPY. + +Strategic Objectives Addressed: + + This proposal directly addresses the Strategic Objective + IST-2002-2.3.2.3 - 'Open development platforms for software and + services' of the Second Call of the Information Society + Technologies Workprogramme. Since this is foundational applied + research, focusing on fundamental design concepts, it will have + significant indirect effects wherever the Python programming + langauge is taught or deployed. It will reduce the current wide + gap between academic computer science and industrial programming + tools, thus contributing to European competitiveness and prosperity, + and enhancing openness and security in the public and private + sector. + +Proposal abstract copied from Part A + +(Actually, we will paste this one INTO Part A) + +The PyPy project will build a flexible, modular, context-aware +implementation of the Open Source programming language Python written +in itself. Python is already a very popular Object-Oriented +Very-High-Level interreted programming language. + +(FN -- Usage statistics indicate that Python ranks sixth in usage +world-wide, after Java, Visual Basic, C, C++, and Perl) + +But the current state-of-the-Art, both in Python and in Computer +Languages in general does not best serve the new needs of the creators +of embedded, networked and distributed software sectors. They have +long needed a language which was more flexible, easier to reduce to +its 'bare-bones' for embedding, and which could dynamically +reconfigure itself and balance loads at the interpreter level. +Consequently, the PyPy project is a collaboration between academic +researchers, who have the skill and vision to produce a new runtime +language architecture for the twenty-first century, SME software and +service providers, who wish to deploy PyPy in their innovative new +business ventures, and older established industrial users who would +like to embed a PyPy application on their ubiquitous products, such as +home entertainment systems or commercial computer printers. + +The project will go through four phases. In the first phase, we will +build a complete PyPy Interpreter. In the second phase, we will build +a complete PyPy compiler that compiles itself. While our new +architecture of Object Spaces can here be considered innovative, this +two phases can be looked at as a way to only produce a state-of-the-Art +implementation of the Python programming language. We will go beyond +this in Phase three, and produce a High-Performance PyPy, which goes +beyond the State of the Art and particularily addresses our modern +needs. Finally in Phase four, we will Validate our design choices +by .... + -Proposal abstract copied from Part A -(if not in English, include an English translation) +*** END OF PAGE 3 *** \ No newline at end of file From dinu at codespeak.net Fri Sep 26 15:44:29 2003 From: dinu at codespeak.net (dinu at codespeak.net) Date: Fri, 26 Sep 2003 15:44:29 +0200 (MEST) Subject: [pypy-svn] rev 1423 - pypy/trunk/doc/objspace Message-ID: <20030926134429.E829D5A281@thoth.codespeak.net> Author: dinu Date: Fri Sep 26 15:44:29 2003 New Revision: 1423 Modified: pypy/trunk/doc/objspace/restrictedpy.txt Log: fixed some typos Modified: pypy/trunk/doc/objspace/restrictedpy.txt ============================================================================== --- pypy/trunk/doc/objspace/restrictedpy.txt (original) +++ pypy/trunk/doc/objspace/restrictedpy.txt Fri Sep 26 15:44:29 2003 @@ -2,7 +2,7 @@ Restricted Python ================== -We are writing a Python interpreter in Python, using Python's well known ability to step behind the algorithmic problems as language. At first glance, one might think this achives nothing but a better understanding for everbody how the interpreter works. This alone would make it worth doing, but we have much larger goals. +We are writing a Python interpreter in Python, using Python's well known ability to step behind the algorithmic problems as language. At first glance, one might think this achieves nothing but a better understanding for everybody how the interpreter works. This alone would make it worth doing, but we have much larger goals. CPython vs. PyPy From dinu at codespeak.net Fri Sep 26 15:59:41 2003 From: dinu at codespeak.net (dinu at codespeak.net) Date: Fri, 26 Sep 2003 15:59:41 +0200 (MEST) Subject: [pypy-svn] rev 1424 - pypy/trunk/doc/objspace Message-ID: <20030926135941.D91875AC58@thoth.codespeak.net> Author: dinu Date: Fri Sep 26 15:59:41 2003 New Revision: 1424 Modified: pypy/trunk/doc/objspace/multimethod.txt pypy/trunk/doc/objspace/objspace.txt pypy/trunk/doc/objspace/restrictedpy.txt pypy/trunk/doc/objspace/stdobjspace.txt Log: fixed some typos Modified: pypy/trunk/doc/objspace/multimethod.txt ============================================================================== --- pypy/trunk/doc/objspace/multimethod.txt (original) +++ pypy/trunk/doc/objspace/multimethod.txt Fri Sep 26 15:59:41 2003 @@ -40,11 +40,11 @@ Delegation --------------- -Delegation is a transparent convertion mechanism between object -implementations. The convertion can give a result of a different type +Delegation is a transparent conversion mechanism between object +implementations. The conversion can give a result of a different type (e.g. int -> float) or of the same type (e.g. W_VeryLongString -> str). There is a global table of delegators. We should not rely on the delegators -to be tried in any particlar order, or at all (e.g. the int -> float delegator +to be tried in any particular order, or at all (e.g. the int -> float delegator could be ignored when we know that no registered function will accept a float anyway). @@ -62,7 +62,7 @@ and the Method Resolution Order are defined, and where attribute look-ups are done. -Instances of user-defined types are implementated as W_UserObjects. +Instances of user-defined types are implemented as W_UserObjects. A user-defined type can inherit from built-in types (maybe more than one, although this is incompatible with CPython). The W_UserObject delegator converts the object into any of these "parent objects" if needed. This is @@ -136,7 +136,7 @@ examples, respectively, C=type(l) and C=int). The restriction is based on the registered class of the first argument ('self' for the new method) in the signature. If this class corresponds to a fixed -type (as advertized by 'statictype'), and this fixed type is C or a +type (as advertised by 'statictype'), and this fixed type is C or a superclass of C, then we keep it. Some multimethods can also be sliced along their second argument, Modified: pypy/trunk/doc/objspace/objspace.txt ============================================================================== --- pypy/trunk/doc/objspace/objspace.txt (original) +++ pypy/trunk/doc/objspace/objspace.txt Fri Sep 26 15:59:41 2003 @@ -9,7 +9,7 @@ + the compiler, which represents the static optimization of the source code into an intermediate format, the bytecode; + the object library, implementing the various types of objects and their semantics; -+ the main loop, which suffles data around and calls the operations defined in the object library according to the bytecode. ++ the main loop, which shuffles data around and calls the operations defined in the object library according to the bytecode. The main loop generally assumes little about the semantics of the objects: they are essentially black boxes (PyObject pointers). The interpreter stack and the variables only contain such black boxes. Every operation is done via calls to the object library, like PyNumber_Add(). @@ -24,7 +24,7 @@ (We should also talk about turning the compiler and the main loop into explicit concepts, as Rocco discussed in pypy-dev.) -I believe that the above examples should give a hint at why we might really want object spaces that behave differently: this is called abstact interpretation in the litterature. But there are also more funny ways to use object spaces, for example: +I believe that the above examples should give a hint at why we might really want object spaces that behave differently: this is called abstract interpretation in the literature. But there are also more funny ways to use object spaces, for example: + running with several object spaces at once, to clearly separate the data into domains + proxy object spaces, which forward all requests over a network to a "Python object server" Modified: pypy/trunk/doc/objspace/restrictedpy.txt ============================================================================== --- pypy/trunk/doc/objspace/restrictedpy.txt (original) +++ pypy/trunk/doc/objspace/restrictedpy.txt Fri Sep 26 15:59:41 2003 @@ -8,13 +8,13 @@ CPython vs. PyPy ------------------- -Compared to the CPython implementation, Python takes the role of the C Code. So actually, we descibe something by Python, which has been coded in C already, with all the restrictions that are implied by C. We are not trying to make the structures of the CPython interpreter more flexible by rewriting things in C, but we want to use Python to give an alternative description of the interpreter. +Compared to the CPython implementation, Python takes the role of the C Code. So actually, we describe something by Python, which has been coded in C already, with all the restrictions that are implied by C. We are not trying to make the structures of the CPython interpreter more flexible by rewriting things in C, but we want to use Python to give an alternative description of the interpreter. The clear advantage is that this description is probably shorter and simpler to read, and many implementation details vanish. The drawback of this approach is that this interpreter will be unbearably slow. To get to a useful interpreter again, we need to apply some mappings to the implementation, later. One rather straight-forward is to do a whole program analysis of the PyPy interpreter and create a C source, again. There are many other ways, but let's stick with the easiest approach, first. -In order to make a C code generator simple, we resrict ourselves to a subset of the Python language, and we adhere to some rules, which make code generation obvious and easy. +In order to make a C code generator simple, we restrict ourselves to a subset of the Python language, and we adhere to some rules, which make code generation obvious and easy. Restricted Python is Runtime Python @@ -36,7 +36,7 @@ RPython Definition -------------------- -It might make sense to define a sublanguage of Python called RPython, with the restrictions depicted below. This is an evolving topic, and we're just collecting things which come up during trying to code the interpreter, so this is no language at all, but an arbitrary set of rules, which are about to be changed all day. +It might make sense to define a sub-language of Python called RPython, with the restrictions depicted below. This is an evolving topic, and we're just collecting things which come up during trying to code the interpreter, so this is no language at all, but an arbitrary set of rules, which are about to be changed all day. Object restrictions @@ -71,7 +71,7 @@ **functions** + statically called functions may use defaults and a variable number of arguments (which may be passed as a list instead of a tuple, so write code that does not depend on it being a tuple). -+ dynamic dispatch enforces use of very simple signatures, equal for all functions to be called in that context. At the moment, this occours in the opcode dispatch, only. ++ dynamic dispatch enforces use of very simple signatures, equal for all functions to be called in that context. At the moment, this occurs in the opcode dispatch, only. **builtin functions** A few builtin functions will be used, while this set is not defined completely, yet. Some builtin functions are special forms: @@ -83,7 +83,7 @@ + may be used with basic types that have a length. But len is a special form that is recognized by the compiler. + If a certain structure is never touched by len, the compiler might save the length field from the underlying structure. -``int, float, ord, chr``... are available as simple convertion functions. +``int, float, ord, chr``... are available as simple conversion functions. ``int, float, str``... have a special meaning as a type inside of isinstance only. **classes** @@ -105,7 +105,7 @@ Example: Integer Types ------------------------- -While implementing the integer type, I (Chris) stumbled over the problem, that integers are quite in flux in CPython right now. Depending on the version, integers either raise an overflow exception or mutate into longs on overflow. Also, shifting right now truncates (upto 2.3) but is going to extend to longs as well. In order to enable us to run the restricted Python stuff in CPython, I needed to use a derived class r_int(int), which always behaves the same: Never leaving its domain, but always staying an integer. +While implementing the integer type, I (Chris) stumbled over the problem, that integers are quite in flux in CPython right now. Depending on the version, integers either raise an overflow exception or mutate into longs on overflow. Also, shifting right now truncates (up to 2.3) but is going to extend to longs as well. In order to enable us to run the restricted Python stuff in CPython, I needed to use a derived class r_int(int), which always behaves the same: Never leaving its domain, but always staying an integer. The r_int type is implemented in a pervasive way: Every operation that involves an r_int creates an r_int as the result. Therefore, the implementation is not cluttered with special type casts. Just the initial value of an emulated integer's intval field is initialized by obj.intval = r_int(val) . This way, the r_int type propagates itself through all operations without extra effort of the programmer. Modified: pypy/trunk/doc/objspace/stdobjspace.txt ============================================================================== --- pypy/trunk/doc/objspace/stdobjspace.txt (original) +++ pypy/trunk/doc/objspace/stdobjspace.txt Fri Sep 26 15:59:41 2003 @@ -4,7 +4,7 @@ The StandardObjectSpace is the direct equivalent of CPython's object library (the "Objects/" subdirectory in the distribution). It is an implementation of the common Python types in a lower-level language. -The StandardObjectSpace defines an abstract parent class, W_Object, and a bunch of subclasses like W_IntObject, W_ListObject, and so on. A wrapped object (a "black box" for the interpreter main loop) is thus an instance of one of these classes. When the main loop invokes an operation, say the addition, between two wrapped objects w1 and w2, the StandardObjectSpace does some internal dispatching (similar to "Object/ abstract.c" in CPython) and invokes a method of the proper W_XyzObject class that can do the operation. The operation itself is done with the primitives allowed by RestrictedPython. The result is constructed as a wrapped object again. For example, compare the following implementation of integer addition with the function "int_add()" in "Object/intobject.c": :: +The StandardObjectSpace defines an abstract parent class, W_Object, and a bunch of subclasses like W_IntObject, W_ListObject, and so on. A wrapped object (a "black box" for the interpreter main loop) is thus an instance of one of these classes. When the main loop invokes an operation, say the addition, between two wrapped objects w1 and w2, the StandardObjectSpace does some internal dispatching (similar to "Object/abstract.c" in CPython) and invokes a method of the proper W_XyzObject class that can do the operation. The operation itself is done with the primitives allowed by RestrictedPython. The result is constructed as a wrapped object again. For example, compare the following implementation of integer addition with the function "int_add()" in "Object/intobject.c": :: def int_int_add(space, w_int1, w_int2): x = w_int1.intval From lac at codespeak.net Fri Sep 26 22:35:01 2003 From: lac at codespeak.net (lac at codespeak.net) Date: Fri, 26 Sep 2003 22:35:01 +0200 (MEST) Subject: [pypy-svn] rev 1425 - pypy/trunk/doc/funding Message-ID: <20030926203501.2E0065B653@thoth.codespeak.net> Author: lac Date: Fri Sep 26 22:34:59 2003 New Revision: 1425 Modified: pypy/trunk/doc/funding/B0.0_preamble.txt pypy/trunk/doc/funding/B2.0.0_new_relevance.txt Log: New doc, the relevance I was supposed to say before. also fixed one typo in preamble as I found it. Modified: pypy/trunk/doc/funding/B0.0_preamble.txt ============================================================================== --- pypy/trunk/doc/funding/B0.0_preamble.txt (original) +++ pypy/trunk/doc/funding/B0.0_preamble.txt Fri Sep 26 22:34:59 2003 @@ -105,7 +105,7 @@ The PyPy project will build a flexible, modular, context-aware implementation of the Open Source programming language Python written in itself. Python is already a very popular Object-Oriented -Very-High-Level interreted programming language. +Very-High-Level interpreted programming language. (FN -- Usage statistics indicate that Python ranks sixth in usage world-wide, after Java, Visual Basic, C, C++, and Perl) Modified: pypy/trunk/doc/funding/B2.0.0_new_relevance.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_new_relevance.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_new_relevance.txt Fri Sep 26 22:34:59 2003 @@ -12,6 +12,7 @@ The introductory sections of the Workprogram for IST 2003/4 contains good reference material. +http://www.cordis.lu/ist/istag-reports.htm 2. Via the Europa web site, http://europa.eu.int there is information on all EU policies and they can be identified and @@ -28,3 +29,180 @@ You must also address where appropriate ERA related issues such as relationships to any Eureka activities, (such as commonality of partners) or relationships to national research programs. + +-------------------------------------- + +B2 Relevance to the Objectives of the IST Priority +-------------------------------------------------- + +The PyPy project will build a flexible, modular, context-aware +implementation of the Open Source programming language Python. In +doing so it will take advantage of the known European strengths in +formal methods and languages outlined in the June 2002 report: 'IST +Advisory Group Software Technologies, Embedded Systems and Distributed +Systems: A European strategy towards an Ambient Intelligent +Environment'. It will go beyond the state of the Art in computer +languages, and produce a runtime system which is much better suited +for the development and deployment of networked, embedded, and mobile +devices than any existing language available today. In doing so +it will adhere to the syntax of the popular programming language, thus +requiring no re-training for the tens of thousands of European Python +programmers. To improve an existing programming language with the +explicit goals of making it more suitable for European industry can +only have a positive effect on European competitiveness. + +Since Python is widely used as a 'first programming language' and has +been very successful as a language to be taught to non-programmers, +particularily women, increasing the demand for Python in the IT +marketplace in general can also have a limited effect in promoting +social cohesion and gender equality. (See section 7 where this is +expanded at great length). + +Because PyPy will be released as Open Source, it will contribute +towards the goals of openness and trust. Because Python is the most +widely used European-designed computer language, (its development +started in 1990, at CWI, Centrum voor Wiskunde en Informatica, the +National Research Institute for Mathematics and Computer Science in +the Netherlands) it will strenghten our leadership in the area of +innovative language design, and increase world-wide awareness of this +fact. + +Thus PyPy is position to be at the heart of the knowledge economy and +provide a viable alternative to American closed source language +monopolies, while increasing innovation and competitiveness in +European businesses and industry and to contributing to the greater +well-being of all European citizens. + +2.1 Relevance to the IST Workprogramme +++++++++++++++++++++++++++++++++++++++ + +The IST thematic priority will contribute directly to realising +European policies for the knowledge society as agreed at the Lisbon +Council of 2000, the Stockholm Council of 2001, the Seville Council of +2002, and as reflected in the e-Europe Action Plan. + +The main targets of IST in FP6 are + + - solving 'trust and confidence' problems so as to improve + dependability of technologies, infrastructures and applications. + +PyPy can contribute to this goal because it is an Open Source +language. Open Source programs are more widely trusted than +proprietary alternatives because they are transparent. Any person +having any doubts or questions about the technology can simply read +the code and find out what it does, unlike with a closed-source +language where one can only hope. Moreover, Python is an extremely +readable language. Readability was one of its design goals. This +makes maintaining Python programs substatially easier than similar +program in less readable languages. It is the maintainability of +computer programs which most directly effects their actual and +perceived reliability. + + - strengthening social cohesion by providing efficient, + intelligent and easy to use systems. + +A primary design goal of Python was to be easier for non-programmers +and non-technical people to learn. If the devices of the future were +programmed in Python, even if they were done so simply for reasons of +economic competitiveness, there could be greater participation by all +memebers of society simply because there is less of a language +barrier. + + - enabling sustainable growth and improving competitiveness both of + large and small businesses as well as the efficiency and transparency + of governments. + +Firstly, governments have been embracing Open Source in recent times +in a big way. To the extent that they will demand programs developed +in an Open Source Language, for reasons of transparency, reliability, +and national security they will benefit from the existence of PyPy as +an Open Source language choice. Secondly, a driving force in this +proposal is the Swedish SME AB Strakt, whose new product, *CAPS +Upphandling*, is a full-featured, transparent, government procurement +system promoting the efficiency of government throughout the tender +process. AB Strakt is interested in PyPy in large part because it +will improve its commercial government offering. Thirdly, the PyPy +Consortium includes the Python Business Forum, an international trade +association of SMEs who develop using the Python programming language. +Since SMEs are the main engines for innovation, growth, and +competitiveness in the IT sector, by supporting these SMEs, and +improving the language they use to develop one can directly have a +positive effect on competitiveness. Moreover, syngeries can be +developed between the SMEs and academia, and SME and the +large industrial players whom we haven't signed on board yet, sigh. + + - harnessing the computing and knowledge management resources + across Europe and bringing them to the desktop of any + researcher, engineer or other end user. + +Open source standards increase interoperability, which make such +resources easier to share. + +Again, quoting from the IST workprogramme, this requires progress in +three main technology building blocks: + + - pushing the limits of miniaturisation and minimising the costs + and power consumption of microelectronic components and + micro-systems. + +Smaller systems need software developed in languages that have a +small footprint. PyPy will be there. + + - developing mobile, wireless, optical and broadband communication + infrastructures as well as software and computing technologies + that are reliable, pervasive, interoperable and can be adapted + to accommodate new applications and services. Europe's strengths + both in communication technologies and in embedded software and + systems provide a clear opportunity to lead and contribute to + the development of the next generation of products and + services. The development of open standards and open source + software will be encouraged when appropriate to ensure + interoperability of solutions and to further innovation. + +This is PyPy in a nutshell. An Open Source language that can be +adapted to accomodate new applications and services. + +- Developing user friendly interfaces which are intuitive, can + interpret all our senses such as speech, vision and touch and + that understand our gestures and various languages. This should + be coupled with more powerful and flexible knowledge + technologies that are semantic-based and context-aware. They + should prepare for the next generation Web and make access to, + and creation of digital content more effective and more + creative. + +Now we need to talk about synergies with the Direct Info/Semantic Web +project that DKFI is also involved with. Alistair needs to write this +bit. + +Also, I have missed some parts, and Nicolas needs to write a bit about +constraint programming. + +2.2 European Dimension +++++++++++++++++++++++ + +I don't know what to say here. + +2.3 Links and Complimentary with Other Programmes ++++++++++++++++++++++++++++++++++++++++++++++++++ + +Direct Info/Semantic Web, correct? + +2.4 Contribution to Community Social Objectives ++++++++++++++++++++++++++++++++++++++++++++++++ +I don't know what to say here, either. Have I said it already? + +2.5 Contributiuon to EC Policies +++++++++++++++++++++++++++++++++ + +There will be transfer of knowledge from research to industry through +the participation of software developing SMEs. Thus the SMEs in the +project will benefit from the cutting-edge, high level research results. +Since the SMEs are in a hurry to commercialise products which use this +research, the academics will see that their research is not wasted -- +locked into tiny languages which have little effect outside of the +academic community. A specialising Just in Time Compiler for Python, +designed for the use of networked and embedded systems will have +immediate effect in reinforcing European dominance in this demanding +competitive field, and thus contribute to the employment of people +in desirable jobs in a rapidly growing field. From tismer at codespeak.net Sun Sep 28 14:51:43 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 28 Sep 2003 14:51:43 +0200 (MEST) Subject: [pypy-svn] rev 1426 - pypy/trunk/doc/objspace Message-ID: <20030928125143.43AFA5A144@thoth.codespeak.net> Author: tismer Date: Sun Sep 28 14:51:42 2003 New Revision: 1426 Modified: pypy/trunk/doc/objspace/restrictedpy.txt Log: Some ReST enhancements, and reflowing all paragraphs to be multiple line, in order to get better diff's next time. Modified: pypy/trunk/doc/objspace/restrictedpy.txt ============================================================================== --- pypy/trunk/doc/objspace/restrictedpy.txt (original) +++ pypy/trunk/doc/objspace/restrictedpy.txt Sun Sep 28 14:51:42 2003 @@ -2,41 +2,74 @@ Restricted Python ================== -We are writing a Python interpreter in Python, using Python's well known ability to step behind the algorithmic problems as language. At first glance, one might think this achieves nothing but a better understanding for everybody how the interpreter works. This alone would make it worth doing, but we have much larger goals. +We are writing a Python interpreter in Python, using Python's well known ability +to step behind the algorithmic problems as language. At first glance, one might +think this achieves nothing but a better understanding for everybody how the +interpreter works. This alone would make it worth doing, but we have much larger +goals. CPython vs. PyPy ------------------- -Compared to the CPython implementation, Python takes the role of the C Code. So actually, we describe something by Python, which has been coded in C already, with all the restrictions that are implied by C. We are not trying to make the structures of the CPython interpreter more flexible by rewriting things in C, but we want to use Python to give an alternative description of the interpreter. - -The clear advantage is that this description is probably shorter and simpler to read, and many implementation details vanish. The drawback of this approach is that this interpreter will be unbearably slow. - -To get to a useful interpreter again, we need to apply some mappings to the implementation, later. One rather straight-forward is to do a whole program analysis of the PyPy interpreter and create a C source, again. There are many other ways, but let's stick with the easiest approach, first. - -In order to make a C code generator simple, we restrict ourselves to a subset of the Python language, and we adhere to some rules, which make code generation obvious and easy. +Compared to the CPython implementation, Python takes the role of the C Code. So +actually, we describe something by Python, which has been coded in C already, +with all the restrictions that are implied by C. We are not trying to make the +structures of the CPython interpreter more flexible by rewriting things in C, +but we want to use Python to give an alternative description of the interpreter. + +The clear advantage is that this description is probably shorter and simpler to +read, and many implementation details vanish. The drawback of this approach is +that this interpreter will be unbearably slow. + +To get to a useful interpreter again, we need to apply some mappings to the +implementation, later. One rather straight-forward is to do a whole program +analysis of the PyPy interpreter and create a C source, again. There are many +other ways, but let's stick with the easiest approach, first. + +In order to make a C code generator simple, we restrict ourselves to a subset of +the Python language, and we adhere to some rules, which make code generation +obvious and easy. Restricted Python is Runtime Python ------------------------------------- -Restricted Python describes a runnable Python interpreter implementation. This is a quite static object that can be suitably described by RPython. But the restrictions do not apply during the startup phase. +Restricted Python describes a runnable Python interpreter implementation. This +is a quite static object that can be suitably described by RPython. But the +restrictions do not apply during the startup phase. PyPy Bootstrap ------------------- -When the PyPy interpreter is started as a CPython program, it can use all of CPython for a while, until it reaches runtime. That is, all executable code will be executed using the full power of Python. - -An example can be found in the implementation, which is quite elegant: For the definition of all the opcodes of the Python interpreter, the module dis is imported and used. This saves us from adding extra modules to PyPy. The import code is run at startup time, and we are allowed to use the CPython builtin import function. - -When the startup code is done, all resulting objects, functions, code blocks etc. must adhere to the runtime restrictions. All initialized modules are written out in a persistent manner. Our current idea is to emit a huge C source file which contains everything created so far. During this process, a whole program analysis is performed, which makes use of the restrictions defined in RPython. This enables the code generator to emit efficient replacements for pure integer objects, for instance. +When the PyPy interpreter is started as a CPython program, it can use all of +CPython for a while, until it reaches runtime. That is, all executable code will +be executed using the full power of Python. + +An example can be found in the implementation, which is quite elegant: For the +definition of all the opcodes of the Python interpreter, the module dis is +imported and used. This saves us from adding extra modules to PyPy. The import +code is run at startup time, and we are allowed to use the CPython builtin +import function. + +When the startup code is done, all resulting objects, functions, code blocks +etc. must adhere to the runtime restrictions. All initialized modules are +written out in a persistent manner. Our current idea is to emit a huge C source +file which contains everything created so far. During this process, a whole +program analysis is performed, which makes use of the restrictions defined in +RPython. This enables the code generator to emit efficient replacements for pure +integer objects, for instance. RPython Definition -------------------- -It might make sense to define a sub-language of Python called RPython, with the restrictions depicted below. This is an evolving topic, and we're just collecting things which come up during trying to code the interpreter, so this is no language at all, but an arbitrary set of rules, which are about to be changed all day. +It might make sense to define a sub-language of Python called RPython, with the +restrictions depicted below. This is an evolving topic, and we're just +collecting things which come up during trying to code the interpreter, so this +is no language at all, but an arbitrary set of rules, which are about to be +changed all day. Object restrictions @@ -45,59 +78,93 @@ We are using **variables** - the same variable in the same context can receive values of different types, at a possible overhead cost. For example, a variable that can contain a wrapped object or None is efficiently implemented as a PyObject* pointer that can be NULL, but a variable that can contain either an integer or a float must be implemented as a union with a type tag in C. + + the same variable in the same context can receive values of different types, + at a possible overhead cost. For example, a variable that can contain a + wrapped object or None is efficiently implemented as a PyObject* pointer that + can be NULL, but a variable that can contain either an integer or a float must + be implemented as a union with a type tag in C. **constants** + all module globals are considered constants. **integer, float, string, boolean** + avoid string methods and complex operations like slicing with a step **tuples** - no variable-length tuples; use them to store or return pairs or n-tuples of values + + no variable-length tuples; use them to store or return pairs or n-tuples of + values **lists** - lists are used as an allocated array; list.append() does naive resizing, so as far as possible use list comprehensions (see below) + + lists are used as an allocated array; list.append() does naive resizing, so as + far as possible use list comprehensions (see below) **dicts** + no dicts **control structures** + all allowed **list comprehensions** - may be used to create allocated, initialized array. the array size must be computable in advance, which implies that we don't allow an if clause. + + may be used to create allocated, initialized array. the array size must be + computable in advance, which implies that we don't allow an if clause. **functions** -+ statically called functions may use defaults and a variable number of arguments (which may be passed as a list instead of a tuple, so write code that does not depend on it being a tuple). -+ dynamic dispatch enforces use of very simple signatures, equal for all functions to be called in that context. At the moment, this occurs in the opcode dispatch, only. ++ statically called functions may use defaults and a variable number of + arguments (which may be passed as a list instead of a tuple, so write code that + does not depend on it being a tuple). + ++ dynamic dispatch enforces use of very simple signatures, equal for all + functions to be called in that context. At the moment, this occurs in the opcode + dispatch, only. **builtin functions** - A few builtin functions will be used, while this set is not defined completely, yet. Some builtin functions are special forms: + + A few builtin functions will be used, while this set is not defined + completely, yet. Some builtin functions are special forms: **range** - does not create an array. It is only allowed in for loops. The step argument must be a constant. + + does not create an array. It is only allowed in for loops. The step argument + must be a constant. **len** -+ may be used with basic types that have a length. But len is a special form that is recognized by the compiler. -+ If a certain structure is never touched by len, the compiler might save the length field from the underlying structure. + ++ may be used with basic types that have a length. But len is a special form + that is recognized by the compiler. + ++ If a certain structure is never touched by len, the compiler might save the + length field from the underlying structure. ``int, float, ord, chr``... are available as simple conversion functions. + ``int, float, str``... have a special meaning as a type inside of isinstance only. **classes** + + methods do not change after startup + methods are never hidden by attributes + inheritance is supported + classes are first-class objects too **exceptions** + + fully supported + see below for restrictions on exceptions raised by built-in operations **objects** - wrapped objects are borrowed from the object space. Just like in CPython, code that needs e.g. a dictionary can use a wrapped dict and the object space operations on it. + + wrapped objects are borrowed from the object space. Just like in CPython, code + that needs e.g. a dictionary can use a wrapped dict and the object space + operations on it. This layout makes the number of types to take care about quite limited. @@ -105,13 +172,29 @@ Example: Integer Types ------------------------- -While implementing the integer type, I (Chris) stumbled over the problem, that integers are quite in flux in CPython right now. Depending on the version, integers either raise an overflow exception or mutate into longs on overflow. Also, shifting right now truncates (up to 2.3) but is going to extend to longs as well. In order to enable us to run the restricted Python stuff in CPython, I needed to use a derived class r_int(int), which always behaves the same: Never leaving its domain, but always staying an integer. - -The r_int type is implemented in a pervasive way: Every operation that involves an r_int creates an r_int as the result. Therefore, the implementation is not cluttered with special type casts. Just the initial value of an emulated integer's intval field is initialized by obj.intval = r_int(val) . This way, the r_int type propagates itself through all operations without extra effort of the programmer. - -This concept looks promising, and since we will need unsigned integers which do not overflow as well, I also created r_uint. It is always a word-sized unsigned integer and never overflows. This will be a building block for things like computing hash values, where wrap-around effects are intended and should be easily coded without lots of explicit mask shuffling. +While implementing the integer type, I (Chris) stumbled over the problem, that +integers are quite in flux in CPython right now. Depending on the version, +integers either raise an overflow exception or mutate into longs on overflow. +Also, shifting right now truncates (up to 2.3) but is going to extend to longs +as well. In order to enable us to run the restricted Python stuff in CPython, I +needed to use a derived class r_int(int), which always behaves the same: Never +leaving its domain, but always staying an integer. + +The r_int type is implemented in a pervasive way: Every operation that involves +an r_int creates an r_int as the result. Therefore, the implementation is not +cluttered with special type casts. Just the initial value of an emulated +integer's intval field is initialized by obj.intval = r_int(val) . This way, the +r_int type propagates itself through all operations without extra effort of the +programmer. + +This concept looks promising, and since we will need unsigned integers which do +not overflow as well, I also created r_uint. It is always a word-sized unsigned +integer and never overflows. This will be a building block for things like +computing hash values, where wrap-around effects are intended and should be +easily coded without lots of explicit mask shuffling. -Now I think to extend this even more and build a full set of primitive types, which are intended to +Now I think to extend this even more and build a full set of primitive types, +which are intended to + define the semantics of the primitive type + give a working implementation for unrestricted Python @@ -136,7 +219,9 @@ # implement using longs -Code with no exception handlers does not raise exceptions. By supplying an exception handler, you ask for error checking. Without, you assure the system that the operation cannot overflow. +Code with no exception handlers does not raise exceptions. By supplying an +exception handler, you ask for error checking. Without, you assure the system +that the operation cannot overflow. Exceptions explicitly raised will always be generated. @@ -144,4 +229,7 @@ Testing ------------ -Besides extra tests which have to be written, PyPy has the advantage that it is runnable on standard CPython. That means, we can run all of PyPy with all exception handling enabled, so we might catch cases where we failed to adhere to our implicit assertions. +Besides extra tests which have to be written, PyPy has the advantage that it is +runnable on standard CPython. That means, we can run all of PyPy with all +exception handling enabled, so we might catch cases where we failed to adhere to +our implicit assertions. From pedronis at codespeak.net Mon Sep 29 10:05:50 2003 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 29 Sep 2003 10:05:50 +0200 (MEST) Subject: [pypy-svn] rev 1427 - in pypy/trunk/src/pypy/objspace/std: . test Message-ID: <20030929080550.BFCAF5AD0D@thoth.codespeak.net> Author: pedronis Date: Mon Sep 29 10:05:48 2003 New Revision: 1427 Modified: pypy/trunk/src/pypy/objspace/std/dictobject.py pypy/trunk/src/pypy/objspace/std/listobject.py pypy/trunk/src/pypy/objspace/std/objectobject.py pypy/trunk/src/pypy/objspace/std/objecttype.py pypy/trunk/src/pypy/objspace/std/test/test_listobject.py pypy/trunk/src/pypy/objspace/std/userobject.py Log: fixed __init__ varargs/kwargs and arg number Modified: pypy/trunk/src/pypy/objspace/std/dictobject.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/dictobject.py (original) +++ pypy/trunk/src/pypy/objspace/std/dictobject.py Mon Sep 29 10:05:48 2003 @@ -85,7 +85,7 @@ result[space.unwrap(w_key)] = space.unwrap(cell.get()) return result -def object_init__Dict_ANY_ANY(space, w_dict, w_args, w_kwds): +def object_init__Dict(space, w_dict, w_args, w_kwds): dict_clear__Dict(space, w_dict) args = space.unpackiterable(w_args) if len(args) == 0: Modified: pypy/trunk/src/pypy/objspace/std/listobject.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/listobject.py (original) +++ pypy/trunk/src/pypy/objspace/std/listobject.py Mon Sep 29 10:05:48 2003 @@ -36,7 +36,7 @@ items = [space.unwrap(w_item) for w_item in w_list.ob_item[:w_list.ob_size]] return list(items) -def object_init__List_ANY_ANY(space, w_list, w_args, w_kwds): +def object_init__List(space, w_list, w_args, w_kwds): if space.is_true(w_kwds): raise OperationError(space.w_TypeError, space.wrap("no keyword arguments expected")) Modified: pypy/trunk/src/pypy/objspace/std/objectobject.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/objectobject.py (original) +++ pypy/trunk/src/pypy/objspace/std/objectobject.py Mon Sep 29 10:05:48 2003 @@ -18,7 +18,7 @@ delegate__ANY.result_class = W_ObjectObject delegate__ANY.priority = PRIORITY_PARENT_TYPE -def object_init__Object_ANY_ANY(space, w_object, w_args, w_kwds): +def object_init__Object(space, w_object, w_args, w_kwds): pass Modified: pypy/trunk/src/pypy/objspace/std/objecttype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/objecttype.py (original) +++ pypy/trunk/src/pypy/objspace/std/objecttype.py Mon Sep 29 10:05:48 2003 @@ -21,7 +21,7 @@ object_hash = StdObjSpace.hash # this is a method in 'object' because it is not an object space operation - object_init = MultiMethod('__init__', 3, varargs=True, keywords=True) + object_init = MultiMethod('__init__', 1, varargs=True, keywords=True) registerimplementation(W_ObjectType) Modified: pypy/trunk/src/pypy/objspace/std/test/test_listobject.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/test/test_listobject.py (original) +++ pypy/trunk/src/pypy/objspace/std/test/test_listobject.py Mon Sep 29 10:05:48 2003 @@ -278,6 +278,14 @@ class AppTestW_ListObject(test.AppTestCase): def setUp(self): self.space = test.objspace('std') + + def test_explicit_init(self): + l = [] + l.__init__([1,2]) + self.assertEquals(l,[1,2]) + list.__init__(l,[1,2,3]) + self.assertEquals(l,[1,2,3]) + def test_extend_list(self): l = [1] l.extend([2]) Modified: pypy/trunk/src/pypy/objspace/std/userobject.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/userobject.py (original) +++ pypy/trunk/src/pypy/objspace/std/userobject.py Mon Sep 29 10:05:48 2003 @@ -186,7 +186,7 @@ next__User = SpecialMethod('next').next_call is_true__User = SpecialMethod('nonzero').nonzero_call -object_init__User_ANY_ANY = SpecialMethod('__init__').argskwds_call +object_init__User = SpecialMethod('__init__').argskwds_call iter__User = SpecialMethod('__iter__').iter_call register_all(vars()) From arigo at codespeak.net Mon Sep 29 12:46:26 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Sep 2003 12:46:26 +0200 (MEST) Subject: [pypy-svn] rev 1428 - pypy/trunk/doc/translation Message-ID: <20030929104626.AE68E5A190@thoth.codespeak.net> Author: arigo Date: Mon Sep 29 12:46:26 2003 New Revision: 1428 Added: pypy/trunk/doc/translation/ pypy/trunk/doc/translation/controlflow.txt Log: Text we discussed about control flow Added: pypy/trunk/doc/translation/controlflow.txt ============================================================================== --- (empty file) +++ pypy/trunk/doc/translation/controlflow.txt Mon Sep 29 12:46:26 2003 @@ -0,0 +1,105 @@ +Hello again + + def f(i): + return g(i)+2 + def g(i): + return i+1 + +f(3) +tspace: +pyxcode.putln('def f(%s):' % sig) +res = frame.eval(executioncontext) +pyxcode.putln('return %s' % res) + + +Pyrex: def f(v1): +dis.dis(f) --> + 2 0 LOAD_GLOBAL 0 (g) + w_result = space.getitem(f.w_globals, w_varname) + + 3 LOAD_FAST 0 (i) + 6 CALL_FUNCTION 1 +space.call(w_function, w_arguments, w_kwds) +space.call(w_g, ("v1",), {}) +Pyrex: v2 = g(v1) + 9 LOAD_CONST 1 (2) +space.wrap(2) + 12 BINARY_ADD +space.add("v2", "constant 2") +Pyrex: v3 = v2 + 2 + 13 RETURN_VALUE + 14 LOAD_CONST 0 (None) + 17 RETURN_VALUE +Pyrex: return v3 + + +Result: +def f(v1): + v2 = g(v1) + v3 = v2 + 2 + return v3 + + +def h(i, j): + if i < 0: + i = j + return i+1 + +Pyrex: def h(v1, v2): +--> interpreter + 3 0 LOAD_FAST 0 (i) + 3 LOAD_CONST 1 (0) + 6 COMPARE_OP 0 (<) +"v3" = space.lt("v1", "constant 0") +Pyrex: v3 = v1 < 0 + 9 JUMP_IF_FALSE 10 (to 22) +space.is_true("v3") +Pyrex: if v3: cinline "goto Label1;" + 12 POP_TOP + + 4 13 LOAD_FAST 1 (j) + 16 STORE_FAST 0 (i) + 19 JUMP_FORWARD 1 (to 23) + >> 22 POP_TOP + +Pyrex: cinline "LabelBytecode23:" # just in case for later + 5 >> 23 LOAD_FAST 0 (i) + 26 LOAD_CONST 2 (1) + 29 BINARY_ADD +space.add("v1", "constant 2") +Pyrex: v4 = v1 + 2 + 30 RETURN_VALUE + 31 LOAD_CONST 0 (None) + 34 RETURN_VALUE +Pyrex: return v4 + +pyrex: cinline "Label1:" + 12 POP_TOP + + 4 13 LOAD_FAST 1 (j) + 16 STORE_FAST 0 (i) +(in the interpreter fastlocals now: i="v2" j="v2") + 19 JUMP_FORWARD 1 (to 23) + >> 22 POP_TOP + +(bytecode 23 already seen!) +Pyrex: v1 = v2 +Pyrex: cinline "goto LabelBytecode23;" + 5 >> 23 LOAD_FAST 0 (i) + +def h(i, j): + if i < 0: + i = j + return i+1 + +def h(v1, v2): + v3 = v1 < 0 + if v3: cinline "goto label1;" + + cinline "labelBytecode23:" + v4=v1+1 + return v4 + cinline "label1:" + v1=v2 + cinline "goto labelBytecode23;" + From hpk at codespeak.net Mon Sep 29 12:57:51 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 29 Sep 2003 12:57:51 +0200 (MEST) Subject: [pypy-svn] rev 1429 - pypy/trunk/doc/translation Message-ID: <20030929105751.4D1ED5A190@thoth.codespeak.net> Author: hpk Date: Mon Sep 29 12:57:50 2003 New Revision: 1429 Added: pypy/trunk/doc/translation/controlflow.asc - copied unchanged from rev 1428, pypy/trunk/doc/translation/controlflow.txt Removed: pypy/trunk/doc/translation/controlflow.txt Log: this is not an ReST file so i renamed it for the time beeing. Deleted: /pypy/trunk/doc/translation/controlflow.txt ============================================================================== --- /pypy/trunk/doc/translation/controlflow.txt Mon Sep 29 12:57:50 2003 +++ (empty file) @@ -1,105 +0,0 @@ -Hello again - - def f(i): - return g(i)+2 - def g(i): - return i+1 - -f(3) -tspace: -pyxcode.putln('def f(%s):' % sig) -res = frame.eval(executioncontext) -pyxcode.putln('return %s' % res) - - -Pyrex: def f(v1): -dis.dis(f) --> - 2 0 LOAD_GLOBAL 0 (g) - w_result = space.getitem(f.w_globals, w_varname) - - 3 LOAD_FAST 0 (i) - 6 CALL_FUNCTION 1 -space.call(w_function, w_arguments, w_kwds) -space.call(w_g, ("v1",), {}) -Pyrex: v2 = g(v1) - 9 LOAD_CONST 1 (2) -space.wrap(2) - 12 BINARY_ADD -space.add("v2", "constant 2") -Pyrex: v3 = v2 + 2 - 13 RETURN_VALUE - 14 LOAD_CONST 0 (None) - 17 RETURN_VALUE -Pyrex: return v3 - - -Result: -def f(v1): - v2 = g(v1) - v3 = v2 + 2 - return v3 - - -def h(i, j): - if i < 0: - i = j - return i+1 - -Pyrex: def h(v1, v2): ---> interpreter - 3 0 LOAD_FAST 0 (i) - 3 LOAD_CONST 1 (0) - 6 COMPARE_OP 0 (<) -"v3" = space.lt("v1", "constant 0") -Pyrex: v3 = v1 < 0 - 9 JUMP_IF_FALSE 10 (to 22) -space.is_true("v3") -Pyrex: if v3: cinline "goto Label1;" - 12 POP_TOP - - 4 13 LOAD_FAST 1 (j) - 16 STORE_FAST 0 (i) - 19 JUMP_FORWARD 1 (to 23) - >> 22 POP_TOP - -Pyrex: cinline "LabelBytecode23:" # just in case for later - 5 >> 23 LOAD_FAST 0 (i) - 26 LOAD_CONST 2 (1) - 29 BINARY_ADD -space.add("v1", "constant 2") -Pyrex: v4 = v1 + 2 - 30 RETURN_VALUE - 31 LOAD_CONST 0 (None) - 34 RETURN_VALUE -Pyrex: return v4 - -pyrex: cinline "Label1:" - 12 POP_TOP - - 4 13 LOAD_FAST 1 (j) - 16 STORE_FAST 0 (i) -(in the interpreter fastlocals now: i="v2" j="v2") - 19 JUMP_FORWARD 1 (to 23) - >> 22 POP_TOP - -(bytecode 23 already seen!) -Pyrex: v1 = v2 -Pyrex: cinline "goto LabelBytecode23;" - 5 >> 23 LOAD_FAST 0 (i) - -def h(i, j): - if i < 0: - i = j - return i+1 - -def h(v1, v2): - v3 = v1 < 0 - if v3: cinline "goto label1;" - - cinline "labelBytecode23:" - v4=v1+1 - return v4 - cinline "label1:" - v1=v2 - cinline "goto labelBytecode23;" - From hpk at codespeak.net Mon Sep 29 13:07:22 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 29 Sep 2003 13:07:22 +0200 (MEST) Subject: [pypy-svn] rev 1430 - pypy/trunk/doc/funding Message-ID: <20030929110722.6856E5A190@thoth.codespeak.net> Author: hpk Date: Mon Sep 29 13:07:21 2003 New Revision: 1430 Modified: pypy/trunk/doc/funding/B5.0_management.txt pypy/trunk/doc/funding/B6.0_detailed_implementation.txt pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/B6.4_workpackage_list.txt pypy/trunk/doc/funding/STREP-template.txt Log: - fixed a couple of ReST problems - small enhancements to a couple of paragraphs (see the diff in detail) Modified: pypy/trunk/doc/funding/B5.0_management.txt ============================================================================== --- pypy/trunk/doc/funding/B5.0_management.txt (original) +++ pypy/trunk/doc/funding/B5.0_management.txt Mon Sep 29 13:07:21 2003 @@ -6,3 +6,5 @@ arising in the project. (Recommended length ? three pages) + + Modified: pypy/trunk/doc/funding/B6.0_detailed_implementation.txt ============================================================================== --- pypy/trunk/doc/funding/B6.0_detailed_implementation.txt (original) +++ pypy/trunk/doc/funding/B6.0_detailed_implementation.txt Mon Sep 29 13:07:21 2003 @@ -42,4 +42,3 @@ more detailed plan. (recommended length, excluding the forms specified above,- up to 15 pages) - Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Mon Sep 29 13:07:21 2003 @@ -24,9 +24,8 @@ Generating down-sized code for embedded systems, load balancing in a distributed network, code generators optimized for number crunching on some processor architectures and re-writing numerical Python - packages should be able to prove that PyPy is more than an - academical exercise but very appropriate for industrial strength - applications. + packages should be able to prove that PyPy is very appropriate + for industrial strength applications. - Supporting Embedded Devices @@ -40,7 +39,7 @@ - Project Documentation and Dissemination - - Maintenance of Tools + - Maintenance of the development environment - Synchronisation with Standard Python @@ -71,9 +70,9 @@ and how to hook in the bytecode compiler. -WP22_: Translation of CPython into regular Python +WP22_: Porting CPython C-sourcecode to regular Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Translate all other parts of CPython which we don't want to + Port all parts of CPython which we don't want to implement in WP21_ into regular Python libraries. These ones should also work without PyPy, being just plain-Python replacements for existing CPython functionality. Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 29 13:07:21 2003 @@ -19,7 +19,7 @@ ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP21_ Development and Completion of the PyPy Interpreter 12 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- -WP22_ Translation of CPython into regular Python 12 +WP22_ Porting CPython C-sourcecode to regular Python 12 ----- ----------------------------------------------------- ----- ---- ---- ---- ---- WP30_ The PyPy Compiler ----- ----------------------------------------------------- ----- ---- ---- ---- ---- Modified: pypy/trunk/doc/funding/STREP-template.txt ============================================================================== --- pypy/trunk/doc/funding/STREP-template.txt (original) +++ pypy/trunk/doc/funding/STREP-template.txt Mon Sep 29 13:07:21 2003 @@ -775,16 +775,16 @@ have been simplified and a larger flexibility is granted to the participants: - ? rules are identical for all participants; - ? rules concentrate on the principles and provisions considered - necessary for an efficient cooperation and the appropriate use and - dissemination of the results; - ? participants may define among themselves the arrangements that - fit them the best within the framework provided in the model - contract. +- rules are identical for all participants; +- rules concentrate on the principles and provisions considered + necessary for an efficient cooperation and the appropriate use and + dissemination of the results; +- participants may define among themselves the arrangements that + fit them the best within the framework provided in the model + contract. - See also comments under B.3.3.1 and B.3.3.2 and address here. +See also comments under B.3.3.1 and B.3.3.2 and address here. B.6 Detailed Implementation Plan @@ -812,8 +812,9 @@ progress and results. Essential elements of the plan are: -a) Detailed Implementation plan introduction ? explaining the structure - of this plan and the overall methodology used to achieve the objectives. + +a) Detailed Implementation plan introduction ? explaining the structure + of this plan and the overall methodology used to achieve the objectives. b) Work planning, showing the timing of the different WPs and their components (Gantt chart or similar). Ensure that the workplan is @@ -1067,7 +1068,8 @@ Further information on ethics requirements and rules are given at the science and ethics website at - http://europa.eu.int/comm/research/science-society/ethics/ethics_en.html. + + http://europa.eu.int/comm/research/science-society/ethics/ethics_en.html. From hpk at codespeak.net Mon Sep 29 13:09:12 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 29 Sep 2003 13:09:12 +0200 (MEST) Subject: [pypy-svn] rev 1431 - pypy/trunk/doc/funding Message-ID: <20030929110912.388E85A190@thoth.codespeak.net> Author: hpk Date: Mon Sep 29 13:09:11 2003 New Revision: 1431 Modified: pypy/trunk/doc/funding/B2.0.0_new_relevance.txt (props changed) pypy/trunk/doc/funding/B6.4_workpackage_list.txt (contents, props changed) pypy/trunk/doc/funding/STREP-template.txt (props changed) Log: fixeol. why do these eol-problems keep coming up? it's really time for some pre-commit hook that prevents a checkin of a text-file without the svn:eol-style property set to 'native'. Modified: pypy/trunk/doc/funding/B6.4_workpackage_list.txt ============================================================================== --- pypy/trunk/doc/funding/B6.4_workpackage_list.txt (original) +++ pypy/trunk/doc/funding/B6.4_workpackage_list.txt Mon Sep 29 13:09:11 2003 @@ -1,61 +1,61 @@ -.. include:: crossreferences.asc - -Workpackage List -==================== - -.. |wpn| replace:: Work-package No -.. |lcn| replace:: Lead contractor No -.. |pm| replace:: Person-months -.. |sm| replace:: Start month -.. |em| replace:: End month -.. |dn| replace:: Deliverable No - -===== ===================================================== ===== ==== ==== ==== ==== -|wpn| Workpackage title |lcn| |pm| |sm| |em| |dn| -===== ===================================================== ===== ==== ==== ==== ==== -WP10_ Coordination and Management 18 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP20_ The PyPy Interpreter ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP21_ Development and Completion of the PyPy Interpreter 12 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP22_ Porting CPython C-sourcecode to regular Python 12 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP30_ The PyPy Compiler ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP31_ Translation of RPython 8 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP32_ Specific Analysis and Redesign 8 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP33_ Low Level PyPy Runtime 6 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP40_ High-performance PyPy-Python ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP41_ Integration of Stackless 9 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP42_ Several Object Implementations 12 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP43_ Translator Optimisations 12 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP44_ Integration of Psyco 12 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP50_ Validation of PyPy ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP51_ Porting PyPy to an Embedded Device 9 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP52_ Evaluation whether to do a small OS in PyPy 3 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP53_ A small Operating System for an Embedded Device 11 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP54_ Code Generator for SIMD Architecture 9 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP55_ Enhanced Numerical Package 6 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP56_ Load Balancing in a Multi-Processor environment 9 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP60_ Project Documentation and Dissemination 25 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP70_ Maintenance of Tools 10 ------ ----------------------------------------------------- ----- ---- ---- ---- ---- -WP80_ Synchronisation with Standard Python 6 -===== ===================================================== ===== ==== ==== ==== ==== +.. include:: crossreferences.asc + +Workpackage List +==================== + +.. |wpn| replace:: Work-package No +.. |lcn| replace:: Lead contractor No +.. |pm| replace:: Person-months +.. |sm| replace:: Start month +.. |em| replace:: End month +.. |dn| replace:: Deliverable No + +===== ===================================================== ===== ==== ==== ==== ==== +|wpn| Workpackage title |lcn| |pm| |sm| |em| |dn| +===== ===================================================== ===== ==== ==== ==== ==== +WP10_ Coordination and Management 18 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP20_ The PyPy Interpreter +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP21_ Development and Completion of the PyPy Interpreter 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP22_ Porting CPython C-sourcecode to regular Python 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP30_ The PyPy Compiler +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP31_ Translation of RPython 8 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP32_ Specific Analysis and Redesign 8 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP33_ Low Level PyPy Runtime 6 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP40_ High-performance PyPy-Python +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP41_ Integration of Stackless 9 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP42_ Several Object Implementations 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP43_ Translator Optimisations 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP44_ Integration of Psyco 12 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP50_ Validation of PyPy +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP51_ Porting PyPy to an Embedded Device 9 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP52_ Evaluation whether to do a small OS in PyPy 3 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP53_ A small Operating System for an Embedded Device 11 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP54_ Code Generator for SIMD Architecture 9 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP55_ Enhanced Numerical Package 6 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP56_ Load Balancing in a Multi-Processor environment 9 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP60_ Project Documentation and Dissemination 25 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP70_ Maintenance of Tools 10 +----- ----------------------------------------------------- ----- ---- ---- ---- ---- +WP80_ Synchronisation with Standard Python 6 +===== ===================================================== ===== ==== ==== ==== ==== From tismer at codespeak.net Mon Sep 29 14:56:38 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 29 Sep 2003 14:56:38 +0200 (MEST) Subject: [pypy-svn] rev 1433 - pypy/trunk/doc/funding/test Message-ID: <20030929125638.381A25A190@thoth.codespeak.net> Author: tismer Date: Mon Sep 29 14:56:37 2003 New Revision: 1433 Added: pypy/trunk/doc/funding/test/ pypy/trunk/doc/funding/test/demo_wp_bad.sxw (contents, props changed) pypy/trunk/doc/funding/test/demo_wp_good.sxw (contents, props changed) Log: test data for work package formatting Added: pypy/trunk/doc/funding/test/demo_wp_bad.sxw ============================================================================== Binary file. No diff available. Added: pypy/trunk/doc/funding/test/demo_wp_good.sxw ============================================================================== Binary file. No diff available. From tismer at codespeak.net Mon Sep 29 15:00:42 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 29 Sep 2003 15:00:42 +0200 (MEST) Subject: [pypy-svn] rev 1434 - pypy/trunk/doc/funding/test Message-ID: <20030929130042.C811A5A2A9@thoth.codespeak.net> Author: tismer Date: Mon Sep 29 15:00:42 2003 New Revision: 1434 Added: pypy/trunk/doc/funding/test/demo_wp_delete.sxw (contents, props changed) Log: demo text with text to be deleted. Does this work via XML? Added: pypy/trunk/doc/funding/test/demo_wp_delete.sxw ============================================================================== Binary file. No diff available. From pedronis at codespeak.net Mon Sep 29 15:21:16 2003 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 29 Sep 2003 15:21:16 +0200 (MEST) Subject: [pypy-svn] rev 1435 - in pypy/trunk/src/pypy/objspace/std: . test Message-ID: <20030929132116.5E1555A2A9@thoth.codespeak.net> Author: pedronis Date: Mon Sep 29 15:21:15 2003 New Revision: 1435 Modified: pypy/trunk/src/pypy/objspace/std/dicttype.py pypy/trunk/src/pypy/objspace/std/floattype.py pypy/trunk/src/pypy/objspace/std/inttype.py pypy/trunk/src/pypy/objspace/std/listtype.py pypy/trunk/src/pypy/objspace/std/objecttype.py pypy/trunk/src/pypy/objspace/std/slicetype.py pypy/trunk/src/pypy/objspace/std/stringtype.py pypy/trunk/src/pypy/objspace/std/test/test_listobject.py pypy/trunk/src/pypy/objspace/std/tupletype.py pypy/trunk/src/pypy/objspace/std/typetype.py pypy/trunk/src/pypy/objspace/std/usertype.py Log: fixed __new__ arg number and varargs/kwargs Modified: pypy/trunk/src/pypy/objspace/std/dicttype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/dicttype.py (original) +++ pypy/trunk/src/pypy/objspace/std/dicttype.py Mon Sep 29 15:21:15 2003 @@ -28,7 +28,7 @@ registerimplementation(W_DictType) -def type_new__DictType_DictType_ANY_ANY(space, w_basetype, w_dicttype, w_args, w_kwds): +def type_new__DictType_DictType(space, w_basetype, w_dicttype, w_args, w_kwds): return space.newdict([]), True Modified: pypy/trunk/src/pypy/objspace/std/floattype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/floattype.py (original) +++ pypy/trunk/src/pypy/objspace/std/floattype.py Mon Sep 29 15:21:15 2003 @@ -9,7 +9,7 @@ registerimplementation(W_FloatType) -def type_new__FloatType_FloatType_ANY_ANY(space, w_basetype, w_floattype, w_args, w_kwds): +def type_new__FloatType_FloatType(space, w_basetype, w_floattype, w_args, w_kwds): if space.is_true(w_kwds): raise OperationError(space.w_TypeError, space.wrap("no keyword arguments expected")) Modified: pypy/trunk/src/pypy/objspace/std/inttype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/inttype.py (original) +++ pypy/trunk/src/pypy/objspace/std/inttype.py Mon Sep 29 15:21:15 2003 @@ -9,7 +9,7 @@ registerimplementation(W_IntType) -def type_new__IntType_IntType_ANY_ANY(space, w_basetype, w_inttype, w_args, w_kwds): +def type_new__IntType_IntType(space, w_basetype, w_inttype, w_args, w_kwds): if space.is_true(w_kwds): raise OperationError(space.w_TypeError, space.wrap("no keyword arguments expected")) Modified: pypy/trunk/src/pypy/objspace/std/listtype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/listtype.py (original) +++ pypy/trunk/src/pypy/objspace/std/listtype.py Mon Sep 29 15:21:15 2003 @@ -19,7 +19,7 @@ registerimplementation(W_ListType) -def type_new__ListType_ListType_ANY_ANY(space, w_basetype, w_listtype, w_args, w_kwds): +def type_new__ListType_ListType(space, w_basetype, w_listtype, w_args, w_kwds): return space.newlist([]), True register_all(vars()) Modified: pypy/trunk/src/pypy/objspace/std/objecttype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/objecttype.py (original) +++ pypy/trunk/src/pypy/objspace/std/objecttype.py Mon Sep 29 15:21:15 2003 @@ -26,7 +26,7 @@ registerimplementation(W_ObjectType) -def type_new__ObjectType_ObjectType_ANY_ANY(space, w_basetype, w_objecttype, w_args, w_kwds): +def type_new__ObjectType_ObjectType(space, w_basetype, w_objecttype, w_args, w_kwds): # XXX 2.2 behavior: ignoring all arguments from objectobject import W_ObjectObject return W_ObjectObject(space), True Modified: pypy/trunk/src/pypy/objspace/std/slicetype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/slicetype.py (original) +++ pypy/trunk/src/pypy/objspace/std/slicetype.py Mon Sep 29 15:21:15 2003 @@ -13,7 +13,7 @@ registerimplementation(W_SliceType) -def type_new__SliceType_SliceType_ANY_ANY(space, w_basetype, w_slicetype, w_args, w_kwds): +def type_new__SliceType_SliceType(space, w_basetype, w_slicetype, w_args, w_kwds): if space.is_true(w_kwds): raise OperationError(space.w_TypeError, space.wrap("no keyword arguments expected")) Modified: pypy/trunk/src/pypy/objspace/std/stringtype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/stringtype.py (original) +++ pypy/trunk/src/pypy/objspace/std/stringtype.py Mon Sep 29 15:21:15 2003 @@ -41,7 +41,7 @@ registerimplementation(W_StringType) -def type_new__StringType_StringType_ANY_ANY(space, w_basetype, w_stringtype, w_args, w_kwds): +def type_new__StringType_StringType(space, w_basetype, w_stringtype, w_args, w_kwds): if space.is_true(w_kwds): raise OperationError(space.w_TypeError, space.wrap("no keyword arguments expected")) Modified: pypy/trunk/src/pypy/objspace/std/test/test_listobject.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/test/test_listobject.py (original) +++ pypy/trunk/src/pypy/objspace/std/test/test_listobject.py Mon Sep 29 15:21:15 2003 @@ -279,8 +279,8 @@ def setUp(self): self.space = test.objspace('std') - def test_explicit_init(self): - l = [] + def test_explicit_new_init(self): + l = list.__new__(list) l.__init__([1,2]) self.assertEquals(l,[1,2]) list.__init__(l,[1,2,3]) Modified: pypy/trunk/src/pypy/objspace/std/tupletype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/tupletype.py (original) +++ pypy/trunk/src/pypy/objspace/std/tupletype.py Mon Sep 29 15:21:15 2003 @@ -9,7 +9,7 @@ registerimplementation(W_TupleType) -def type_new__TupleType_TupleType_ANY_ANY(space, w_basetype, w_tupletype, w_args, w_kwds): +def type_new__TupleType_TupleType(space, w_basetype, w_tupletype, w_args, w_kwds): if space.is_true(w_kwds): raise OperationError(space.w_TypeError, space.wrap("no keyword arguments expected")) Modified: pypy/trunk/src/pypy/objspace/std/typetype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/typetype.py (original) +++ pypy/trunk/src/pypy/objspace/std/typetype.py Mon Sep 29 15:21:15 2003 @@ -27,12 +27,12 @@ # the class 'type'. # Attention, this internally returns a tuple (w_result, flag), # where 'flag' specifies whether we would like __init__() to be called. - type_new = MultiMethod('__new__', 4, varargs=True, keywords=True, + type_new = MultiMethod('__new__', 2, varargs=True, keywords=True, mmframeclass=NewMmFrame) registerimplementation(W_TypeType) -def type_new__TypeType_TypeType_ANY_ANY(space, w_basetype, w_typetype, w_args, w_kwds): +def type_new__TypeType_TypeType(space, w_basetype, w_typetype, w_args, w_kwds): if space.is_true(w_kwds): raise OperationError(space.w_TypeError, space.wrap("no keyword arguments expected")) Modified: pypy/trunk/src/pypy/objspace/std/usertype.py ============================================================================== --- pypy/trunk/src/pypy/objspace/std/usertype.py (original) +++ pypy/trunk/src/pypy/objspace/std/usertype.py Mon Sep 29 15:21:15 2003 @@ -53,7 +53,7 @@ # but we're documenting it here as there seems no better place!!! # The problem is actually that, currently, several types such as # int and float just cannot be CALLED -- this needs to be fixed soon. -def type_new__UserType_UserType_ANY_ANY(space, w_basetype, w_usertype, w_args, w_kwds): +def type_new__UserType_UserType(space, w_basetype, w_usertype, w_args, w_kwds): import typetype # XXX this uses the following algorithm: # walk the __mro__ of the user type @@ -74,7 +74,7 @@ return w_newobj, True raise AssertionError, "execution should not get here" -def type_new__ANY_UserType_ANY_ANY(space, w_basetype, w_usertype, w_args, w_kwds): +def type_new__ANY_UserType(space, w_basetype, w_usertype, w_args, w_kwds): import typetype from userobject import morph_into_user_object, getsinglebuiltintype assert w_basetype is getsinglebuiltintype(space, w_usertype) From tismer at codespeak.net Mon Sep 29 15:23:07 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 29 Sep 2003 15:23:07 +0200 (MEST) Subject: [pypy-svn] rev 1436 - pypy/trunk/doc/translation Message-ID: <20030929132307.B1BC35B9D8@thoth.codespeak.net> Author: tismer Date: Mon Sep 29 15:23:07 2003 New Revision: 1436 Modified: pypy/trunk/doc/translation/controlflow.asc Log: changed the content to be a literal block. Modified: pypy/trunk/doc/translation/controlflow.asc ============================================================================== --- pypy/trunk/doc/translation/controlflow.asc (original) +++ pypy/trunk/doc/translation/controlflow.asc Mon Sep 29 15:23:07 2003 @@ -1,105 +1,107 @@ -Hello again - - def f(i): - return g(i)+2 - def g(i): - return i+1 - -f(3) -tspace: -pyxcode.putln('def f(%s):' % sig) -res = frame.eval(executioncontext) -pyxcode.putln('return %s' % res) - - -Pyrex: def f(v1): -dis.dis(f) --> - 2 0 LOAD_GLOBAL 0 (g) - w_result = space.getitem(f.w_globals, w_varname) - - 3 LOAD_FAST 0 (i) - 6 CALL_FUNCTION 1 -space.call(w_function, w_arguments, w_kwds) -space.call(w_g, ("v1",), {}) -Pyrex: v2 = g(v1) - 9 LOAD_CONST 1 (2) -space.wrap(2) - 12 BINARY_ADD -space.add("v2", "constant 2") -Pyrex: v3 = v2 + 2 - 13 RETURN_VALUE - 14 LOAD_CONST 0 (None) - 17 RETURN_VALUE -Pyrex: return v3 - - -Result: -def f(v1): - v2 = g(v1) - v3 = v2 + 2 - return v3 - - -def h(i, j): - if i < 0: - i = j - return i+1 - -Pyrex: def h(v1, v2): ---> interpreter - 3 0 LOAD_FAST 0 (i) - 3 LOAD_CONST 1 (0) - 6 COMPARE_OP 0 (<) -"v3" = space.lt("v1", "constant 0") -Pyrex: v3 = v1 < 0 - 9 JUMP_IF_FALSE 10 (to 22) -space.is_true("v3") -Pyrex: if v3: cinline "goto Label1;" - 12 POP_TOP - - 4 13 LOAD_FAST 1 (j) - 16 STORE_FAST 0 (i) - 19 JUMP_FORWARD 1 (to 23) - >> 22 POP_TOP - -Pyrex: cinline "LabelBytecode23:" # just in case for later - 5 >> 23 LOAD_FAST 0 (i) - 26 LOAD_CONST 2 (1) - 29 BINARY_ADD -space.add("v1", "constant 2") -Pyrex: v4 = v1 + 2 - 30 RETURN_VALUE - 31 LOAD_CONST 0 (None) - 34 RETURN_VALUE -Pyrex: return v4 - -pyrex: cinline "Label1:" - 12 POP_TOP - - 4 13 LOAD_FAST 1 (j) - 16 STORE_FAST 0 (i) -(in the interpreter fastlocals now: i="v2" j="v2") - 19 JUMP_FORWARD 1 (to 23) - >> 22 POP_TOP - -(bytecode 23 already seen!) -Pyrex: v1 = v2 -Pyrex: cinline "goto LabelBytecode23;" - 5 >> 23 LOAD_FAST 0 (i) - -def h(i, j): - if i < 0: - i = j - return i+1 - -def h(v1, v2): - v3 = v1 < 0 - if v3: cinline "goto label1;" - - cinline "labelBytecode23:" - v4=v1+1 - return v4 - cinline "label1:" - v1=v2 - cinline "goto labelBytecode23;" +:: + Hello again + + def f(i): + return g(i)+2 + def g(i): + return i+1 + + f(3) + tspace: + pyxcode.putln('def f(%s):' % sig) + res = frame.eval(executioncontext) + pyxcode.putln('return %s' % res) + + + Pyrex: def f(v1): + dis.dis(f) --> + 2 0 LOAD_GLOBAL 0 (g) + w_result = space.getitem(f.w_globals, w_varname) + + 3 LOAD_FAST 0 (i) + 6 CALL_FUNCTION 1 + space.call(w_function, w_arguments, w_kwds) + space.call(w_g, ("v1",), {}) + Pyrex: v2 = g(v1) + 9 LOAD_CONST 1 (2) + space.wrap(2) + 12 BINARY_ADD + space.add("v2", "constant 2") + Pyrex: v3 = v2 + 2 + 13 RETURN_VALUE + 14 LOAD_CONST 0 (None) + 17 RETURN_VALUE + Pyrex: return v3 + + + Result: + def f(v1): + v2 = g(v1) + v3 = v2 + 2 + return v3 + + + def h(i, j): + if i < 0: + i = j + return i+1 + + Pyrex: def h(v1, v2): + --> interpreter + 3 0 LOAD_FAST 0 (i) + 3 LOAD_CONST 1 (0) + 6 COMPARE_OP 0 (<) + "v3" = space.lt("v1", "constant 0") + Pyrex: v3 = v1 < 0 + 9 JUMP_IF_FALSE 10 (to 22) + space.is_true("v3") + Pyrex: if v3: cinline "goto Label1;" + 12 POP_TOP + + 4 13 LOAD_FAST 1 (j) + 16 STORE_FAST 0 (i) + 19 JUMP_FORWARD 1 (to 23) + >> 22 POP_TOP + + Pyrex: cinline "LabelBytecode23:" # just in case for later + 5 >> 23 LOAD_FAST 0 (i) + 26 LOAD_CONST 2 (1) + 29 BINARY_ADD + space.add("v1", "constant 2") + Pyrex: v4 = v1 + 2 + 30 RETURN_VALUE + 31 LOAD_CONST 0 (None) + 34 RETURN_VALUE + Pyrex: return v4 + + pyrex: cinline "Label1:" + 12 POP_TOP + + 4 13 LOAD_FAST 1 (j) + 16 STORE_FAST 0 (i) + (in the interpreter fastlocals now: i="v2" j="v2") + 19 JUMP_FORWARD 1 (to 23) + >> 22 POP_TOP + + (bytecode 23 already seen!) + Pyrex: v1 = v2 + Pyrex: cinline "goto LabelBytecode23;" + 5 >> 23 LOAD_FAST 0 (i) + + def h(i, j): + if i < 0: + i = j + return i+1 + + def h(v1, v2): + v3 = v1 < 0 + if v3: cinline "goto label1;" + + cinline "labelBytecode23:" + v4=v1+1 + return v4 + cinline "label1:" + v1=v2 + cinline "goto labelBytecode23;" + From tismer at codespeak.net Mon Sep 29 15:23:58 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 29 Sep 2003 15:23:58 +0200 (MEST) Subject: [pypy-svn] rev 1437 - pypy/trunk/doc/translation Message-ID: <20030929132358.E52085A2A9@thoth.codespeak.net> Author: tismer Date: Mon Sep 29 15:23:58 2003 New Revision: 1437 Added: pypy/trunk/doc/translation/controlflow.txt - copied unchanged from rev 1436, pypy/trunk/doc/translation/controlflow.asc Removed: pypy/trunk/doc/translation/controlflow.asc Log: renamed to .txt, since it now works as ReST Deleted: /pypy/trunk/doc/translation/controlflow.asc ============================================================================== --- /pypy/trunk/doc/translation/controlflow.asc Mon Sep 29 15:23:58 2003 +++ (empty file) @@ -1,107 +0,0 @@ -:: - - Hello again - - def f(i): - return g(i)+2 - def g(i): - return i+1 - - f(3) - tspace: - pyxcode.putln('def f(%s):' % sig) - res = frame.eval(executioncontext) - pyxcode.putln('return %s' % res) - - - Pyrex: def f(v1): - dis.dis(f) --> - 2 0 LOAD_GLOBAL 0 (g) - w_result = space.getitem(f.w_globals, w_varname) - - 3 LOAD_FAST 0 (i) - 6 CALL_FUNCTION 1 - space.call(w_function, w_arguments, w_kwds) - space.call(w_g, ("v1",), {}) - Pyrex: v2 = g(v1) - 9 LOAD_CONST 1 (2) - space.wrap(2) - 12 BINARY_ADD - space.add("v2", "constant 2") - Pyrex: v3 = v2 + 2 - 13 RETURN_VALUE - 14 LOAD_CONST 0 (None) - 17 RETURN_VALUE - Pyrex: return v3 - - - Result: - def f(v1): - v2 = g(v1) - v3 = v2 + 2 - return v3 - - - def h(i, j): - if i < 0: - i = j - return i+1 - - Pyrex: def h(v1, v2): - --> interpreter - 3 0 LOAD_FAST 0 (i) - 3 LOAD_CONST 1 (0) - 6 COMPARE_OP 0 (<) - "v3" = space.lt("v1", "constant 0") - Pyrex: v3 = v1 < 0 - 9 JUMP_IF_FALSE 10 (to 22) - space.is_true("v3") - Pyrex: if v3: cinline "goto Label1;" - 12 POP_TOP - - 4 13 LOAD_FAST 1 (j) - 16 STORE_FAST 0 (i) - 19 JUMP_FORWARD 1 (to 23) - >> 22 POP_TOP - - Pyrex: cinline "LabelBytecode23:" # just in case for later - 5 >> 23 LOAD_FAST 0 (i) - 26 LOAD_CONST 2 (1) - 29 BINARY_ADD - space.add("v1", "constant 2") - Pyrex: v4 = v1 + 2 - 30 RETURN_VALUE - 31 LOAD_CONST 0 (None) - 34 RETURN_VALUE - Pyrex: return v4 - - pyrex: cinline "Label1:" - 12 POP_TOP - - 4 13 LOAD_FAST 1 (j) - 16 STORE_FAST 0 (i) - (in the interpreter fastlocals now: i="v2" j="v2") - 19 JUMP_FORWARD 1 (to 23) - >> 22 POP_TOP - - (bytecode 23 already seen!) - Pyrex: v1 = v2 - Pyrex: cinline "goto LabelBytecode23;" - 5 >> 23 LOAD_FAST 0 (i) - - def h(i, j): - if i < 0: - i = j - return i+1 - - def h(v1, v2): - v3 = v1 < 0 - if v3: cinline "goto label1;" - - cinline "labelBytecode23:" - v4=v1+1 - return v4 - cinline "label1:" - v1=v2 - cinline "goto labelBytecode23;" - From arigo at codespeak.net Mon Sep 29 15:24:49 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Sep 2003 15:24:49 +0200 (MEST) Subject: [pypy-svn] rev 1438 - pypy/trunk/doc/objspace Message-ID: <20030929132449.D45F45A2A9@thoth.codespeak.net> Author: arigo Date: Mon Sep 29 15:24:49 2003 New Revision: 1438 Added: pypy/trunk/doc/objspace/basicblock.asc Log: FlowObjSpace interface definition Added: pypy/trunk/doc/objspace/basicblock.asc ============================================================================== --- (empty file) +++ pypy/trunk/doc/objspace/basicblock.asc Mon Sep 29 15:24:49 2003 @@ -0,0 +1,36 @@ +the CtlFlowObjSpace is supposed to produce the following object-model +(currently just a linear list of objects referencing each other, sorry +no UML diagream :-) + +the reason we want this objectmodel as the source for doing the +translation is that otherwise we get very-hard-to-read low-level/pyrex +code with lots of gotos etc.pp. + +With the CtlFlow-object model we can basically reconstruct some +source-structure (without examining the bytecode). + +class BasicBlock: + .inputargs = [list-of-input-locals] + .locals = [list-of-all-locals-incluing-inputargs] + .operations = [list-of-operations] + .branch = + +class Variable: + pass + +class Constant: + .value = ... + +class SpaceOperation: + .opname = 'add' + .args = [list-of-variables] + .result = + +class Branch: + .args = [list-of-variables] + .target = + +class ConditionalBranch: + .condition = + .iftrue = + .iffalse = From hpk at codespeak.net Mon Sep 29 15:24:55 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 29 Sep 2003 15:24:55 +0200 (MEST) Subject: [pypy-svn] rev 1439 - pypy/trunk/src/pypy/translator Message-ID: <20030929132455.4C75E5A2A9@thoth.codespeak.net> Author: hpk Date: Mon Sep 29 15:24:54 2003 New Revision: 1439 Added: pypy/trunk/src/pypy/translator/ Log: the directory to contain "everything related" to translating pypy-RPython-code to low-level languages. The translator is supposed to work on the output of the upcoming "FlowObjSpace". the translator directory will probably also contain what was formerly in AnnObjSpace because that functionality will also operate on output from FlowObjSpace. From tismer at codespeak.net Mon Sep 29 15:54:44 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 29 Sep 2003 15:54:44 +0200 (MEST) Subject: [pypy-svn] rev 1440 - pypy/trunk/doc/funding/test Message-ID: <20030929135444.A042C5A2A9@thoth.codespeak.net> Author: tismer Date: Mon Sep 29 15:54:43 2003 New Revision: 1440 Modified: pypy/trunk/doc/funding/test/demo_wp_delete.sxw Log: enhanced delete tag Modified: pypy/trunk/doc/funding/test/demo_wp_delete.sxw ============================================================================== Binary files. No diff available. From dinu at codespeak.net Mon Sep 29 16:46:24 2003 From: dinu at codespeak.net (dinu at codespeak.net) Date: Mon, 29 Sep 2003 16:46:24 +0200 (MEST) Subject: [pypy-svn] rev 1441 - pypy/trunk/doc/funding/test Message-ID: <20030929144624.2520D5A486@thoth.codespeak.net> Author: dinu Date: Mon Sep 29 16:46:23 2003 New Revision: 1441 Added: pypy/trunk/doc/funding/test/demo_wp_delete_sf.sxw (contents, props changed) Log: test after deleting special tags Added: pypy/trunk/doc/funding/test/demo_wp_delete_sf.sxw ============================================================================== Binary file. No diff available. From dinu at codespeak.net Mon Sep 29 16:53:15 2003 From: dinu at codespeak.net (dinu at codespeak.net) Date: Mon, 29 Sep 2003 16:53:15 +0200 (MEST) Subject: [pypy-svn] rev 1442 - pypy/trunk/doc/funding/test Message-ID: <20030929145315.6CD005A486@thoth.codespeak.net> Author: dinu Date: Mon Sep 29 16:53:14 2003 New Revision: 1442 Modified: pypy/trunk/doc/funding/test/demo_wp_delete_sf.sxw Log: now with correct content.xml file Modified: pypy/trunk/doc/funding/test/demo_wp_delete_sf.sxw ============================================================================== Binary files. No diff available. From jriehl at codespeak.net Mon Sep 29 17:10:34 2003 From: jriehl at codespeak.net (jriehl at codespeak.net) Date: Mon, 29 Sep 2003 17:10:34 +0200 (MEST) Subject: [pypy-svn] rev 1443 - in pypy/trunk/src/pypy/objspace/flow: . test Message-ID: <20030929151034.73CE85A486@thoth.codespeak.net> Author: jriehl Date: Mon Sep 29 17:10:33 2003 New Revision: 1443 Added: pypy/trunk/src/pypy/objspace/flow/ pypy/trunk/src/pypy/objspace/flow/objspace.py pypy/trunk/src/pypy/objspace/flow/test/ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Log: Initial checkin of control flow analysis object space (no code, stupid test cases) Added: pypy/trunk/src/pypy/objspace/flow/objspace.py ============================================================================== Added: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Mon Sep 29 17:10:33 2003 @@ -0,0 +1,32 @@ +import autopath +from pypy.tool import test + +class TestFlowOjSpace(test.TestCase): + def setUp(self): + self.space = test.objspace('flow') + + def codetest(sefl, source, functionname, args_w): + glob = {} + exec source in glob + func = glob[functionname] + w_args = self.space.newtuple(args_w) + w_func = self.space.wrap(func) + w_kwds = self.space.newdict([]) + return self.space.build_flow(w_func, w_args, w_kwds) + + def test_nothing(self): + x = self.codetest("def f():\n" + " pass\n") + self.assertEquals(len(x), 1) + + def test_ifthenelse(self): + x = self.codetest("def f(i, j):\n" + " if i < 0:\n" + " i = j\n" + " return g(i) + 1\n", + 'f', [W_Anything()]) + self.assertEquals(x.graph_string(), + "") + +if __name__ == '__main__': + test.main() From hpk at codespeak.net Mon Sep 29 17:12:33 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 29 Sep 2003 17:12:33 +0200 (MEST) Subject: [pypy-svn] rev 1444 - pypy/trunk/src/pypy/translator/test Message-ID: <20030929151233.EA7355A486@thoth.codespeak.net> Author: hpk Date: Mon Sep 29 17:12:32 2003 New Revision: 1444 Added: pypy/trunk/src/pypy/translator/test/ pypy/trunk/src/pypy/translator/test/autopath.py pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: The first tests for emmiting the pyrex code... Added: pypy/trunk/src/pypy/translator/test/autopath.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/translator/test/autopath.py Mon Sep 29 17:12:32 2003 @@ -0,0 +1,78 @@ +""" +self cloning, automatic path configuration + +copy this into any subdirectory of pypy from which scripts need +to be run, typically all of the test subdirs. +The idea is that any such script simply issues + + import autopath + +and this will make sure that the parent directory containing "pypy" +is in sys.path. + +If you modify the master "autopath.py" version (in pypy/tool/autopath.py) +you can directly run it which will copy itself on all autopath.py files +it finds under the pypy root directory. + +This module always provides these attributes: + + pypydir pypy root directory path + this_dir directory where this autopath.py resides + +""" + + +def __dirinfo(part): + """ return (partdir, this_dir) and insert parent of partdir + into sys.path. If the parent directories dont have the part + an EnvironmentError is raised.""" + + import sys, os + try: + head = this_dir = os.path.abspath(os.path.dirname(__file__)) + except NameError: + head = this_dir = os.path.abspath(os.path.dirname(sys.argv[0])) + + while head: + partdir = head + head, tail = os.path.split(head) + if tail == part: + sys.path = [p for p in sys.path if not p.startswith(head)] + if head not in sys.path: + sys.path.insert(0, head) + return partdir, this_dir + + raise EnvironmentError, "'%s' missing in '%r'" % (pathpart,this_path) + +def __clone(): + """ clone master version of autopath.py into all subdirs """ + from os.path import join, walk + if not this_dir.endswith(join('pypy','tool')): + raise EnvironmentError("can only clone master version " + "'%s'" % join(pypydir, 'tool',_myname)) + + + def sync_walker(arg, dirname, fnames): + if _myname in fnames: + fn = join(dirname, _myname) + f = open(fn, 'rwb+') + try: + if f.read() == arg: + print "checkok", fn + else: + print "syncing", fn + f = open(fn, 'w') + f.write(arg) + finally: + f.close() + s = open(join(pypydir, 'tool', _myname), 'rb').read() + walk(pypydir, sync_walker, s) + +_myname = 'autopath.py' + +# set guaranteed attributes + +pypydir, this_dir = __dirinfo('pypy') + +if __name__ == '__main__': + __clone() Added: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Mon Sep 29 17:12:32 2003 @@ -0,0 +1,105 @@ + +import autopath +from pypy.tool import test + +from pypy.translator.genpyrex import genpyrex + +class BasicBlock: + def __init__(self, input_args, locals, operations, branch): + self.input_args = input_args + self.locals = locals + self.operations = operations + self.branch = branch + +class Variable: + def __init__(self, pseudoname): + self.pseudoname = pseudoname + +class Constant: + def __init__(self, value): + self.value = value + +class SpaceOperation: + def __init__(self, opname, args, result, branch): + self.opname = opname + self.args = args # list of variables + self.result = result # + self.branch = branch # branch + +class Branch: + def __init__(self, args, target): + self.args = args # list of variables + self.target = target # basic block instance + +class ConditionalBranch: + def __init__(self, condition, ifbranch, elsebranch): + self.condition = condition + self.ifbranch = ifbranch + self.elsebranch = elsebranch + +class EndBranch: + def __init__(self, returnvalue): + self.returnvalue = returnvalue + +class FunctionGraph: + def __init__(self, startblock, functionname): + self.startblock = startblock + self.functionname = functionname + +class TestCase(test.IntTestCase): + def test_simple_func(self): + """ + one test source: + def f(x): + return x+1 + """ + x = Variable("x") + result = Variable("result") + op = SpaceOperation("add", [x, Constant(1)], result) + endbranch = EndBranch(result) + block = BasicBlock([x], [x], + [op], + endbranch) + fun = FunctionGraph(block, "f") + result = genpyrex(fun) + self.assertEquals(result, """ +def f(x): + result = x + 1 + return result +""") + + def test_if(self): + """ + one test source: + def f(i, j): + if i < 0: + i = j + return i + """ + i = Variable("i") + j = Variable("j") + + endbranchelse = EndBranch(i) + endbranchif = EndBranch(j) + + conditionres = Variable("conditionres") + conditionop = SpaceOperation("lt", [i, Constant(0)], conditionres) + + conditionalbranch = ConditionalBranch(condition, endbranchif, endbranchelse) + + startblock = BasicBlock([i, j], [i, j, conditionres], + [conditionop], + conditionalbranch) + fun = FunctionGraph(startblock, "f") + result = genpyrex(fun) + self.assertEquals(result, """ +def f(i, j): + conditionres = i < 0 + if conditionres: cinline "goto label1;" + return i + cinline "label1:" + return j +""") + +if __name__ == '__main__': + test.main() From hpk at codespeak.net Mon Sep 29 17:14:16 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 29 Sep 2003 17:14:16 +0200 (MEST) Subject: [pypy-svn] rev 1445 - pypy/trunk/src/pypy/translator/test Message-ID: <20030929151416.9E4AC5A486@thoth.codespeak.net> Author: hpk Date: Mon Sep 29 17:14:16 2003 New Revision: 1445 Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: I resolved some whitespace problems Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/test_sourcegen.py (original) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Mon Sep 29 17:14:16 2003 @@ -77,15 +77,15 @@ return i """ i = Variable("i") - j = Variable("j") + j = Variable("j") - endbranchelse = EndBranch(i) + endbranchelse = EndBranch(i) endbranchif = EndBranch(j) conditionres = Variable("conditionres") - conditionop = SpaceOperation("lt", [i, Constant(0)], conditionres) + conditionop = SpaceOperation("lt", [i, Constant(0)], conditionres) - conditionalbranch = ConditionalBranch(condition, endbranchif, endbranchelse) + conditionalbranch = ConditionalBranch(condition, endbranchif, endbranchelse) startblock = BasicBlock([i, j], [i, j, conditionres], [conditionop], From hpk at codespeak.net Mon Sep 29 17:15:45 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 29 Sep 2003 17:15:45 +0200 (MEST) Subject: [pypy-svn] rev 1446 - pypy/trunk/src/pypy/translator/test Message-ID: <20030929151545.7A8135A486@thoth.codespeak.net> Author: hpk Date: Mon Sep 29 17:15:44 2003 New Revision: 1446 Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: Some more whitespace problems resolved... Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/test_sourcegen.py (original) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Mon Sep 29 17:15:44 2003 @@ -33,9 +33,9 @@ class ConditionalBranch: def __init__(self, condition, ifbranch, elsebranch): - self.condition = condition - self.ifbranch = ifbranch - self.elsebranch = elsebranch + self.condition = condition + self.ifbranch = ifbranch + self.elsebranch = elsebranch class EndBranch: def __init__(self, returnvalue): @@ -43,8 +43,8 @@ class FunctionGraph: def __init__(self, startblock, functionname): - self.startblock = startblock - self.functionname = functionname + self.startblock = startblock + self.functionname = functionname class TestCase(test.IntTestCase): def test_simple_func(self): From guenter at codespeak.net Mon Sep 29 17:18:33 2003 From: guenter at codespeak.net (guenter at codespeak.net) Date: Mon, 29 Sep 2003 17:18:33 +0200 (MEST) Subject: [pypy-svn] rev 1447 - pypy/trunk/src/pypy/translator/test Message-ID: <20030929151833.CC37B5A486@thoth.codespeak.net> Author: guenter Date: Mon Sep 29 17:18:33 2003 New Revision: 1447 Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: From jriehl at codespeak.net Mon Sep 29 17:20:59 2003 From: jriehl at codespeak.net (jriehl at codespeak.net) Date: Mon, 29 Sep 2003 17:20:59 +0200 (MEST) Subject: [pypy-svn] rev 1448 - pypy/trunk/src/pypy/objspace/flow/test Message-ID: <20030929152059.621D85A486@thoth.codespeak.net> Author: jriehl Date: Mon Sep 29 17:20:58 2003 New Revision: 1448 Added: pypy/trunk/src/pypy/objspace/flow/test/__init__.py pypy/trunk/src/pypy/objspace/flow/test/autopath.py Log: Adding autopath to flow control unit test directory. Added: pypy/trunk/src/pypy/objspace/flow/test/__init__.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/objspace/flow/test/__init__.py Mon Sep 29 17:20:58 2003 @@ -0,0 +1 @@ +# Empty Added: pypy/trunk/src/pypy/objspace/flow/test/autopath.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/objspace/flow/test/autopath.py Mon Sep 29 17:20:58 2003 @@ -0,0 +1,78 @@ +""" +self cloning, automatic path configuration + +copy this into any subdirectory of pypy from which scripts need +to be run, typically all of the test subdirs. +The idea is that any such script simply issues + + import autopath + +and this will make sure that the parent directory containing "pypy" +is in sys.path. + +If you modify the master "autopath.py" version (in pypy/tool/autopath.py) +you can directly run it which will copy itself on all autopath.py files +it finds under the pypy root directory. + +This module always provides these attributes: + + pypydir pypy root directory path + this_dir directory where this autopath.py resides + +""" + + +def __dirinfo(part): + """ return (partdir, this_dir) and insert parent of partdir + into sys.path. If the parent directories dont have the part + an EnvironmentError is raised.""" + + import sys, os + try: + head = this_dir = os.path.abspath(os.path.dirname(__file__)) + except NameError: + head = this_dir = os.path.abspath(os.path.dirname(sys.argv[0])) + + while head: + partdir = head + head, tail = os.path.split(head) + if tail == part: + sys.path = [p for p in sys.path if not p.startswith(head)] + if head not in sys.path: + sys.path.insert(0, head) + return partdir, this_dir + + raise EnvironmentError, "'%s' missing in '%r'" % (pathpart,this_path) + +def __clone(): + """ clone master version of autopath.py into all subdirs """ + from os.path import join, walk + if not this_dir.endswith(join('pypy','tool')): + raise EnvironmentError("can only clone master version " + "'%s'" % join(pypydir, 'tool',_myname)) + + + def sync_walker(arg, dirname, fnames): + if _myname in fnames: + fn = join(dirname, _myname) + f = open(fn, 'rwb+') + try: + if f.read() == arg: + print "checkok", fn + else: + print "syncing", fn + f = open(fn, 'w') + f.write(arg) + finally: + f.close() + s = open(join(pypydir, 'tool', _myname), 'rb').read() + walk(pypydir, sync_walker, s) + +_myname = 'autopath.py' + +# set guaranteed attributes + +pypydir, this_dir = __dirinfo('pypy') + +if __name__ == '__main__': + __clone() From tomek at codespeak.net Mon Sep 29 18:33:26 2003 From: tomek at codespeak.net (tomek at codespeak.net) Date: Mon, 29 Sep 2003 18:33:26 +0200 (MEST) Subject: [pypy-svn] rev 1449 - in pypy/trunk/src/pypy/translator: . test Message-ID: <20030929163326.E0C135AB2F@thoth.codespeak.net> Author: tomek Date: Mon Sep 29 18:33:26 2003 New Revision: 1449 Added: pypy/trunk/src/pypy/translator/__init__.py pypy/trunk/src/pypy/translator/controlflow.py pypy/trunk/src/pypy/translator/genpyrex.py pypy/trunk/src/pypy/translator/test/__init__.py Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: I refactored the files, and put temporary classes to controlflow.py We won't need these classes anymore, when the controlObjectSpace is ready. We need them for testing. Added: pypy/trunk/src/pypy/translator/__init__.py ============================================================================== Added: pypy/trunk/src/pypy/translator/controlflow.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/translator/controlflow.py Mon Sep 29 18:33:26 2003 @@ -0,0 +1,46 @@ + +import autopath + + +class BasicBlock: + def __init__(self, input_args, locals, operations, branch): + self.input_args = input_args + self.locals = locals + self.operations = operations + self.branch = branch + +class Variable: + def __init__(self, pseudoname): + self.pseudoname = pseudoname + +class Constant: + def __init__(self, value): + self.value = value + +class SpaceOperation: + def __init__(self, opname, args, result, branch): + self.opname = opname + self.args = args # list of variables + self.result = result # + self.branch = branch # branch + +class Branch: + def __init__(self, args, target): + self.args = args # list of variables + self.target = target # basic block instance + +class ConditionalBranch: + def __init__(self, condition, ifbranch, elsebranch): + self.condition = condition + self.ifbranch = ifbranch + self.elsebranch = elsebranch + +class EndBranch: + def __init__(self, returnvalue): + self.returnvalue = returnvalue + +class FunctionGraph: + def __init__(self, startblock, functionname): + self.startblock = startblock + self.functionname = functionname + Added: pypy/trunk/src/pypy/translator/genpyrex.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/translator/genpyrex.py Mon Sep 29 18:33:26 2003 @@ -0,0 +1,92 @@ + +import autopath +from pypy.tool import test +from pypy.interpreter.baseobjspace import ObjSpace +from pypy.translater.controlflow import * + +class GenPyrex: + def __init__(self, functiongraph): + self.functiongraph = functiongraph + ops = {} + oparity = {} + for (opname, opsymbol, arity, _) in ObjSpace.MethodTable: + ops[opname] = opsymbol + oparity[opname] = arity + self.ops = ops + self.oparity = oparity + + def emitcode(self): + self.blockids = {} + self.lines = [] + self.indent = 0 + self.createCodeFromGraph() + return "\n".join(self.lines) + + def putline(self, line): + self.lines.append(" " * self.indent + line) + + def createCodeFromGraph(self): + fun = self.functiongraph + inputargnames = [ var.pseudoname for var in fun.startblock.input_args ] + params = ", ".join(inputargnames) + self.putline("def %s(%s):" % (fun.functionname, params)) + self.indent += 1 + self.createCodeFromBasicBlock(fun.startblock) + self.indent -= 1 + + def _str(self, obj): + if isinstance(obj, Variable): + return obj.pseudoname + else: + return repr(obj.value) + + def createCodeFromBasicBlock(self, block): + if self.blockids.has_key(block): + self.putline('cinline "goto Label%s;"' % self.blockids[block]) + return + + blockids = self.blockids + blockids.setdefault(block, len(blockids)) + + self.putline('cinline "Label%s:"' % blockids[block]) + for op in block.operations: + opsymbol = self.ops[op.opname] + arity = self.oparity[op.opname] + assert(arity == len(op.args)) + argsnames = [self._str(arg) for arg in args] + if arity == 1 or arity == 3 or "a" <= opsymbol[0] <= "z": + + self.putline("%s = %s(%s)" % (result.pseudoname, opsymbol, ", ".join([argnames])) + else: + self.putline("%s = %s %s %s") % (result.pseudoname, argnames[0], opsymbol, argnames[1])) + + self.dispatchBranch(block.branch) + + def dispatchBranch(self, branch): + method = getattr(self, "createCodeFrom" + branch.__class__.__name__) + method(branch) + + def createCodeFromBranch(self, branch): + _str = self._str + block = branch.target + sourceargs = [_str(arg) for arg in branch.args] + targetargs = [arg.pseudoname for arg in block.input_args] + assert(len(sourceargs) == len(targetargs)) + if sourceargs: + self.putline("%s = %s" % (", ".join(targetargs), ", ".join(sourceargs)) + + self.createCodeFromBasicBlock(block) + + def createCodeFromEndBranch(self, branch): + self.putline("return %s" % self._str(branch.returnvalue)) + + + def createCodeFromConditionalBranch(self, branch): + self.putline("if %s:" % self._str(branch.condition)) + self.indent += 1 + self.dispatchBranch(ifbranch) + self.indent -= 1 + self.putline("else:") + self.indent += 1 + self.dispatchBranch(elsebranch) + self.indent -= 1 Added: pypy/trunk/src/pypy/translator/test/__init__.py ============================================================================== Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/test_sourcegen.py (original) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Mon Sep 29 18:33:26 2003 @@ -2,49 +2,9 @@ import autopath from pypy.tool import test -from pypy.translator.genpyrex import genpyrex +from pypy.translator.genpyrex import GenPyrex +from pypy.translator.controlflow import * -class BasicBlock: - def __init__(self, input_args, locals, operations, branch): - self.input_args = input_args - self.locals = locals - self.operations = operations - self.branch = branch - -class Variable: - def __init__(self, pseudoname): - self.pseudoname = pseudoname - -class Constant: - def __init__(self, value): - self.value = value - -class SpaceOperation: - def __init__(self, opname, args, result, branch): - self.opname = opname - self.args = args # list of variables - self.result = result # - self.branch = branch # branch - -class Branch: - def __init__(self, args, target): - self.args = args # list of variables - self.target = target # basic block instance - -class ConditionalBranch: - def __init__(self, condition, ifbranch, elsebranch): - self.condition = condition - self.ifbranch = ifbranch - self.elsebranch = elsebranch - -class EndBranch: - def __init__(self, returnvalue): - self.returnvalue = returnvalue - -class FunctionGraph: - def __init__(self, startblock, functionname): - self.startblock = startblock - self.functionname = functionname class TestCase(test.IntTestCase): def test_simple_func(self): @@ -91,7 +51,7 @@ [conditionop], conditionalbranch) fun = FunctionGraph(startblock, "f") - result = genpyrex(fun) + result = GenPyrex(fun).emitcode() self.assertEquals(result, """ def f(i, j): conditionres = i < 0 From tomek at codespeak.net Mon Sep 29 18:40:50 2003 From: tomek at codespeak.net (tomek at codespeak.net) Date: Mon, 29 Sep 2003 18:40:50 +0200 (MEST) Subject: [pypy-svn] rev 1450 - pypy/trunk/src/pypy/translator Message-ID: <20030929164050.6846E5AB2F@thoth.codespeak.net> Author: tomek Date: Mon Sep 29 18:40:49 2003 New Revision: 1450 Modified: pypy/trunk/src/pypy/translator/genpyrex.py Log: small typos fixed Modified: pypy/trunk/src/pypy/translator/genpyrex.py ============================================================================== --- pypy/trunk/src/pypy/translator/genpyrex.py (original) +++ pypy/trunk/src/pypy/translator/genpyrex.py Mon Sep 29 18:40:49 2003 @@ -2,7 +2,7 @@ import autopath from pypy.tool import test from pypy.interpreter.baseobjspace import ObjSpace -from pypy.translater.controlflow import * +from pypy.translator.controlflow import * class GenPyrex: def __init__(self, functiongraph): @@ -37,8 +37,10 @@ def _str(self, obj): if isinstance(obj, Variable): return obj.pseudoname - else: + elif isinstance(obj, Constant): return repr(obj.value) + else: + raise ValueError("Unknow class: %s" % obj.__class__) def createCodeFromBasicBlock(self, block): if self.blockids.has_key(block): @@ -56,9 +58,9 @@ argsnames = [self._str(arg) for arg in args] if arity == 1 or arity == 3 or "a" <= opsymbol[0] <= "z": - self.putline("%s = %s(%s)" % (result.pseudoname, opsymbol, ", ".join([argnames])) + self.putline("%s = %s(%s)" % (result.pseudoname, opsymbol, ", ".join([argnames]))) else: - self.putline("%s = %s %s %s") % (result.pseudoname, argnames[0], opsymbol, argnames[1])) + self.putline("%s = %s %s %s" % (result.pseudoname, argnames[0], opsymbol, argnames[1])) self.dispatchBranch(block.branch) @@ -73,7 +75,7 @@ targetargs = [arg.pseudoname for arg in block.input_args] assert(len(sourceargs) == len(targetargs)) if sourceargs: - self.putline("%s = %s" % (", ".join(targetargs), ", ".join(sourceargs)) + self.putline("%s = %s" % (", ".join(targetargs), ", ".join(sourceargs))) self.createCodeFromBasicBlock(block) From tomek at codespeak.net Mon Sep 29 18:49:42 2003 From: tomek at codespeak.net (tomek at codespeak.net) Date: Mon, 29 Sep 2003 18:49:42 +0200 (MEST) Subject: [pypy-svn] rev 1451 - in pypy/trunk/src/pypy/translator: . test Message-ID: <20030929164942.A2FC05AB2F@thoth.codespeak.net> Author: tomek Date: Mon Sep 29 18:49:42 2003 New Revision: 1451 Modified: pypy/trunk/src/pypy/translator/controlflow.py pypy/trunk/src/pypy/translator/genpyrex.py pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: The tests are running, but failing Modified: pypy/trunk/src/pypy/translator/controlflow.py ============================================================================== --- pypy/trunk/src/pypy/translator/controlflow.py (original) +++ pypy/trunk/src/pypy/translator/controlflow.py Mon Sep 29 18:49:42 2003 @@ -18,11 +18,10 @@ self.value = value class SpaceOperation: - def __init__(self, opname, args, result, branch): + def __init__(self, opname, args, result): self.opname = opname self.args = args # list of variables self.result = result # - self.branch = branch # branch class Branch: def __init__(self, args, target): Modified: pypy/trunk/src/pypy/translator/genpyrex.py ============================================================================== --- pypy/trunk/src/pypy/translator/genpyrex.py (original) +++ pypy/trunk/src/pypy/translator/genpyrex.py Mon Sep 29 18:49:42 2003 @@ -55,12 +55,12 @@ opsymbol = self.ops[op.opname] arity = self.oparity[op.opname] assert(arity == len(op.args)) - argsnames = [self._str(arg) for arg in args] + argnames = [self._str(arg) for arg in op.args] if arity == 1 or arity == 3 or "a" <= opsymbol[0] <= "z": - self.putline("%s = %s(%s)" % (result.pseudoname, opsymbol, ", ".join([argnames]))) + self.putline("%s = %s(%s)" % (op.result.pseudoname, opsymbol, ", ".join([argnames]))) else: - self.putline("%s = %s %s %s" % (result.pseudoname, argnames[0], opsymbol, argnames[1])) + self.putline("%s = %s %s %s" % (op.result.pseudoname, argnames[0], opsymbol, argnames[1])) self.dispatchBranch(block.branch) @@ -86,9 +86,10 @@ def createCodeFromConditionalBranch(self, branch): self.putline("if %s:" % self._str(branch.condition)) self.indent += 1 - self.dispatchBranch(ifbranch) + self.dispatchBranch(branch.ifbranch) self.indent -= 1 self.putline("else:") self.indent += 1 - self.dispatchBranch(elsebranch) + self.dispatchBranch(branch.elsebranch) self.indent -= 1 + Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/test_sourcegen.py (original) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Mon Sep 29 18:49:42 2003 @@ -21,7 +21,7 @@ [op], endbranch) fun = FunctionGraph(block, "f") - result = genpyrex(fun) + result = GenPyrex(fun).emitcode() self.assertEquals(result, """ def f(x): result = x + 1 @@ -45,7 +45,7 @@ conditionres = Variable("conditionres") conditionop = SpaceOperation("lt", [i, Constant(0)], conditionres) - conditionalbranch = ConditionalBranch(condition, endbranchif, endbranchelse) + conditionalbranch = ConditionalBranch(conditionres, endbranchif, endbranchelse) startblock = BasicBlock([i, j], [i, j, conditionres], [conditionop], From tismer at codespeak.net Mon Sep 29 20:04:33 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 29 Sep 2003 20:04:33 +0200 (MEST) Subject: [pypy-svn] rev 1452 - pypy/trunk/doc/funding/makedoc Message-ID: <20030929180433.CBF095AB2F@thoth.codespeak.net> Author: tismer Date: Mon Sep 29 20:04:30 2003 New Revision: 1452 Added: pypy/trunk/doc/funding/makedoc/ pypy/trunk/doc/funding/makedoc/crossreferences.asc pypy/trunk/doc/funding/makedoc/doclist.asc pypy/trunk/doc/funding/makedoc/mkdoclist.py pypy/trunk/doc/funding/makedoc/mkxref2.py pypy/trunk/doc/funding/makedoc/part_b.pdf (contents, props changed) pypy/trunk/doc/funding/makedoc/part_b.sxw (contents, props changed) pypy/trunk/doc/funding/makedoc/wp-tablebegin.asc pypy/trunk/doc/funding/makedoc/wp-tableend.asc pypy/trunk/doc/funding/makedoc/wp-toptable.asc Log: first ugly generated PDF document :-) Added: pypy/trunk/doc/funding/makedoc/crossreferences.asc ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/makedoc/crossreferences.asc Mon Sep 29 20:04:30 2003 @@ -0,0 +1,25 @@ +.. _WP10: #wp10 +.. _WP20: #wp20 +.. _WP21: #wp21 +.. _WP22: #wp22 +.. _WP30: #wp30 +.. _WP31: #wp31 +.. _WP32: #wp32 +.. _WP33: #wp33 +.. _WP40: #wp40 +.. _WP41: #wp41 +.. _WP42: #wp42 +.. _WP43: #wp43 +.. _WP44: #wp44 +.. _WP50: #wp50 +.. _WP51: #wp51 +.. _WP52: #wp52 +.. _WP53: #wp53 +.. _WP54: #wp54 +.. _WP55: #wp55 +.. _WP56: #wp56 +.. _WP60: #wp60 +.. _WP70: #wp70 +.. _WP80: #wp80 + +.. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file Added: pypy/trunk/doc/funding/makedoc/doclist.asc ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/makedoc/doclist.asc Mon Sep 29 20:04:30 2003 @@ -0,0 +1,37 @@ +B0.0_preamble.html +B1.0_objectives.html +B2.0.0_new_relevance.html +B3.0_impact.html +B3.1_standards.html +B4.0_resources.html +B4.1_subcontracting.html +B4.2_other_countries.html +B5.0_management.html +B6.0_detailed_implementation.html +B6.1_plan_introduction.html +B6.4_workpackage_list.html +B6.7.wp10_management.html +B6.7.wp20_interpreter.html +B6.7.wp21_interpreter_completion.html +B6.7.wp22_cpython_to_python.html +B6.7.wp30_compiler.html +B6.7.wp31_translation_of_rpython.html +B6.7.wp32_bootstrap_redesign.html +B6.7.wp33_bootstrap_runtime.html +B6.7.wp40_high_performance.html +B6.7.wp41_integrate_stackless.html +B6.7.wp42_several_obj_impl.html +B6.7.wp43_translator_optimisations.html +B6.7.wp44_integrate_psyco.html +B6.7.wp50_validations.html +B6.7.wp51_embedded_device.html +B6.7.wp52_small_os_pre.html +B6.7.wp53_small_os.html +B6.7.wp54_simd_architecture.html +B6.7.wp55_numerical_package.html +B6.7.wp56_load_balancing.html +B6.7.wp60_documentation.html +B6.7.wp70_maintenance.html +B6.7.wp80_synchronisation.html +B7.0_gender_ethical.html +B7.2_other_issues.html Added: pypy/trunk/doc/funding/makedoc/mkdoclist.py ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/makedoc/mkdoclist.py Mon Sep 29 20:04:30 2003 @@ -0,0 +1,67 @@ +import os, sys, re + +def get_file_list(): + flis = os.listdir("..") + pat = "[bB][0-9].[0-9]" + lis = [] + for name in flis: + if re.match(pat, name): + pre, post = os.path.splitext(name) + if post.lower() == ".txt": + lis.append(pre) + lis.sort() + return lis + +def mangle_wp_file(txt): + lines = txt.split("\n") + for line in lines: + if line.startswith(".. |wp|"): + break + else: + # no WP file + return txt + #.. |wp| replace:: WP21 + wp = line.split()[-1] + # make a local target + lines[:0] =[ + ".. _%s:" % wp.lower(), + "", + ":DELETE:BEGIN", + "", + "This is the anchor %s_" % wp, + "", + ":DELETE:END", + ""] + # merge the crossref file in, with modifications + for line in lines: + if line.startswith(".. include:: crossreferences.asc"): + break + inset = [] + matchline = line + for line in file("crossreferences.asc"): + parts = line.split() + if len(parts)>=2 and parts[1] == "_%s:" % wp: + continue + inset.append(line.rstrip()) + p = lines.index(matchline) + lines[p:p+1] = inset + txt = "\n".join(lines) + return txt + +def make_new_files(names): + for name in names: + txt = file(os.path.join("..", name+".txt")).read() + txt = mangle_wp_file(txt) + file(name+".txt", "w").write(txt) + +def make_doc_list(names): + f = file("doclist.asc", "w") + for name in names: + print>>f, "%s.html" % name + +def make_doc(): + names = get_file_list() + make_new_files(names) + make_doc_list(names) + + \ No newline at end of file Added: pypy/trunk/doc/funding/makedoc/mkxref2.py ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/makedoc/mkxref2.py Mon Sep 29 20:04:30 2003 @@ -0,0 +1,21 @@ +# mkxref2.py +# +# Generate crossreferences.asc +# this is generated from the upper directory, +# creating local references into +# the local document. + +import os, sys, re + +def build_xref_file(): + lines = file("../crossreferences.asc").readlines() + for i in range(len(lines)): + parts = lines[i].split() + if len(parts) >= 2: + if parts[0] == ".." and parts[1].startswith("_") and parts[1].endswith(":"): + txt = parts[1][1:-1] + lines[i] = "%s %s %s\n" % (parts[0], parts[1], "#"+txt.lower()) + file("crossreferences.asc", "w").writelines(lines) + +if __name__ == "__main__": + build_xref_file() \ No newline at end of file Added: pypy/trunk/doc/funding/makedoc/part_b.pdf ============================================================================== Binary file. No diff available. Added: pypy/trunk/doc/funding/makedoc/part_b.sxw ============================================================================== Binary file. No diff available. Added: pypy/trunk/doc/funding/makedoc/wp-tablebegin.asc ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/makedoc/wp-tablebegin.asc Mon Sep 29 20:04:30 2003 @@ -0,0 +1,11 @@ + +.. start a surounding single table cell with a minimum height. + +.. raw:: html + +



+

+ + + + + +
Added: pypy/trunk/doc/funding/makedoc/wp-tableend.asc ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/makedoc/wp-tableend.asc Mon Sep 29 20:04:30 2003 @@ -0,0 +1,8 @@ + +.. end a single surrounding table. + +.. raw:: html + +
Added: pypy/trunk/doc/funding/makedoc/wp-toptable.asc ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/makedoc/wp-toptable.asc Mon Sep 29 20:04:30 2003 @@ -0,0 +1,185 @@ + +.. create a nearly exact table for the WPs +.. note that you must define *all* the |xxx| references. + +.. raw:: html + +

+ +

+ +:DELETE:BEGIN + +toplevel ++++++++++++ + +sublevel1 +.................. + +sublevel2 +########## + +:DELETE:END + +|title| +============== + +.. raw:: html + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+

Workpackage number +

+ 		+ + +|wp| + +.. raw:: html + + + +

Start date or starting + event:

+ 		+ + +|start| + +.. raw:: html + + +
+

Participant id

+ 		+ + +|p1| + +.. raw:: html + + + + + +|p2| + +.. raw:: html + + + + + +|p3| + +.. raw:: html + + + + + +|p4| + +.. raw:: html + + + + + +|p5| + +.. raw:: html + + + + + +|p6| + +.. raw:: html + + +
+

Person-months per + participant:

+ 		+ + +|m1| + +.. raw:: html + + + + + +|m2| + +.. raw:: html + + + + + +|m3| + +.. raw:: html + + + + + +|m4| + +.. raw:: html + + + + + +|m5| + +.. raw:: html + + + + + +|m6| + +.. raw:: html + + +
From tismer at codespeak.net Mon Sep 29 20:09:46 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 29 Sep 2003 20:09:46 +0200 (MEST) Subject: [pypy-svn] rev 1453 - pypy/trunk/doc/funding/makedoc Message-ID: <20030929180946.9F4B15AB2F@thoth.codespeak.net> Author: tismer Date: Mon Sep 29 20:09:45 2003 New Revision: 1453 Modified: pypy/trunk/doc/funding/makedoc/ (props changed) pypy/trunk/doc/funding/makedoc/crossreferences.asc (contents, props changed) pypy/trunk/doc/funding/makedoc/doclist.asc (contents, props changed) pypy/trunk/doc/funding/makedoc/mkdoclist.py (contents, props changed) pypy/trunk/doc/funding/makedoc/mkxref2.py (contents, props changed) pypy/trunk/doc/funding/makedoc/wp-tablebegin.asc (contents, props changed) pypy/trunk/doc/funding/makedoc/wp-tableend.asc (contents, props changed) pypy/trunk/doc/funding/makedoc/wp-toptable.asc (contents, props changed) Log: fixed eol of new files. Modified: pypy/trunk/doc/funding/makedoc/crossreferences.asc ============================================================================== --- pypy/trunk/doc/funding/makedoc/crossreferences.asc (original) +++ pypy/trunk/doc/funding/makedoc/crossreferences.asc Mon Sep 29 20:09:45 2003 @@ -1,25 +1,25 @@ -.. _WP10: #wp10 -.. _WP20: #wp20 -.. _WP21: #wp21 -.. _WP22: #wp22 -.. _WP30: #wp30 -.. _WP31: #wp31 -.. _WP32: #wp32 -.. _WP33: #wp33 -.. _WP40: #wp40 -.. _WP41: #wp41 -.. _WP42: #wp42 -.. _WP43: #wp43 -.. _WP44: #wp44 -.. _WP50: #wp50 -.. _WP51: #wp51 -.. _WP52: #wp52 -.. _WP53: #wp53 -.. _WP54: #wp54 -.. _WP55: #wp55 -.. _WP56: #wp56 -.. _WP60: #wp60 -.. _WP70: #wp70 -.. _WP80: #wp80 - +.. _WP10: #wp10 +.. _WP20: #wp20 +.. _WP21: #wp21 +.. _WP22: #wp22 +.. _WP30: #wp30 +.. _WP31: #wp31 +.. _WP32: #wp32 +.. _WP33: #wp33 +.. _WP40: #wp40 +.. _WP41: #wp41 +.. _WP42: #wp42 +.. _WP43: #wp43 +.. _WP44: #wp44 +.. _WP50: #wp50 +.. _WP51: #wp51 +.. _WP52: #wp52 +.. _WP53: #wp53 +.. _WP54: #wp54 +.. _WP55: #wp55 +.. _WP56: #wp56 +.. _WP60: #wp60 +.. _WP70: #wp70 +.. _WP80: #wp80 + .. |e| unicode:: 0x20 .. doesn't work with plain spaces \ No newline at end of file Modified: pypy/trunk/doc/funding/makedoc/doclist.asc ============================================================================== --- pypy/trunk/doc/funding/makedoc/doclist.asc (original) +++ pypy/trunk/doc/funding/makedoc/doclist.asc Mon Sep 29 20:09:45 2003 @@ -1,37 +1,37 @@ -B0.0_preamble.html -B1.0_objectives.html -B2.0.0_new_relevance.html -B3.0_impact.html -B3.1_standards.html -B4.0_resources.html -B4.1_subcontracting.html -B4.2_other_countries.html -B5.0_management.html -B6.0_detailed_implementation.html -B6.1_plan_introduction.html -B6.4_workpackage_list.html -B6.7.wp10_management.html -B6.7.wp20_interpreter.html -B6.7.wp21_interpreter_completion.html -B6.7.wp22_cpython_to_python.html -B6.7.wp30_compiler.html -B6.7.wp31_translation_of_rpython.html -B6.7.wp32_bootstrap_redesign.html -B6.7.wp33_bootstrap_runtime.html -B6.7.wp40_high_performance.html -B6.7.wp41_integrate_stackless.html -B6.7.wp42_several_obj_impl.html -B6.7.wp43_translator_optimisations.html -B6.7.wp44_integrate_psyco.html -B6.7.wp50_validations.html -B6.7.wp51_embedded_device.html -B6.7.wp52_small_os_pre.html -B6.7.wp53_small_os.html -B6.7.wp54_simd_architecture.html -B6.7.wp55_numerical_package.html -B6.7.wp56_load_balancing.html -B6.7.wp60_documentation.html -B6.7.wp70_maintenance.html -B6.7.wp80_synchronisation.html -B7.0_gender_ethical.html -B7.2_other_issues.html +B0.0_preamble.html +B1.0_objectives.html +B2.0.0_new_relevance.html +B3.0_impact.html +B3.1_standards.html +B4.0_resources.html +B4.1_subcontracting.html +B4.2_other_countries.html +B5.0_management.html +B6.0_detailed_implementation.html +B6.1_plan_introduction.html +B6.4_workpackage_list.html +B6.7.wp10_management.html +B6.7.wp20_interpreter.html +B6.7.wp21_interpreter_completion.html +B6.7.wp22_cpython_to_python.html +B6.7.wp30_compiler.html +B6.7.wp31_translation_of_rpython.html +B6.7.wp32_bootstrap_redesign.html +B6.7.wp33_bootstrap_runtime.html +B6.7.wp40_high_performance.html +B6.7.wp41_integrate_stackless.html +B6.7.wp42_several_obj_impl.html +B6.7.wp43_translator_optimisations.html +B6.7.wp44_integrate_psyco.html +B6.7.wp50_validations.html +B6.7.wp51_embedded_device.html +B6.7.wp52_small_os_pre.html +B6.7.wp53_small_os.html +B6.7.wp54_simd_architecture.html +B6.7.wp55_numerical_package.html +B6.7.wp56_load_balancing.html +B6.7.wp60_documentation.html +B6.7.wp70_maintenance.html +B6.7.wp80_synchronisation.html +B7.0_gender_ethical.html +B7.2_other_issues.html Modified: pypy/trunk/doc/funding/makedoc/mkdoclist.py ============================================================================== --- pypy/trunk/doc/funding/makedoc/mkdoclist.py (original) +++ pypy/trunk/doc/funding/makedoc/mkdoclist.py Mon Sep 29 20:09:45 2003 @@ -1,67 +1,67 @@ -import os, sys, re - -def get_file_list(): - flis = os.listdir("..") - pat = "[bB][0-9].[0-9]" - lis = [] - for name in flis: - if re.match(pat, name): - pre, post = os.path.splitext(name) - if post.lower() == ".txt": - lis.append(pre) - lis.sort() - return lis - -def mangle_wp_file(txt): - lines = txt.split("\n") - for line in lines: - if line.startswith(".. |wp|"): - break - else: - # no WP file - return txt - #.. |wp| replace:: WP21 - wp = line.split()[-1] - # make a local target - lines[:0] =[ - ".. _%s:" % wp.lower(), - "", - ":DELETE:BEGIN", - "", - "This is the anchor %s_" % wp, - "", - ":DELETE:END", - ""] - # merge the crossref file in, with modifications - for line in lines: - if line.startswith(".. include:: crossreferences.asc"): - break - inset = [] - matchline = line - for line in file("crossreferences.asc"): - parts = line.split() - if len(parts)>=2 and parts[1] == "_%s:" % wp: - continue - inset.append(line.rstrip()) - p = lines.index(matchline) - lines[p:p+1] = inset - txt = "\n".join(lines) - return txt - -def make_new_files(names): - for name in names: - txt = file(os.path.join("..", name+".txt")).read() - txt = mangle_wp_file(txt) - file(name+".txt", "w").write(txt) - -def make_doc_list(names): - f = file("doclist.asc", "w") - for name in names: - print>>f, "%s.html" % name - -def make_doc(): - names = get_file_list() - make_new_files(names) - make_doc_list(names) - +import os, sys, re + +def get_file_list(): + flis = os.listdir("..") + pat = "[bB][0-9].[0-9]" + lis = [] + for name in flis: + if re.match(pat, name): + pre, post = os.path.splitext(name) + if post.lower() == ".txt": + lis.append(pre) + lis.sort() + return lis + +def mangle_wp_file(txt): + lines = txt.split("\n") + for line in lines: + if line.startswith(".. |wp|"): + break + else: + # no WP file + return txt + #.. |wp| replace:: WP21 + wp = line.split()[-1] + # make a local target + lines[:0] =[ + ".. _%s:" % wp.lower(), + "", + ":DELETE:BEGIN", + "", + "This is the anchor %s_" % wp, + "", + ":DELETE:END", + ""] + # merge the crossref file in, with modifications + for line in lines: + if line.startswith(".. include:: crossreferences.asc"): + break + inset = [] + matchline = line + for line in file("crossreferences.asc"): + parts = line.split() + if len(parts)>=2 and parts[1] == "_%s:" % wp: + continue + inset.append(line.rstrip()) + p = lines.index(matchline) + lines[p:p+1] = inset + txt = "\n".join(lines) + return txt + +def make_new_files(names): + for name in names: + txt = file(os.path.join("..", name+".txt")).read() + txt = mangle_wp_file(txt) + file(name+".txt", "w").write(txt) + +def make_doc_list(names): + f = file("doclist.asc", "w") + for name in names: + print>>f, "%s.html" % name + +def make_doc(): + names = get_file_list() + make_new_files(names) + make_doc_list(names) + \ No newline at end of file Modified: pypy/trunk/doc/funding/makedoc/mkxref2.py ============================================================================== --- pypy/trunk/doc/funding/makedoc/mkxref2.py (original) +++ pypy/trunk/doc/funding/makedoc/mkxref2.py Mon Sep 29 20:09:45 2003 @@ -1,21 +1,21 @@ -# mkxref2.py -# -# Generate crossreferences.asc -# this is generated from the upper directory, -# creating local references into -# the local document. - -import os, sys, re - -def build_xref_file(): - lines = file("../crossreferences.asc").readlines() - for i in range(len(lines)): - parts = lines[i].split() - if len(parts) >= 2: - if parts[0] == ".." and parts[1].startswith("_") and parts[1].endswith(":"): - txt = parts[1][1:-1] - lines[i] = "%s %s %s\n" % (parts[0], parts[1], "#"+txt.lower()) - file("crossreferences.asc", "w").writelines(lines) - -if __name__ == "__main__": +# mkxref2.py +# +# Generate crossreferences.asc +# this is generated from the upper directory, +# creating local references into +# the local document. + +import os, sys, re + +def build_xref_file(): + lines = file("../crossreferences.asc").readlines() + for i in range(len(lines)): + parts = lines[i].split() + if len(parts) >= 2: + if parts[0] == ".." and parts[1].startswith("_") and parts[1].endswith(":"): + txt = parts[1][1:-1] + lines[i] = "%s %s %s\n" % (parts[0], parts[1], "#"+txt.lower()) + file("crossreferences.asc", "w").writelines(lines) + +if __name__ == "__main__": build_xref_file() \ No newline at end of file Modified: pypy/trunk/doc/funding/makedoc/wp-tablebegin.asc ============================================================================== --- pypy/trunk/doc/funding/makedoc/wp-tablebegin.asc (original) +++ pypy/trunk/doc/funding/makedoc/wp-tablebegin.asc Mon Sep 29 20:09:45 2003 @@ -1,11 +1,11 @@ - -.. start a surounding single table cell with a minimum height. - -.. raw:: html - -



-

- - - - + +
+ +.. start a surounding single table cell with a minimum height. + +.. raw:: html + +



+

+ + + + - -
Modified: pypy/trunk/doc/funding/makedoc/wp-tableend.asc ============================================================================== --- pypy/trunk/doc/funding/makedoc/wp-tableend.asc (original) +++ pypy/trunk/doc/funding/makedoc/wp-tableend.asc Mon Sep 29 20:09:45 2003 @@ -1,8 +1,8 @@ - -.. end a single surrounding table. - -.. raw:: html - -
+ +.. end a single surrounding table. + +.. raw:: html + +
Modified: pypy/trunk/doc/funding/makedoc/wp-toptable.asc ============================================================================== --- pypy/trunk/doc/funding/makedoc/wp-toptable.asc (original) +++ pypy/trunk/doc/funding/makedoc/wp-toptable.asc Mon Sep 29 20:09:45 2003 @@ -1,185 +1,185 @@ - -.. create a nearly exact table for the WPs -.. note that you must define *all* the |xxx| references. - -.. raw:: html - -

- -

- -:DELETE:BEGIN - -toplevel -+++++++++++ - -sublevel1 -.................. - -sublevel2 -########## - -:DELETE:END - -|title| -============== - -.. raw:: html - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-

Workpackage number -

- 		- - -|wp| - -.. raw:: html - - - -

Start date or starting - event:

- 		- - -|start| - -.. raw:: html - - -
-

Participant id

- 		- - -|p1| - -.. raw:: html - - - - - -|p2| - -.. raw:: html - - - - - -|p3| - -.. raw:: html - - - - - -|p4| - -.. raw:: html - - - - - -|p5| - -.. raw:: html - - - - - -|p6| - -.. raw:: html - - -
-

Person-months per - participant:

- 		- - -|m1| - -.. raw:: html - - - - - -|m2| - -.. raw:: html - - - - - -|m3| - -.. raw:: html - - - - - -|m4| - -.. raw:: html - - - - - -|m5| - -.. raw:: html - - - - - -|m6| - -.. raw:: html - - -
+ +.. create a nearly exact table for the WPs +.. note that you must define *all* the |xxx| references. + +.. raw:: html + +

+ +

+ +:DELETE:BEGIN + +toplevel ++++++++++++ + +sublevel1 +.................. + +sublevel2 +########## + +:DELETE:END + +|title| +============== + +.. raw:: html + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+

Workpackage number +

+ 		+ + +|wp| + +.. raw:: html + + + +

Start date or starting + event:

+ 		+ + +|start| + +.. raw:: html + + +
+

Participant id

+ 		+ + +|p1| + +.. raw:: html + + + + + +|p2| + +.. raw:: html + + + + + +|p3| + +.. raw:: html + + + + + +|p4| + +.. raw:: html + + + + + +|p5| + +.. raw:: html + + + + + +|p6| + +.. raw:: html + + +
+

Person-months per + participant:

+ 		+ + +|m1| + +.. raw:: html + + + + + +|m2| + +.. raw:: html + + + + + +|m3| + +.. raw:: html + + + + + +|m4| + +.. raw:: html + + + + + +|m5| + +.. raw:: html + + + + + +|m6| + +.. raw:: html + + +
From tismer at codespeak.net Mon Sep 29 20:10:25 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 29 Sep 2003 20:10:25 +0200 (MEST) Subject: [pypy-svn] rev 1454 - pypy/trunk/src/pypy/tool Message-ID: <20030929181025.976F35AB2F@thoth.codespeak.net> Author: tismer Date: Mon Sep 29 20:10:25 2003 New Revision: 1454 Modified: pypy/trunk/src/pypy/tool/fixeol Log: added .asc to the files to be eol dependent. Modified: pypy/trunk/src/pypy/tool/fixeol ============================================================================== --- pypy/trunk/src/pypy/tool/fixeol (original) +++ pypy/trunk/src/pypy/tool/fixeol Mon Sep 29 20:10:25 2003 @@ -52,7 +52,7 @@ def fixpyfiles(ignored, dirname, fnames): numpyfiles = 0 for fname in fnames: - if fname.endswith('.py') or fname.endswith('.txt'): + if fname.endswith('.py') or fname.endswith('.txt') or fname.endswith('.asc'): asserttextfile(os.path.join(dirname, fname)) numpyfiles += 1 if numpyfiles: From tismer at codespeak.net Mon Sep 29 20:45:40 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 29 Sep 2003 20:45:40 +0200 (MEST) Subject: [pypy-svn] rev 1455 - pypy/trunk/doc/funding/makedoc Message-ID: <20030929184540.075E15AB2F@thoth.codespeak.net> Author: tismer Date: Mon Sep 29 20:45:40 2003 New Revision: 1455 Modified: pypy/trunk/doc/funding/makedoc/mkdoclist.py pypy/trunk/doc/funding/makedoc/part_b.sxw Log: temporary steps... Modified: pypy/trunk/doc/funding/makedoc/mkdoclist.py ============================================================================== --- pypy/trunk/doc/funding/makedoc/mkdoclist.py (original) +++ pypy/trunk/doc/funding/makedoc/mkdoclist.py Mon Sep 29 20:45:40 2003 @@ -24,14 +24,15 @@ wp = line.split()[-1] # make a local target lines[:0] =[ - ".. _%s:" % wp.lower(), - "", ":DELETE:BEGIN", "", "This is the anchor %s_" % wp, "", ":DELETE:END", - ""] + "", + ".. _%s:" % wp.lower(), + "", + ] # merge the crossref file in, with modifications for line in lines: if line.startswith(".. include:: crossreferences.asc"): @@ -64,4 +65,5 @@ make_new_files(names) make_doc_list(names) - \ No newline at end of file +if __name__ == "__main__": + make_doc() \ No newline at end of file Modified: pypy/trunk/doc/funding/makedoc/part_b.sxw ============================================================================== Binary files. No diff available. From jriehl at codespeak.net Mon Sep 29 20:47:26 2003 From: jriehl at codespeak.net (jriehl at codespeak.net) Date: Mon, 29 Sep 2003 20:47:26 +0200 (MEST) Subject: [pypy-svn] rev 1456 - in pypy/trunk/src/pypy/objspace/flow: . test Message-ID: <20030929184726.9C2445AB2F@thoth.codespeak.net> Author: jriehl Date: Mon Sep 29 20:47:25 2003 New Revision: 1456 Added: pypy/trunk/src/pypy/objspace/flow/__init__.py (contents, props changed) pypy/trunk/src/pypy/objspace/flow/wrapper.py (contents, props changed) Modified: pypy/trunk/src/pypy/objspace/flow/ (props changed) pypy/trunk/src/pypy/objspace/flow/objspace.py (contents, props changed) pypy/trunk/src/pypy/objspace/flow/test/ (props changed) pypy/trunk/src/pypy/objspace/flow/test/__init__.py (props changed) pypy/trunk/src/pypy/objspace/flow/test/autopath.py (contents, props changed) pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py (contents, props changed) Log: End of the day checkin for first day of sprint Added: pypy/trunk/src/pypy/objspace/flow/__init__.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/objspace/flow/__init__.py Mon Sep 29 20:47:25 2003 @@ -0,0 +1,2 @@ +from objspace import FlowObjSpace +Space = FlowObjSpace Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/objspace.py Mon Sep 29 20:47:25 2003 @@ -0,0 +1,100 @@ +import operator +import pypy +from pypy.interpreter.baseobjspace import ObjSpace +from pypy.interpreter.pycode import PyCode +from pypy.objspace.flow.wrapper import * +from pypy.translator.controlflow import * + +class FlowObjSpace(ObjSpace): + def initialize(self): + self.w_builtins = W_Variable() + #self.make_builtins() + #self.make_sys() + + def newdict(self, items_w): + # XXX Issue a delayed command to create a dictionary + return W_Variable() + + def newtuple(self, args_w): + # XXX Issue a delayed command to create a tuple and assign to a new W_Variable + return W_Variable() + + def getattr(self, w_obj, w_key): + # XXX Issue a delayed command + return W_Variable() + + def wrap(self, obj): + if isinstance(obj, W_Object): + raise TypeError("already wrapped: " + repr(obj)) + return W_Constant(obj) + + def unwrap(self, w_obj): + if isinstance(w_obj, W_Object): + return w_obj.unwrap() + else: + raise TypeError("not wrapped: " + repr(w_obj)) + + def build_flow(self, func, w_args, w_kwds): + """ + """ + code = func.func_code + bytecode = PyCode() + bytecode._from_code(code) + w_globals = self.wrap(func.func_globals) + frame = bytecode.create_frame(self, w_globals) + arg_list = [W_Variable() for w in frame.fastlocals_w] + frame.setfastscope(arg_list) + self._crnt_ops = [] + self._crnt_block = BasicBlock(arg_list, [], self._crnt_ops, None) + self._graph = FunctionGraph(self._crnt_block, code.co_name) + frames = [frame] + while len(frames) > 0: + crnt_frame = frames.pop() + ret_val = crnt_frame.run() + self._crnt_block.branch = EndBranch(ret_val) + g = self._graph + del self._graph + del self._crnt_block + del self._crnt_ops + return g + +# ______________________________________________________________________ + +def make_op(name, symbol, arity, specialnames): + if hasattr(FlowObjSpace, name): + return # Shouldn't do it + + op = getattr(operator, name, None) + if not op: + return # Can't do it + + def generic_operator(self, *args_w): + assert len(args_w) == arity, name+" got the wrong number of arguments" + args = [] + for w_arg in args_w: + try: + arg = self.unwrap(w_arg) + except UnwrapException: + break + else: + args.append(arg) + else: + # All arguments are constants: call the operator now + try: + result = op(*args) + except: + self.reraise() + else: + return self.wrap(result) + + w_result = W_Variable() + self._crnt_ops.append(SpaceOperation(name, args_w, w_result)) + return w_result + + setattr(FlowObjSpace, name, generic_operator) + +for line in ObjSpace.MethodTable: + make_op(*line) + +# ______________________________________________________________________ +# End of objspace.py Modified: pypy/trunk/src/pypy/objspace/flow/test/autopath.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/test/autopath.py (original) +++ pypy/trunk/src/pypy/objspace/flow/test/autopath.py Mon Sep 29 20:47:25 2003 @@ -1,78 +1,78 @@ -""" -self cloning, automatic path configuration - -copy this into any subdirectory of pypy from which scripts need -to be run, typically all of the test subdirs. -The idea is that any such script simply issues - - import autopath - -and this will make sure that the parent directory containing "pypy" -is in sys.path. - -If you modify the master "autopath.py" version (in pypy/tool/autopath.py) -you can directly run it which will copy itself on all autopath.py files -it finds under the pypy root directory. - -This module always provides these attributes: - - pypydir pypy root directory path - this_dir directory where this autopath.py resides - -""" - - -def __dirinfo(part): - """ return (partdir, this_dir) and insert parent of partdir - into sys.path. If the parent directories dont have the part - an EnvironmentError is raised.""" - - import sys, os - try: - head = this_dir = os.path.abspath(os.path.dirname(__file__)) - except NameError: - head = this_dir = os.path.abspath(os.path.dirname(sys.argv[0])) - - while head: - partdir = head - head, tail = os.path.split(head) - if tail == part: - sys.path = [p for p in sys.path if not p.startswith(head)] - if head not in sys.path: - sys.path.insert(0, head) - return partdir, this_dir - - raise EnvironmentError, "'%s' missing in '%r'" % (pathpart,this_path) - -def __clone(): - """ clone master version of autopath.py into all subdirs """ - from os.path import join, walk - if not this_dir.endswith(join('pypy','tool')): - raise EnvironmentError("can only clone master version " - "'%s'" % join(pypydir, 'tool',_myname)) - - - def sync_walker(arg, dirname, fnames): - if _myname in fnames: - fn = join(dirname, _myname) - f = open(fn, 'rwb+') - try: - if f.read() == arg: - print "checkok", fn - else: - print "syncing", fn - f = open(fn, 'w') - f.write(arg) - finally: - f.close() - s = open(join(pypydir, 'tool', _myname), 'rb').read() - walk(pypydir, sync_walker, s) - -_myname = 'autopath.py' - -# set guaranteed attributes - -pypydir, this_dir = __dirinfo('pypy') - -if __name__ == '__main__': - __clone() +""" +self cloning, automatic path configuration + +copy this into any subdirectory of pypy from which scripts need +to be run, typically all of the test subdirs. +The idea is that any such script simply issues + + import autopath + +and this will make sure that the parent directory containing "pypy" +is in sys.path. + +If you modify the master "autopath.py" version (in pypy/tool/autopath.py) +you can directly run it which will copy itself on all autopath.py files +it finds under the pypy root directory. + +This module always provides these attributes: + + pypydir pypy root directory path + this_dir directory where this autopath.py resides + +""" + + +def __dirinfo(part): + """ return (partdir, this_dir) and insert parent of partdir + into sys.path. If the parent directories dont have the part + an EnvironmentError is raised.""" + + import sys, os + try: + head = this_dir = os.path.abspath(os.path.dirname(__file__)) + except NameError: + head = this_dir = os.path.abspath(os.path.dirname(sys.argv[0])) + + while head: + partdir = head + head, tail = os.path.split(head) + if tail == part: + sys.path = [p for p in sys.path if not p.startswith(head)] + if head not in sys.path: + sys.path.insert(0, head) + return partdir, this_dir + + raise EnvironmentError, "'%s' missing in '%r'" % (pathpart,this_path) + +def __clone(): + """ clone master version of autopath.py into all subdirs """ + from os.path import join, walk + if not this_dir.endswith(join('pypy','tool')): + raise EnvironmentError("can only clone master version " + "'%s'" % join(pypydir, 'tool',_myname)) + + + def sync_walker(arg, dirname, fnames): + if _myname in fnames: + fn = join(dirname, _myname) + f = open(fn, 'rwb+') + try: + if f.read() == arg: + print "checkok", fn + else: + print "syncing", fn + f = open(fn, 'w') + f.write(arg) + finally: + f.close() + s = open(join(pypydir, 'tool', _myname), 'rb').read() + walk(pypydir, sync_walker, s) + +_myname = 'autopath.py' + +# set guaranteed attributes + +pypydir, this_dir = __dirinfo('pypy') + +if __name__ == '__main__': + __clone() Modified: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Mon Sep 29 20:47:25 2003 @@ -1,32 +1,35 @@ import autopath from pypy.tool import test +from pypy.objspace.flow.wrapper import * +from pypy.translator.controlflow import * + class TestFlowOjSpace(test.TestCase): def setUp(self): self.space = test.objspace('flow') - def codetest(sefl, source, functionname, args_w): + def codetest(self, source, functionname, args_w): glob = {} exec source in glob func = glob[functionname] w_args = self.space.newtuple(args_w) - w_func = self.space.wrap(func) w_kwds = self.space.newdict([]) - return self.space.build_flow(w_func, w_args, w_kwds) + return self.space.build_flow(func, w_args, w_kwds) def test_nothing(self): x = self.codetest("def f():\n" - " pass\n") - self.assertEquals(len(x), 1) + " pass\n", + 'f', []) + self.assertEquals(x.functionname, 'f') + self.assertEquals(x.startblock.branch.__class__, EndBranch) def test_ifthenelse(self): x = self.codetest("def f(i, j):\n" " if i < 0:\n" " i = j\n" " return g(i) + 1\n", - 'f', [W_Anything()]) - self.assertEquals(x.graph_string(), - "") + 'f', [W_Variable()]) + if __name__ == '__main__': test.main() Added: pypy/trunk/src/pypy/objspace/flow/wrapper.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/objspace/flow/wrapper.py Mon Sep 29 20:47:25 2003 @@ -0,0 +1,76 @@ +# ______________________________________________________________________ +"""Wrapper objects for the control flow analysis object space.""" +# ______________________________________________________________________ + +# This is kinda a hack, but at the same time, I don't see why this was defined +# in the object space module in the annotation object space. + +class UnwrapException(Exception): + pass + +# ______________________________________________________________________ + +class W_Object(object): + """Abstract base class. do not instantiate.""" + + force = None # See cloningcontext.py + + def __new__(cls, *args, **kwd): + assert cls is not W_Object + return object.__new__(cls) + + def __init__(self): + pass + + def __repr__(self): + s = self.argsrepr() + if len(s) > 100: + s = s[:25] + "..." + s[-25:] + return "%s(%s)" % (self.__class__.__name__, s) + + def argsrepr(self): + return "" + + def unwrap(self): + # XXX Somehow importing this at module level doesn't work + raise UnwrapException(self) + + def __eq__(self, other): + return type(other) is type(self) + + def __ne__(self, other): + return not self.__eq__(other) + +# ______________________________________________________________________ + +class W_Variable(W_Object): + pass + +# ______________________________________________________________________ + +class W_Constant(W_Object): + """A specific constant value.""" + + def __init__(self, value): + self.value = value + + def argsrepr(self): + return repr(self.value) + + def unwrap(self): + return self.value + + def __eq__(self, other): + return type(other) is type(self) and self.value == other.value + + def __len__(self): + return len(self.value) + + def __getitem__(self, key): + return self.value[key] + + def __setitem__(self, key, value): + self.value[key] = value + +# ______________________________________________________________________ +# End of wrapper.py From arigo at codespeak.net Tue Sep 30 10:33:21 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 10:33:21 +0200 (MEST) Subject: [pypy-svn] rev 1457 - in pypy/trunk/src/pypy/translator: . test Message-ID: <20030930083321.4EE8C5AB2F@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 10:33:20 2003 New Revision: 1457 Modified: pypy/trunk/src/pypy/translator/ (props changed) pypy/trunk/src/pypy/translator/__init__.py (props changed) pypy/trunk/src/pypy/translator/controlflow.py (props changed) pypy/trunk/src/pypy/translator/genpyrex.py (props changed) pypy/trunk/src/pypy/translator/test/ (props changed) pypy/trunk/src/pypy/translator/test/__init__.py (props changed) pypy/trunk/src/pypy/translator/test/autopath.py (props changed) pypy/trunk/src/pypy/translator/test/test_sourcegen.py (props changed) Log: fixeol From hpk at codespeak.net Tue Sep 30 10:55:31 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 10:55:31 +0200 (MEST) Subject: [pypy-svn] rev 1458 - in pypy/trunk/src: . pypy/translator/test Message-ID: <20030930085531.46E915AB2F@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 10:55:30 2003 New Revision: 1458 Added: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Modified: pypy/trunk/src/ (props changed) pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: - added to the 'src' directory the following externals 'pyrex' (which points to the hacked version of Pyrex with the "cinline" statement 'vpath' (which is used to handle creating temporary files etc.) - so in order to run the translator you need to checkout the 'src' directory of pypy which contains the above externals. - now the translator actually has two unit-tests that check if the produced C-module produces the expected results when we call the now-C function. Added: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Tue Sep 30 10:55:30 2003 @@ -0,0 +1,58 @@ + +import autopath +from pypy.tool import test + +from vpath.local import Path, mkdtemp +import os, sys + +def make_module_from_pyxstring(string, num=[0]): + tmpdir = mkdtemp() + n = num[0] = num[0]+1 + pyxfile = tmpdir.join('test%d.pyx' %n) + pyxfile.write(string) + make_c_from_pyxfile(pyxfile) + module = make_module_from_c(pyxfile) + #print "made module", module + return module + +def make_module_from_c(pyxfile): + from distutils.core import setup + from distutils.extension import Extension + from pyrex.Distutils import build_ext + + dirpath = pyxfile.dirname() + lastdir = os.curdir + os.chdir(str(dirpath)) + try: + modname = pyxfile.purebasename() + #print "modname", modname + setup( + name = "testmodules", + ext_modules=[ + Extension(modname, [str(pyxfile)]) + ], + cmdclass = {'build_ext': build_ext}, + script_name = 'setup.py', + script_args = ['build_ext', '--inplace', '-q', '--quiet'] + ) + # XXX not a nice way to import a module + sys.path.insert(0, dirpath) + exec "import %(modname)s as testmodule" % locals() + sys.path.pop(0) + finally: + os.chdir(lastdir) + return testmodule + +def make_c_from_pyxfile(pyxfile): + from pyrex.Compiler.Main import CompilationOptions, Context, PyrexError + try: + options = CompilationOptions(show_version = 0, + use_listing_file = 0, + output_file = None) + context = Context(options.include_path) + result = context.compile(str(pyxfile), options, c_only = 1) + if result.num_errors > 0: + raise ValueError, "failure %s" % result + except PyrexError, e: + print >>sys.stderr, e + cfile = pyxfile.newsuffix('.c') Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/test_sourcegen.py (original) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Tue Sep 30 10:55:30 2003 @@ -5,6 +5,7 @@ from pypy.translator.genpyrex import GenPyrex from pypy.translator.controlflow import * +from buildpyxmodule import make_module_from_pyxstring class TestCase(test.IntTestCase): def test_simple_func(self): @@ -22,11 +23,8 @@ endbranch) fun = FunctionGraph(block, "f") result = GenPyrex(fun).emitcode() - self.assertEquals(result, """ -def f(x): - result = x + 1 - return result -""") + mod = make_module_from_pyxstring(result) + self.assertEquals(mod.f(1), 2) def test_if(self): """ @@ -52,14 +50,9 @@ conditionalbranch) fun = FunctionGraph(startblock, "f") result = GenPyrex(fun).emitcode() - self.assertEquals(result, """ -def f(i, j): - conditionres = i < 0 - if conditionres: cinline "goto label1;" - return i - cinline "label1:" - return j -""") + mod = make_module_from_pyxstring(result) + self.assertEquals(mod.f(-1, 42), 42) + self.assertEquals(mod.f(3, 5), 3) if __name__ == '__main__': test.main() From jriehl at codespeak.net Tue Sep 30 11:02:24 2003 From: jriehl at codespeak.net (jriehl at codespeak.net) Date: Tue, 30 Sep 2003 11:02:24 +0200 (MEST) Subject: [pypy-svn] rev 1459 - pypy/trunk/src/pypy/objspace/flow Message-ID: <20030930090224.EF8ED5AB2F@thoth.codespeak.net> Author: jriehl Date: Tue Sep 30 11:02:24 2003 New Revision: 1459 Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py Log: Added is_true() method to flow object space. Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/objspace.py Tue Sep 30 11:02:24 2003 @@ -1,10 +1,13 @@ +# ______________________________________________________________________ import operator import pypy from pypy.interpreter.baseobjspace import ObjSpace from pypy.interpreter.pycode import PyCode from pypy.objspace.flow.wrapper import * from pypy.translator.controlflow import * +from pypy.objspace.flow.cloningcontext import IndeterminateCondition +# ______________________________________________________________________ class FlowObjSpace(ObjSpace): def initialize(self): self.w_builtins = W_Variable() @@ -58,6 +61,17 @@ del self._crnt_ops return g + # ____________________________________________________________ + def is_true(self, w_obj): + try: + obj = self.unwrap(w_obj) + except UnwrapException: + pass + else: + return bool(obj) + context = self.getexecutioncontext() + return context.guessbool() + # ______________________________________________________________________ def make_op(name, symbol, arity, specialnames): From hpk at codespeak.net Tue Sep 30 11:27:39 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 11:27:39 +0200 (MEST) Subject: [pypy-svn] rev 1460 - in pypy/trunk/src: . pypy/translator/test Message-ID: <20030930092739.B3CD35AB2F@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 11:27:39 2003 New Revision: 1460 Modified: pypy/trunk/src/ (props changed) pypy/trunk/src/pypy/translator/test/buildpyxmodule.py (contents, props changed) pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: - fixed various issues - now Pyrex is named Pyrex (instead of pyrex) - fixed python-2.3 issue - there is still too verbose output Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/buildpyxmodule.py (original) +++ pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Tue Sep 30 11:27:39 2003 @@ -10,6 +10,7 @@ n = num[0] = num[0]+1 pyxfile = tmpdir.join('test%d.pyx' %n) pyxfile.write(string) + print "made pyxfile", pyxfile make_c_from_pyxfile(pyxfile) module = make_module_from_c(pyxfile) #print "made module", module @@ -18,14 +19,14 @@ def make_module_from_c(pyxfile): from distutils.core import setup from distutils.extension import Extension - from pyrex.Distutils import build_ext + from Pyrex.Distutils import build_ext dirpath = pyxfile.dirname() lastdir = os.curdir os.chdir(str(dirpath)) try: modname = pyxfile.purebasename() - #print "modname", modname + print "modname", modname setup( name = "testmodules", ext_modules=[ @@ -33,18 +34,21 @@ ], cmdclass = {'build_ext': build_ext}, script_name = 'setup.py', - script_args = ['build_ext', '--inplace', '-q', '--quiet'] + script_args = ['build_ext', '--inplace', '-q'] # , '--quiet'] ) # XXX not a nice way to import a module - sys.path.insert(0, dirpath) + print "inserting path to sys.path", dirpath + sys.path.insert(0, '.') + print "import %(modname)s as testmodule" % locals() exec "import %(modname)s as testmodule" % locals() sys.path.pop(0) finally: os.chdir(lastdir) + #dirpath.rmtree() return testmodule def make_c_from_pyxfile(pyxfile): - from pyrex.Compiler.Main import CompilationOptions, Context, PyrexError + from Pyrex.Compiler.Main import CompilationOptions, Context, PyrexError try: options = CompilationOptions(show_version = 0, use_listing_file = 0, Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/test_sourcegen.py (original) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Tue Sep 30 11:27:39 2003 @@ -5,7 +5,7 @@ from pypy.translator.genpyrex import GenPyrex from pypy.translator.controlflow import * -from buildpyxmodule import make_module_from_pyxstring +from pypy.translator.test.buildpyxmodule import make_module_from_pyxstring class TestCase(test.IntTestCase): def test_simple_func(self): From hpk at codespeak.net Tue Sep 30 12:09:00 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 12:09:00 +0200 (MEST) Subject: [pypy-svn] rev 1461 - pypy/trunk/src/pypy/translator/test Message-ID: <20030930100900.D62AA5AB2F@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 12:09:00 2003 New Revision: 1461 Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Log: - some tries to get the output less verbose but it doesn'T work on python-2.3/distutils which seems to always print out verbose information. Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/buildpyxmodule.py (original) +++ pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Tue Sep 30 12:09:00 2003 @@ -5,12 +5,14 @@ from vpath.local import Path, mkdtemp import os, sys +debug = 0 + def make_module_from_pyxstring(string, num=[0]): tmpdir = mkdtemp() n = num[0] = num[0]+1 pyxfile = tmpdir.join('test%d.pyx' %n) pyxfile.write(string) - print "made pyxfile", pyxfile + if debug: print "made pyxfile", pyxfile make_c_from_pyxfile(pyxfile) module = make_module_from_c(pyxfile) #print "made module", module @@ -21,12 +23,19 @@ from distutils.extension import Extension from Pyrex.Distutils import build_ext + #try: + # from distutils.log import set_threshold + # set_threshold(10000) + #except ImportError: + # print "ERROR IMPORTING" + # pass + dirpath = pyxfile.dirname() lastdir = os.curdir os.chdir(str(dirpath)) try: modname = pyxfile.purebasename() - print "modname", modname + if debug: print "modname", modname setup( name = "testmodules", ext_modules=[ @@ -34,12 +43,12 @@ ], cmdclass = {'build_ext': build_ext}, script_name = 'setup.py', - script_args = ['build_ext', '--inplace', '-q'] # , '--quiet'] + script_args = ['build_ext', '--inplace', '-q', '--quiet'] ) # XXX not a nice way to import a module - print "inserting path to sys.path", dirpath + if debug: print "inserting path to sys.path", dirpath sys.path.insert(0, '.') - print "import %(modname)s as testmodule" % locals() + if debug: print "import %(modname)s as testmodule" % locals() exec "import %(modname)s as testmodule" % locals() sys.path.pop(0) finally: From hpk at codespeak.net Tue Sep 30 12:12:33 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 12:12:33 +0200 (MEST) Subject: [pypy-svn] rev 1462 - pypy/trunk/src/pypy/translator/test Message-ID: <20030930101233.1A01C5AB2F@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 12:12:32 2003 New Revision: 1462 Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Log: - reversing the options for 'distutils.core.setup' helps to to decrease verbosity Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/buildpyxmodule.py (original) +++ pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Tue Sep 30 12:12:32 2003 @@ -43,7 +43,7 @@ ], cmdclass = {'build_ext': build_ext}, script_name = 'setup.py', - script_args = ['build_ext', '--inplace', '-q', '--quiet'] + script_args = ['-q', 'build_ext', '--inplace'] ) # XXX not a nice way to import a module if debug: print "inserting path to sys.path", dirpath From hpk at codespeak.net Tue Sep 30 13:14:20 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 13:14:20 +0200 (MEST) Subject: [pypy-svn] rev 1463 - in pypy/trunk/src/pypy/translator: . test Message-ID: <20030930111420.22A515AB2F@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 13:14:19 2003 New Revision: 1463 Modified: pypy/trunk/src/pypy/translator/controlflow.py pypy/trunk/src/pypy/translator/genpyrex.py pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: - added a simple while loop which translates correctly to a C-extension module. Modified: pypy/trunk/src/pypy/translator/controlflow.py ============================================================================== --- pypy/trunk/src/pypy/translator/controlflow.py (original) +++ pypy/trunk/src/pypy/translator/controlflow.py Tue Sep 30 13:14:19 2003 @@ -24,12 +24,17 @@ self.result = result # class Branch: - def __init__(self, args, target): + def __init__(self, args=None, target=None): + self.set(args, target) + def set(self, args, target): self.args = args # list of variables self.target = target # basic block instance class ConditionalBranch: - def __init__(self, condition, ifbranch, elsebranch): + def __init__(self, condition=None, ifbranch=None, elsebranch=None): + self.set(condition, ifbranch, elsebranch) + + def set(self, condition, ifbranch, elsebranch): self.condition = condition self.ifbranch = ifbranch self.elsebranch = elsebranch Modified: pypy/trunk/src/pypy/translator/genpyrex.py ============================================================================== --- pypy/trunk/src/pypy/translator/genpyrex.py (original) +++ pypy/trunk/src/pypy/translator/genpyrex.py Tue Sep 30 13:14:19 2003 @@ -74,7 +74,7 @@ sourceargs = [_str(arg) for arg in branch.args] targetargs = [arg.pseudoname for arg in block.input_args] assert(len(sourceargs) == len(targetargs)) - if sourceargs: + if sourceargs and sourceargs != targetargs: self.putline("%s = %s" % (", ".join(targetargs), ", ".join(sourceargs))) self.createCodeFromBasicBlock(block) Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/test_sourcegen.py (original) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Tue Sep 30 13:14:19 2003 @@ -54,5 +54,40 @@ self.assertEquals(mod.f(-1, 42), 42) self.assertEquals(mod.f(3, 5), 3) + def test_while(self): + """ + one test source: + def f(i): + while i > 0: + i = i - 1 + return i + """ + i = Variable("i") + conditionres = Variable("conditionres") + conditionop = SpaceOperation("gt", [i, Constant(0)], conditionres) + decop = SpaceOperation("add", [i, Constant(-1)], i) + + conditionbranch = ConditionalBranch() + headerbranch = Branch() + whileblock = BasicBlock([i], [i], [decop], headerbranch) + whilebranch = Branch([i], whileblock) + endbranch = EndBranch(i) + conditionbranch.set(conditionres, whilebranch, endbranch) + + headerblock = BasicBlock([i], [i, conditionres], + [conditionop], conditionbranch) + + headerbranch.set([i], headerblock) + + startblock = BasicBlock([i], [i], + [], headerbranch) + + + fun = FunctionGraph(startblock, "f") + result = GenPyrex(fun).emitcode() + mod = make_module_from_pyxstring(result) + self.assertEquals(mod.f(42), 0) + self.assertEquals(mod.f(-3), -3) + if __name__ == '__main__': test.main() From hpk at codespeak.net Tue Sep 30 13:41:03 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 13:41:03 +0200 (MEST) Subject: [pypy-svn] rev 1464 - pypy/trunk/src/pypy/translator/test Message-ID: <20030930114103.E994B5AB2F@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 13:41:03 2003 New Revision: 1464 Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: - added a more complex while-loop with some summing (all of these tests with tomek) Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/test_sourcegen.py (original) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Tue Sep 30 13:41:03 2003 @@ -82,12 +82,52 @@ startblock = BasicBlock([i], [i], [], headerbranch) - fun = FunctionGraph(startblock, "f") result = GenPyrex(fun).emitcode() mod = make_module_from_pyxstring(result) self.assertEquals(mod.f(42), 0) self.assertEquals(mod.f(-3), -3) + def test_while_sum(self): + """ + one test source: + def f(i): + sum = 0 + while i > 0: + sum = sum + i + i = i - 1 + return sum + """ + i = Variable("i") + sum = Variable("sum") + + conditionres = Variable("conditionres") + conditionop = SpaceOperation("gt", [i, Constant(0)], conditionres) + decop = SpaceOperation("add", [i, Constant(-1)], i) + addop = SpaceOperation("add", [i, sum], sum) + + conditionbranch = ConditionalBranch() + headerbranch = Branch() + headerbranch2 = Branch() + whileblock = BasicBlock([i, sum], [i, sum], [addop, decop], headerbranch2) + whilebranch = Branch([i, sum], whileblock) + + endbranch = EndBranch(sum) + conditionbranch.set(conditionres, whilebranch, endbranch) + + headerblock = BasicBlock([i, sum], [i, conditionres], + [conditionop], conditionbranch) + + headerbranch.set([i, Constant(0)], headerblock) + headerbranch2.set([i, sum], headerblock) + startblock = BasicBlock([i], [i, sum], + [], headerbranch) + + fun = FunctionGraph(startblock, "f") + result = GenPyrex(fun).emitcode() + mod = make_module_from_pyxstring(result) + self.assertEquals(mod.f(3), 6) + self.assertEquals(mod.f(-3), 0) + if __name__ == '__main__': test.main() From stephan at codespeak.net Tue Sep 30 15:24:17 2003 From: stephan at codespeak.net (stephan at codespeak.net) Date: Tue, 30 Sep 2003 15:24:17 +0200 (MEST) Subject: [pypy-svn] rev 1465 - pypy/trunk/doc/funding/tools Message-ID: <20030930132417.66D8E5AB2F@thoth.codespeak.net> Author: stephan Date: Tue Sep 30 15:24:16 2003 New Revision: 1465 Added: pypy/trunk/doc/funding/tools/ pypy/trunk/doc/funding/tools/format.py Log: added basic tool to replace table style Added: pypy/trunk/doc/funding/tools/format.py ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/tools/format.py Tue Sep 30 15:24:16 2003 @@ -0,0 +1,167 @@ +import re +import sys + +START = re.compile(r'\s*<[^/][^<]*?>',re.DOTALL) +STOP = re.compile(r'\s*',re.DOTALL) +EMPTY = re.compile(r'\s*<[^/][^<]*?/>',re.DOTALL) +OTHER = re.compile(r'\s*<[!\?][^<]*>') +ANY = re.compile(r'[^<]*',re.DOTALL) +tab = ' ' + +tabprop = """""" + +class parse(object): + def __init__(self,text): + self.text = text + + def start(self,txt):pass + def stop(self,txt):pass + def empty(self,txt):pass + def other(self,txt):pass + def any(self,txt):pass + def parse(self): + content = self.text[:] + while content: + m = OTHER.match(content) + if m is not None: + #print 'OTHER:',m.group() + content = content[m.span()[1]:] + self.other(m.group()) + continue + m = EMPTY.match(content) + if m is not None: + #print 'EMPTY:',m.group() + content = content[m.span()[1]:] + self.empty(m.group()) + continue + m = START.match(content) + if m is not None: + #print 'START:',m.group() + content = content[m.span()[1]:] + self.start(m.group()) + continue + m = STOP.match(content) + if m is not None: + #print 'STOP:',m.group() + content = content[m.span()[1]:] + self.stop(m.group()) + continue + m = ANY.match(content) + if m is not None: + #print 'ANY:',m.group() + txt = m.group() + txt.strip() + self.any(txt) + content = content[m.span()[1]:] + continue + +class format(parse): + def __init__(self,*argl,**argd): + super(format,self).__init__(*argl,**argd) + self.level = 0 + self.outbuf = [] + + def __str__(self): + self.parse() + return '\n'.join(self.outbuf) + + def start(self,txt): + self.outbuf.append(tab*self.level+txt) + self.level += 1 + + def stop(self,txt): + self.level -= 1 + self.outbuf.append(tab*self.level+txt) + + def any(self,txt): + self.outbuf.append(tab*self.level+txt) + + def empty(self,txt): + self.outbuf.append(tab*self.level+txt) + + def other(self,txt): + self.outbuf.append(tab*self.level+txt) + +class findTableName(parse): + def __init__(self,*argl,**argd): + super(findTableName,self).__init__(*argl,**argd) + self.tabname = {} + self.currTable = '' + + def any(self,txt): + if self.currTable and txt.find('Workpackage number') != -1: + self.tabname[self.currTable] = True + + def stop(self,txt): + if txt.startswith('> sys.stderr,self.tabnames + self.inStyle = True + + def __str__(self): + self.parse() + return ''.join(self.outbuf) + + def start(self,txt): + txt = txt.strip() + if txt.startswith('> sys.stderr,'found table-cell' + self.inStyle = True + self.outbuf.append(txt) + + def stop(self,txt): + txt = txt.strip() + if self.inStyle: + self.inStyle = False + self.outbuf.append(txt) + + def other(self,txt): + txt = txt.strip() + self.outbuf.append(txt) + + def empty(self,txt): + txt = txt.strip() + if self.inStyle and txt.find('style:properties') != -1: + self.outbuf.append(tabprop) + self.inStyle = False + else: + self.outbuf.append(txt) + + def any(self,txt): + txt = txt.strip() + if txt: + self.outbuf.append(txt) + +def changeStyle(txt): + f = format(str(replaceMargin(txt))) + return str(f) + +if __name__ == '__main__': + import sys + fn = sys.argv[1] + content = open(fn).read() + f = format(str(replaceMargin(content))) + print f From dinu at codespeak.net Tue Sep 30 15:26:35 2003 From: dinu at codespeak.net (dinu at codespeak.net) Date: Tue, 30 Sep 2003 15:26:35 +0200 (MEST) Subject: [pypy-svn] rev 1466 - pypy/trunk/doc/funding/tools Message-ID: <20030930132635.93E335AB2F@thoth.codespeak.net> Author: dinu Date: Tue Sep 30 15:26:34 2003 New Revision: 1466 Added: pypy/trunk/doc/funding/tools/zipmod.py Log: initial checkin Added: pypy/trunk/doc/funding/tools/zipmod.py ============================================================================== --- (empty file) +++ pypy/trunk/doc/funding/tools/zipmod.py Tue Sep 30 15:26:34 2003 @@ -0,0 +1,72 @@ +#! /usr/bin/env python + +import os, sys, zipfile + +# s = """..... :DELETE:BEGIN:DELETE:END ----------""" * 2 + +def filter(s): + "Remove string all parts of a string within certain tags." + + text0 = '' + text1 = '' + tag0 = ":DELETE:BEGIN" + tag1 = ":DELETE:END" + + list = [] + while 1: + p = s.find(tag0) + if p == -1: break + s0 = s[:p] + t0 = s[p-len(text0):p] + t1 = s[p+len(tag0):p+len(tag0)+len(text1)] + list.append(s[:p-len(text0)]) + s = s[p+len(tag0)+len(text1):] + p = s.find(tag1) + if p == -1: break + s0 = s[:p] + t0 = s[p-len(text0):p] + t1 = s[p+len(tag1):p+len(tag1)+len(text1)] + s = s[p+len(tag1)+len(text1):] + list.append(s) + + return ''.join(list) + + +def modifyOpenOfficeContent(docPath, filename, func): + "Read the content XML file from a zipped Open Office document." + + # read ZIP file, build {filename:open(filename).read(), ...} + zip = zipfile.ZipFile(docPath, 'r') + filenames = zip.namelist() + filemap = {} + for fn in filenames: + filemap[fn] = zip.read(fn) + zip.close() + + # modify specific filename by applying func to it + content = filemap[filename] + modContent = func(content) + filemap[filename] = modContent + + # build temp. directory structure as in ZIP file + # (zipfile has no extract or modify method!) + tmp = 'tmp' + try: os.mkdir(tmp) + except OSError: pass + for fn in filenames: + dirName = os.path.join(tmp, os.path.dirname(fn)) + try: os.mkdir(dirName) + except OSError: pass + destPath = os.path.join(tmp, fn) + open(destPath, 'w').write(filemap[fn]) + + # rebuild a copy of the original ZIP file from tmp. dir. + zip = zipfile.ZipFile(docPath, 'w', zipfile.ZIP_DEFLATED) + for fn in filenames: + destPath = os.path.join(tmp, fn) + zip.write(destPath, fn) + + +if __name__ == '__main__': + docPath = sys.argv[1] + modifyOpenOfficeContent(docPath, 'content.xml', filter) From stephan at codespeak.net Tue Sep 30 15:31:44 2003 From: stephan at codespeak.net (stephan at codespeak.net) Date: Tue, 30 Sep 2003 15:31:44 +0200 (MEST) Subject: [pypy-svn] rev 1467 - pypy/trunk/doc/funding/tools Message-ID: <20030930133144.DEFE45AB2F@thoth.codespeak.net> Author: stephan Date: Tue Sep 30 15:31:44 2003 New Revision: 1467 Modified: pypy/trunk/doc/funding/tools/zipmod.py Log: added support for filter stuff Modified: pypy/trunk/doc/funding/tools/zipmod.py ============================================================================== --- pypy/trunk/doc/funding/tools/zipmod.py (original) +++ pypy/trunk/doc/funding/tools/zipmod.py Tue Sep 30 15:31:44 2003 @@ -1,6 +1,7 @@ #! /usr/bin/env python import os, sys, zipfile +from format import changeStyle # s = """..... :DELETE:BEGIN:DELETE:END ----------""" * 2 @@ -67,6 +68,12 @@ zip.write(destPath, fn) +def filterAll(txt): + txt = changeStyle(txt) + txt = filter(txt) + + return txt + if __name__ == '__main__': docPath = sys.argv[1] - modifyOpenOfficeContent(docPath, 'content.xml', filter) + modifyOpenOfficeContent(docPath, 'content.xml', filterAll) From stephan at codespeak.net Tue Sep 30 15:35:09 2003 From: stephan at codespeak.net (stephan at codespeak.net) Date: Tue, 30 Sep 2003 15:35:09 +0200 (MEST) Subject: [pypy-svn] rev 1468 - pypy/trunk/doc/funding/tools Message-ID: <20030930133509.D97645AB2F@thoth.codespeak.net> Author: stephan Date: Tue Sep 30 15:35:09 2003 New Revision: 1468 Modified: pypy/trunk/doc/funding/tools/format.py Log: removed debug prints from format.py Modified: pypy/trunk/doc/funding/tools/format.py ============================================================================== --- pypy/trunk/doc/funding/tools/format.py (original) +++ pypy/trunk/doc/funding/tools/format.py Tue Sep 30 15:35:09 2003 @@ -116,7 +116,7 @@ def __init__(self,*argl,**argd): super(replaceMargin,self).__init__(*argl,**argd) self.tabnames = findTableName(self.text)() - print >> sys.stderr,self.tabnames + #print >> sys.stderr,self.tabnames self.inStyle = True def __str__(self): @@ -128,7 +128,7 @@ if txt.startswith('> sys.stderr,'found table-cell' + #print >> sys.stderr,'found table-cell' self.inStyle = True self.outbuf.append(txt) From hpk at codespeak.net Tue Sep 30 15:40:47 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 15:40:47 +0200 (MEST) Subject: [pypy-svn] rev 1469 - pypy/trunk/src/pypy/translator/test Message-ID: <20030930134047.8B1655AB2F@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 15:40:47 2003 New Revision: 1469 Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Log: - fixed the translation tests to remove temporary directories Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/buildpyxmodule.py (original) +++ pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Tue Sep 30 15:40:47 2003 @@ -31,7 +31,7 @@ # pass dirpath = pyxfile.dirname() - lastdir = os.curdir + lastdir = Path() os.chdir(str(dirpath)) try: modname = pyxfile.purebasename() @@ -52,8 +52,8 @@ exec "import %(modname)s as testmodule" % locals() sys.path.pop(0) finally: - os.chdir(lastdir) - #dirpath.rmtree() + os.chdir(str(lastdir)) + dirpath.rmtree() return testmodule def make_c_from_pyxfile(pyxfile): From hpk at codespeak.net Tue Sep 30 17:20:03 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 17:20:03 +0200 (MEST) Subject: [pypy-svn] rev 1470 - pypy/trunk/src/pypy/translator/test Message-ID: <20030930152003.528545A63A@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 17:20:02 2003 New Revision: 1470 Added: pypy/trunk/src/pypy/translator/test/make_dot.py Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py pypy/trunk/src/pypy/translator/test/test_sourcegen.py Log: beginnings of a tool to visualize the control-flow Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/buildpyxmodule.py (original) +++ pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Tue Sep 30 17:20:02 2003 @@ -53,7 +53,8 @@ sys.path.pop(0) finally: os.chdir(str(lastdir)) - dirpath.rmtree() + if not debug: + dirpath.rmtree() return testmodule def make_c_from_pyxfile(pyxfile): Added: pypy/trunk/src/pypy/translator/test/make_dot.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/translator/test/make_dot.py Tue Sep 30 17:20:02 2003 @@ -0,0 +1,120 @@ +# + +""" +""" + +import autopath +from pypy.translator.controlflow import * +import os + +class Source: + def __init__(self): + self.lines = [] + + def getsource(self): + content = "\n".join(self.lines) + + return """digraph test { + node [fontname=Times]; + edge [fontname=Times]; + %(content)s + } + """ % locals() + + def putnode(self, name, node): + self.lines.append('%(name)s [shape=box, label=%(name)s];' % locals()) + + def putedge(self, name, othername): + self.lines.append('%(name)s -> %(othername)s;' % locals()) + +#class Node: +# def __init__(self, **kwargs): +# self.__dict__.update(kwargs) +# +#class NodeFunctionGraph(Node): +# def dotsource(self): + +class DotGen: + def __init__(self): + self.graph = {} # node1 : [node1,node2,node3] } + self.names = {} # obj : string + self.blocknum = 0 + self.branchnum = 0 + + def get_source(self, obj): + self.dispatch(obj) + source = Source() + for obj,edges in self.graph.items(): + objname = self.names[obj] + source.putnode(objname, obj) + for edge in edges: + source.putedge(objname, edge) + return source.getsource() + + def dispatch(self, obj): + try: + return self.names[obj] + except KeyError: + pass + typename = obj.__class__.__name__ + method = getattr(self, "gen_%s" % typename, None) + if method is not None: + return method(obj) + + raise ValueError, "unknown type for %r" % typename + + def append_edge(self, obj, otherobj): + points_to = self.graph.setdefault(obj, []) + points_to.append(otherobj) + + def gen_FunctionGraph(self, obj): + name = obj.functionname + self.names[obj] = name + self.append_edge(obj, self.dispatch(obj.startblock)) + return name + + def gen_BasicBlock(self, obj): + name = "block%s" % self.blocknum + self.blocknum += 1 + self.names[obj] = name + self.append_edge(obj, self.dispatch(obj.branch)) + return name + + def gen_Branch(self, obj): + name = "branch%s" % self.branchnum + self.branchnum += 1 + self.names[obj] = name + self.append_edge(obj, self.dispatch(obj.target)) + return name + + def gen_ConditionalBranch(self, obj): + name = "Condbranch%s" % self.branchnum + self.branchnum += 1 + self.names[obj] = name + self.append_edge(obj, self.dispatch(obj.ifbranch)) + self.append_edge(obj, self.dispatch(obj.elsebranch)) + return name + + def gen_EndBranch(self, obj): + name = "endbranch%s" % self.branchnum + self.branchnum += 1 + self.names[obj] = name + self.graph[obj] = [] + return name + #self.append_edge(self.dispatch(obj.target)) + +def make_ps(fun): + dotgen = DotGen() + + from vpath.local import Path + from vpath.adapter.process import exec_cmd + dest = Path('/tmp/testgraph.dot') + dest.write(dotgen.get_source(fun)) + psdest = dest.newsuffix('.ps') + out = exec_cmd('dot -Tps %s' % str(dest)) + psdest.write(out) + print "wrote", psdest + +#if __name__ == '__main__': + + Modified: pypy/trunk/src/pypy/translator/test/test_sourcegen.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/test_sourcegen.py (original) +++ pypy/trunk/src/pypy/translator/test/test_sourcegen.py Tue Sep 30 17:20:02 2003 @@ -6,6 +6,7 @@ from pypy.translator.controlflow import * from pypy.translator.test.buildpyxmodule import make_module_from_pyxstring +#from pypy.translator.test.make_dot import make_ps class TestCase(test.IntTestCase): def test_simple_func(self): @@ -83,6 +84,8 @@ [], headerbranch) fun = FunctionGraph(startblock, "f") + #make_ps(fun) + result = GenPyrex(fun).emitcode() mod = make_module_from_pyxstring(result) self.assertEquals(mod.f(42), 0) From arigo at codespeak.net Tue Sep 30 17:21:30 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 17:21:30 +0200 (MEST) Subject: [pypy-svn] rev 1471 - in pypy/trunk/src/pypy: interpreter objspace/flowobjspace/flow/test translator Message-ID: <20030930152130.8E6925A63A@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 17:21:29 2003 New Revision: 1471 Added: pypy/trunk/src/pypy/objspace/flow/flowcontext.py (contents, props changed) - copied, changed from rev 1456, pypy/trunk/src/pypy/objspace/ann/cloningcontext.py Modified: pypy/trunk/src/pypy/interpreter/nestedscope.py pypy/trunk/src/pypy/interpreter/pycode.py pypy/trunk/src/pypy/interpreter/pyframe.py pypy/trunk/src/pypy/objspace/flow/objspace.py pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py pypy/trunk/src/pypy/objspace/flow/wrapper.py pypy/trunk/src/pypy/translator/controlflow.py Log: Control flow object space starts to work! Modified: pypy/trunk/src/pypy/interpreter/nestedscope.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/nestedscope.py (original) +++ pypy/trunk/src/pypy/interpreter/nestedscope.py Tue Sep 30 17:21:29 2003 @@ -65,6 +65,10 @@ "an unexpected number of free variables") self.cells = [Cell() for i in range(ncellvars)] + closure + def getclosure(self): + ncellvars = len(self.code.co_cellvars) # not part of the closure + return self.cells[ncellvars:] + def fast2locals(self): PyInterpFrame.fast2locals(self) freevarnames = self.code.co_cellvars + self.code.co_freevars Modified: pypy/trunk/src/pypy/interpreter/pycode.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/pycode.py (original) +++ pypy/trunk/src/pypy/interpreter/pycode.py Tue Sep 30 17:21:29 2003 @@ -4,6 +4,7 @@ The bytecode interpreter itself is implemented by the PyFrame class. """ +import dis from pypy.interpreter import eval @@ -93,6 +94,12 @@ # class bodies only have CO_NEWLOCALS. return not (self.co_flags & CO_OPTIMIZED) + def getjoinpoints(self): + """Compute the bytecode positions that are potential join points + (for FlowObjSpace)""" + # first approximation + return dis.findlabels(self.co_code) + def enhanceclass(baseclass, newclass, cache={}): # this is a bit too dynamic for RPython, but it looks nice Modified: pypy/trunk/src/pypy/interpreter/pyframe.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/pyframe.py (original) +++ pypy/trunk/src/pypy/interpreter/pyframe.py Tue Sep 30 17:21:29 2003 @@ -33,21 +33,8 @@ if code.dictscope_needed(): self.w_locals = space.newdict([]) # set to None by Frame.__init__ - -## def XXXclone(self): -## f = self.__class__() -## f.space = self.space -## f.bytecode = self.bytecode -## f.w_globals = self.w_globals -## f.w_locals = self.w_locals -## f.w_builtins = self.w_builtins -## f.valuestack = self.valuestack.clone() -## f.blockstack = self.blockstack.clone() -## f.localcells = [x.clone() for x in self.localcells] -## f.nestedcells = [x.clone() for x in self.nestedcells] -## f.last_exception = self.last_exception -## f.next_instr = self.next_instr -## return f + def getclosure(self): + return None def eval(self, executioncontext): "Interpreter main loop!" @@ -107,6 +94,32 @@ if attr == 'f_code': return self.space.wrap(self.code) raise OperationError(self.space.w_AttributeError, w_attr) + ### cloning (for FlowObjSpace) ### + + def getflowstate(self): + mergeablestate = self.getfastscope() + self.valuestack.items + nonmergeablestate = ( + self.blockstack.items[:], + self.last_exception, + self.next_instr, + ) + return mergeablestate, nonmergeablestate + + def setflowstate(self, (mergeablestate, nonmergeablestate)): + self.setfastscope(mergeablestate[:len(self.fastlocals_w)]) + self.valuestack.items = mergeablestate[len(self.fastlocals_w):] + ( + self.blockstack.items, + self.last_exception, + self.next_instr, + ) = nonmergeablestate + + def clone(self): + # Clone the frame, making a copy of the mutable state + cls = self.__class__ + f = cls(self.space, self.code, self.w_globals, self.getclosure()) + f.setflowstate(self.getflowstate()) + ### Frame Blocks ### Copied: pypy/trunk/src/pypy/objspace/flow/flowcontext.py (from rev 1456, pypy/trunk/src/pypy/objspace/ann/cloningcontext.py) ============================================================================== --- pypy/trunk/src/pypy/objspace/ann/cloningcontext.py (original) +++ pypy/trunk/src/pypy/objspace/flow/flowcontext.py Tue Sep 30 17:21:29 2003 @@ -1,178 +1,154 @@ -from pypy.interpreter.executioncontext import ExecutionContext, Stack +from pypy.interpreter.executioncontext import ExecutionContext +from pypy.interpreter.miscutils import Stack from pypy.interpreter.pyframe \ - import ControlFlowException, ExitFrame, PyFrame, Cell -from pypy.objspace.ann.wrapper \ - import union, compatible_frames, unify_frames, W_Anything, W_Constant, \ - W_KnownKeysContainer - -class FunctionInfo(object): - - def __init__(self, ec): - self.ec = ec - self.result = None - self.clones = [] # List of (next_instr, w_obj, force) tuples - self.knownframes = {} # Mapping from next_instr to list of frames - - def addresult(self, result): - self.result = union(self.result, result) - - def addclone(self, next_instr, stack_level, block_level, w_obj, force): - self.clones.append((next_instr, stack_level, block_level, w_obj, force)) - - def getclone(self): - if not self.clones: - return None - next_instr, stack_level, block_level, w_obj, force = self.clones.pop() - frames = self.knownframes[next_instr] - for f in frames: - assert f.next_instr == next_instr - if (f.valuestack.depth() == stack_level and - f.blockstack.depth() == block_level): - f = self.ec.clone_frame(f) - f.restarting = (w_obj, force) - return f - assert False, "no suitable clone found -- impossible" - - def addknown(self, frame): - frames = self.knownframes.setdefault(frame.next_instr, []) - frames.append(frame) - - def iterknown(self, frame): - return iter(self.knownframes.get(frame.next_instr, [])) - - def getresult(self): - return self.result - -class IndeterminateCondition(ControlFlowException): - - def __init__(self, w_obj): - ControlFlowException.__init__(self) - self.w_obj = w_obj - - def action(self, frame, last_instr, context): - info = context.getfunctioninfo(frame) - info.addclone(last_instr, - frame.valuestack.depth(), - frame.blockstack.depth(), - self.w_obj, - True) - info.addclone(last_instr, - frame.valuestack.depth(), - frame.blockstack.depth(), - self.w_obj, - False) - # Abandon this frame; the two clones will take over - raise ExitFrame(None) + import ControlFlowException, ExitFrame, PyFrame +from pypy.objspace.flow.wrapper import W_Variable, W_Constant, UnwrapException +from pypy.translator.controlflow import * + +class SpamBlock(BasicBlock): + + def __init__(self, framestate): + mergeablestate, unmergeablestate = framestate + newstate = [] + inputargs = [] + for w in mergeablestate: + if isinstance(w, W_Variable): + w = W_Variable() # make fresh variables + inputargs.append(w) # collects all variables + newstate.append(w) + self.framestate = newstate, unmergeablestate + BasicBlock.__init__(self, inputargs, inputargs, [], None) + + def patchframe(self, frame, executioncontext): + frame.setflowstate(self.framestate) + executioncontext.crnt_block = self + executioncontext.crnt_ops = self.operations + + def union(self, other): + mergeablestate1, unmergeablestate1 = self.framestate + mergeablestate2, unmergeablestate2 = other.framestate + assert unmergeablestate1 == unmergeablestate2, ( + "non mergeable states reached:\n%r\n%r" % ( + unmergeablestate1, unmergeablestate2)) + assert len(mergeablestate1) == len(mergeablestate2), ( + "non mergeable states (different value stack depth)") + + newstate = [] + for w1, w2 in zip(mergeablestate1, mergeablestate2): + if w1 == w2: + w = w1 + else: + w = W_Variable() + newstate.append(w) + if newstate == mergeablestate1: + return self + elif newstate == mergeablestate2: + return other + else: + return SpamBlock((newstate, unmergeablestate1)) -class CloningExecutionContext(ExecutionContext): +class EggBlock(BasicBlock): - def __init__(self, space): - ExecutionContext.__init__(self, space) - self.functioninfos = {} - # {(bytecode, w_globals): FunctionInfo(), ...} + def __init__(self, prevblock, booloutcome): + BasicBlock.__init__(self, [], prevblock.locals, [], None) + self.prevblock = prevblock + self.booloutcome = booloutcome + + def patchframe(self, frame, executioncontext): + parentblocks = [] + block = self + while isinstance(block, EggBlock): + block = block.prevblock + parentblocks.append(block) + # parentblocks = [Egg, Egg, ..., Egg, Spam] not including self + block.patchframe(frame, executioncontext) + replaylist = self.operations + prevblock = self + for block in parentblocks: + replaylist = ReplayList(block.operations, + prevblock, prevblock.booloutcome, + replaylist) + prevblock = block + executioncontext.crnt_ops = replaylist + +class ReplayList: + + def __init__(self, listtoreplay, nextblock, booloutcome, nextreplaylist): + self.listtoreplay = listtoreplay + self.nextblock = nextblock + self.booloutcome = booloutcome + self.nextreplaylist = nextreplaylist + self.index = 0 + + def append(self, operation): + assert operation == self.listtoreplay[self.index] + self.index += 1 - def getfunctioninfo(self, frame, new=False): - from pypy.interpreter.pyframe import AppFrame, PyFrame - if isinstance(frame, AppFrame): - print "getfunctioninfo from AppFrame", frame.bytecode.co_name - return FunctionInfo(self) - - key = self.makekey(frame) - info = self.functioninfos.get(key) - if info is None: - if not new: - raise KeyError, repr(key) - self.functioninfos[key] = info = FunctionInfo(self) - return info - - def makekey(self, frame): - if isinstance(frame.w_globals, W_Constant): - return (frame.bytecode, id(frame.w_globals.value)) - if isinstance(frame.w_globals, W_KnownKeysContainer): - return (frame.bytecode, id(frame.w_globals.args_w)) + def finished(self): + return self.index == len(self.listtoreplay) + +class FlowExecutionContext(ExecutionContext): + + def __init__(self, space, code, globals): + ExecutionContext.__init__(self, space) + self.code = code + self.w_globals = w_globals = space.wrap(globals) + frame = code.create_frame(space, w_globals) + arg_list = [W_Variable() for w in frame.fastlocals_w] + frame.setfastscope(arg_list) + self.joinpoints = {} + for joinpoint in code.getjoinpoints(): + self.joinpoints[joinpoint] = None + initialblock = SpamBlock(frame.getflowstate()) + self.pendingblocks = [initialblock] + self.graph = FunctionGraph(initialblock, code.co_name) def bytecode_trace(self, frame): - if frame.restarting is not None: - w_obj, flag = frame.restarting - frame.restarting = None - w_obj.force = flag + if isinstance(self.crnt_ops, ReplayList): return - - info = self.getfunctioninfo(frame) - for f in info.iterknown(frame): - if compatible_frames(frame, f): - c1, c2 = unify_frames(frame, f) - if not c2: - # A fixpoint; abandon this frame - raise ExitFrame(None) - return - info.addknown(self.clone_frame(frame)) - - def eval_frame(self, frame): - assert self.space is frame.space - frame.restarting = None - info = self.getfunctioninfo(frame, True) - while frame is not None: - result = ExecutionContext.eval_frame(self, frame) - info.addresult(result) - frame = info.getclone() - w_result = info.getresult() - if w_result is None: - raise TypeError("no result at all?!?!") - return w_result - - def clone_frame(self, frame): - return frame.clone() - - f = frame.clone() - #f = PyFrame(self.space, frame.bytecode, frame.w_globals, frame.w_locals) - f.valuestack = clonevaluestack(frame.valuestack) - f.blockstack = cloneblockstack(frame.blockstack) - f.last_exception = frame.last_exception - f.next_instr = frame.next_instr - f.localcells = clonecells(frame.localcells) - f.nestedcells = clonecells(frame.nestedcells) - return f - -class HelperExecutionContext(CloningExecutionContext): - - def eval_frame(self, frame): - frame.key = object() - result = CloningExecutionContext.eval_frame(self, frame) - return result - - def makekey(self, frame): - return frame.key - - def clone_frame(self, frame): - f = CloningExecutionContext.clone_frame(self, frame) - f.key = frame.key - return f - -def cloneblockstack(stk): - newstk = Stack() - newstk.items = stk.items[:] - return newstk - -def clonevaluestack(stk): - newstk = Stack() - for item in stk.items: - try: - newitem = item.clone() - except AttributeError: - newitem = item - newstk.push(newitem) - return newstk - -def clonecells(cells): - """Clone a list of cells.""" - newcells = [] - for cell in cells: - try: - value = cell.get() - except ValueError: - newcell = Cell() - else: - newcell = Cell(value) - newcells.append(newcell) - return newcells + next_instr = frame.next_instr + if next_instr in self.joinpoints: + block = self.joinpoints[next_instr] + currentframestate = frame.getflowstate() + newblock = SpamBlock(currentframestate) + if block is not None: + newblock = block.union(newblock) + finished = newblock is block + outputargs = [] + for w_output, w_target in zip(currentframestate[0], + newblock.framestate[0]): + if isinstance(w_target, W_Variable): + outputargs.append(w_output) + self.crnt_block.closeblock(Branch(outputargs, newblock)) + # phew + if finished: + raise ExitFrame(None) + newblock.patchframe(frame, self) + self.joinpoints[next_instr] = newblock + + def guessbool(self, w_condition): + if not isinstance(self.crnt_ops, ReplayList): + block = self.crnt_block + ifegg = EggBlock(block, True) + elseegg = EggBlock(block, False) + ifbranch = Branch([], ifegg) + elsebranch = Branch([], elseegg) + branch = ConditionalBranch(w_condition, ifbranch, elsebranch) + block.closeblock(branch) + self.pendingblocks.append(ifegg) + self.pendingblocks.append(elseegg) + raise ExitFrame(None) + replaylist = self.crnt_ops + assert replaylist.finished() + self.crnt_block = replaylist.nextblock + self.crnt_ops = replaylist.nextreplaylist + return replaylist.booloutcome + + def build_flow(self): + while self.pendingblocks: + block = self.pendingblocks.pop(0) + frame = self.code.create_frame(self.space, self.w_globals) + block.patchframe(frame, self) + w_result = frame.eval(self) + if w_result is not None: + self.crnt_block.closeblock(EndBranch(w_result)) Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/objspace.py Tue Sep 30 17:21:29 2003 @@ -1,11 +1,12 @@ # ______________________________________________________________________ -import operator +import sys, operator import pypy from pypy.interpreter.baseobjspace import ObjSpace from pypy.interpreter.pycode import PyCode +from pypy.interpreter.error import OperationError from pypy.objspace.flow.wrapper import * from pypy.translator.controlflow import * -from pypy.objspace.flow.cloningcontext import IndeterminateCondition +from pypy.objspace.flow import flowcontext # ______________________________________________________________________ class FlowObjSpace(ObjSpace): @@ -37,19 +38,43 @@ else: raise TypeError("not wrapped: " + repr(w_obj)) - def build_flow(self, func, w_args, w_kwds): + def getexecutioncontext(self): + return self.executioncontext + + def reraise(self): + #import traceback + #traceback.print_exc() + #ec = self.getexecutioncontext() # .framestack.items[-1] + #ec.print_detailed_traceback(self) + + etype, evalue, etb = sys.exc_info() + if etype is OperationError: + raise etype, evalue, etb # just re-raise it + name = etype.__name__ + if hasattr(self, 'w_' + name): + nt = getattr(self, 'w_' + name) + nv = object.__new__(nt) + if isinstance(evalue, etype): + nv.args = evalue.args + else: + print [etype, evalue, nt, nv], + print '!!!!!!!!' + nv.args = (evalue,) + else: + nt = etype + nv = evalue + raise OperationError, OperationError(nt, nv), etb + + def build_flow(self, func): """ """ code = func.func_code - bytecode = PyCode() - bytecode._from_code(code) - w_globals = self.wrap(func.func_globals) - frame = bytecode.create_frame(self, w_globals) - arg_list = [W_Variable() for w in frame.fastlocals_w] - frame.setfastscope(arg_list) - self._crnt_ops = [] - self._crnt_block = BasicBlock(arg_list, [], self._crnt_ops, None) - self._graph = FunctionGraph(self._crnt_block, code.co_name) + code = PyCode()._from_code(code) + ec = flowcontext.FlowExecutionContext(self, code, func.func_globals) + self.executioncontext = ec + ec.build_flow() + return ec.graph + frames = [frame] while len(frames) > 0: crnt_frame = frames.pop() @@ -70,7 +95,7 @@ else: return bool(obj) context = self.getexecutioncontext() - return context.guessbool() + return context.guessbool(w_obj) # ______________________________________________________________________ @@ -94,6 +119,7 @@ args.append(arg) else: # All arguments are constants: call the operator now + #print >> sys.stderr, 'Constant operation:', op, args try: result = op(*args) except: @@ -102,7 +128,8 @@ return self.wrap(result) w_result = W_Variable() - self._crnt_ops.append(SpaceOperation(name, args_w, w_result)) + spaceop = SpaceOperation(name, args_w, w_result) + self.executioncontext.crnt_ops.append(spaceop) return w_result setattr(FlowObjSpace, name, generic_operator) Modified: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Tue Sep 30 17:21:29 2003 @@ -8,27 +8,34 @@ def setUp(self): self.space = test.objspace('flow') - def codetest(self, source, functionname, args_w): + def codetest(self, source, functionname): glob = {} exec source in glob func = glob[functionname] - w_args = self.space.newtuple(args_w) - w_kwds = self.space.newdict([]) - return self.space.build_flow(func, w_args, w_kwds) + return self.space.build_flow(func) def test_nothing(self): x = self.codetest("def f():\n" " pass\n", - 'f', []) + 'f') self.assertEquals(x.functionname, 'f') self.assertEquals(x.startblock.branch.__class__, EndBranch) - def test_ifthenelse(self): + def test_simplebranch(self): x = self.codetest("def f(i, j):\n" " if i < 0:\n" + " return i\n" + " return j\n", + 'f') + + def test_ifthenelse(self): + x = self.codetest("def g(i):\n" + " pass\n" + "def f(i, j):\n" + " if i < 0:\n" " i = j\n" " return g(i) + 1\n", - 'f', [W_Variable()]) + 'f') if __name__ == '__main__': Modified: pypy/trunk/src/pypy/objspace/flow/wrapper.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/wrapper.py (original) +++ pypy/trunk/src/pypy/objspace/flow/wrapper.py Tue Sep 30 17:21:29 2003 @@ -5,6 +5,8 @@ # This is kinda a hack, but at the same time, I don't see why this was defined # in the object space module in the annotation object space. +from pypy.translator.controlflow import Variable, Constant + class UnwrapException(Exception): pass @@ -43,16 +45,20 @@ # ______________________________________________________________________ -class W_Variable(W_Object): - pass +class W_Variable(W_Object, Variable): + counter = 0 + + def __init__(self): + Variable.__init__(self, 'v%d' % W_Variable.counter) + W_Variable.counter += 1 # ______________________________________________________________________ -class W_Constant(W_Object): +class W_Constant(W_Object, Constant): """A specific constant value.""" def __init__(self, value): - self.value = value + Constant.__init__(self, value) def argsrepr(self): return repr(self.value) Modified: pypy/trunk/src/pypy/translator/controlflow.py ============================================================================== --- pypy/trunk/src/pypy/translator/controlflow.py (original) +++ pypy/trunk/src/pypy/translator/controlflow.py Tue Sep 30 17:21:29 2003 @@ -3,11 +3,14 @@ class BasicBlock: - def __init__(self, input_args, locals, operations, branch): + def __init__(self, input_args, locals, operations, branch=None): self.input_args = input_args self.locals = locals self.operations = operations self.branch = branch + def closeblock(self, branch): + self.operations = tuple(self.operations) # should no longer change + self.branch = branch class Variable: def __init__(self, pseudoname): @@ -22,6 +25,11 @@ self.opname = opname self.args = args # list of variables self.result = result # + def __eq__(self, other): + return (self.__class__ is other.__class__ and + self.opname == other.opname and + self.args == other.args and + self.result == other.result) class Branch: def __init__(self, args=None, target=None): From stephan at codespeak.net Tue Sep 30 17:32:22 2003 From: stephan at codespeak.net (stephan at codespeak.net) Date: Tue, 30 Sep 2003 17:32:22 +0200 (MEST) Subject: [pypy-svn] rev 1472 - pypy/trunk/doc/funding/tools Message-ID: <20030930153222.80B235A63A@thoth.codespeak.net> Author: stephan Date: Tue Sep 30 17:32:21 2003 New Revision: 1472 Modified: pypy/trunk/doc/funding/tools/format.py pypy/trunk/doc/funding/tools/zipmod.py Log: fxed some errors in format Modified: pypy/trunk/doc/funding/tools/format.py ============================================================================== --- pypy/trunk/doc/funding/tools/format.py (original) +++ pypy/trunk/doc/funding/tools/format.py Tue Sep 30 17:32:21 2003 @@ -13,6 +13,7 @@ class parse(object): def __init__(self,text): self.text = text + self.index = 0 def start(self,txt):pass def stop(self,txt):pass @@ -20,39 +21,40 @@ def other(self,txt):pass def any(self,txt):pass def parse(self): - content = self.text[:] - while content: - m = OTHER.match(content) + content = self.text + txtlen = len(content) + while txtlen > self.index: + m = OTHER.match(content,self.index) if m is not None: - #print 'OTHER:',m.group() - content = content[m.span()[1]:] + #print 'OTHER:',m.group(),m.span() + self.index = m.span()[1] self.other(m.group()) continue - m = EMPTY.match(content) + m = EMPTY.match(content,self.index) if m is not None: - #print 'EMPTY:',m.group() - content = content[m.span()[1]:] + #print 'EMPTY:',m.group(),m.span() + self.index = m.span()[1] self.empty(m.group()) continue - m = START.match(content) + m = START.match(content,self.index) if m is not None: - #print 'START:',m.group() - content = content[m.span()[1]:] + #print 'START:',m.group(),m.span() + self.index = m.span()[1] self.start(m.group()) continue - m = STOP.match(content) + m = STOP.match(content,self.index) if m is not None: - #print 'STOP:',m.group() - content = content[m.span()[1]:] + #print 'STOP:',m.group(),m.span() + self.index = m.span()[1] self.stop(m.group()) continue - m = ANY.match(content) + m = ANY.match(content,self.index) if m is not None: - #print 'ANY:',m.group() + #print 'ANY:',m.group(),m.span() + self.index = m.span()[1] txt = m.group() txt.strip() self.any(txt) - content = content[m.span()[1]:] continue class format(parse): @@ -60,6 +62,7 @@ super(format,self).__init__(*argl,**argd) self.level = 0 self.outbuf = [] + self.stack = [] def __str__(self): self.parse() @@ -124,6 +127,12 @@ return ''.join(self.outbuf) def start(self,txt): + i = txt.find(' ') + if i: + elem = txt[1:i] + else: + elem = txt[1:-1] + self.stack.append(elem) txt = txt.strip() if txt.startswith('')) + self.outbuf.append('' % self.stack.pop()) def other(self,txt): txt = txt.strip() @@ -158,6 +169,7 @@ def changeStyle(txt): f = format(str(replaceMargin(txt))) return str(f) + #return str(replaceMargin(txt)) if __name__ == '__main__': import sys Modified: pypy/trunk/doc/funding/tools/zipmod.py ============================================================================== --- pypy/trunk/doc/funding/tools/zipmod.py (original) +++ pypy/trunk/doc/funding/tools/zipmod.py Tue Sep 30 17:32:21 2003 @@ -58,13 +58,18 @@ dirName = os.path.join(tmp, os.path.dirname(fn)) try: os.mkdir(dirName) except OSError: pass - destPath = os.path.join(tmp, fn) - open(destPath, 'w').write(filemap[fn]) + destPath = os.path.join(tmp, fn.replace('/',os.sep)) + d,f = os.path.split(destPath) + try: + os.makedirs(d) + except:pass + fs = open(destPath, 'wb') + fs.write(filemap[fn]) # rebuild a copy of the original ZIP file from tmp. dir. zip = zipfile.ZipFile(docPath, 'w', zipfile.ZIP_DEFLATED) for fn in filenames: - destPath = os.path.join(tmp, fn) + destPath = os.path.join(tmp, fn.replace('/',os.sep)) zip.write(destPath, fn) From arigo at codespeak.net Tue Sep 30 17:41:50 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 17:41:50 +0200 (MEST) Subject: [pypy-svn] rev 1473 - in pypy/trunk/src/pypy: objspace/flow objspace/flow/test translator Message-ID: <20030930154150.7ADDF5A63A@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 17:41:49 2003 New Revision: 1473 Modified: pypy/trunk/src/pypy/objspace/flow/flowcontext.py pypy/trunk/src/pypy/objspace/flow/objspace.py pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py pypy/trunk/src/pypy/objspace/flow/wrapper.py pypy/trunk/src/pypy/translator/controlflow.py Log: It really works! Modified: pypy/trunk/src/pypy/objspace/flow/flowcontext.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/flowcontext.py (original) +++ pypy/trunk/src/pypy/objspace/flow/flowcontext.py Tue Sep 30 17:41:49 2003 @@ -82,6 +82,7 @@ def append(self, operation): assert operation == self.listtoreplay[self.index] + operation.result = self.listtoreplay[self.index].result self.index += 1 def finished(self): Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/objspace.py Tue Sep 30 17:41:49 2003 @@ -23,9 +23,9 @@ # XXX Issue a delayed command to create a tuple and assign to a new W_Variable return W_Variable() - def getattr(self, w_obj, w_key): - # XXX Issue a delayed command - return W_Variable() + #def getattr(self, w_obj, w_key): + # # XXX Issue a delayed command + # return W_Variable() def wrap(self, obj): if isinstance(obj, W_Object): @@ -127,10 +127,9 @@ else: return self.wrap(result) - w_result = W_Variable() - spaceop = SpaceOperation(name, args_w, w_result) + spaceop = SpaceOperation(name, args_w, W_Variable()) self.executioncontext.crnt_ops.append(spaceop) - return w_result + return spaceop.result setattr(FlowObjSpace, name, generic_operator) Modified: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Tue Sep 30 17:41:49 2003 @@ -27,6 +27,7 @@ " return i\n" " return j\n", 'f') + import pdb; pdb.set_trace() def test_ifthenelse(self): x = self.codetest("def g(i):\n" Modified: pypy/trunk/src/pypy/objspace/flow/wrapper.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/wrapper.py (original) +++ pypy/trunk/src/pypy/objspace/flow/wrapper.py Tue Sep 30 17:41:49 2003 @@ -52,6 +52,9 @@ Variable.__init__(self, 'v%d' % W_Variable.counter) W_Variable.counter += 1 + def argsrepr(self): + return self.pseudoname + # ______________________________________________________________________ class W_Constant(W_Object, Constant): Modified: pypy/trunk/src/pypy/translator/controlflow.py ============================================================================== --- pypy/trunk/src/pypy/translator/controlflow.py (original) +++ pypy/trunk/src/pypy/translator/controlflow.py Tue Sep 30 17:41:49 2003 @@ -30,6 +30,9 @@ self.opname == other.opname and self.args == other.args and self.result == other.result) + def __repr__(self): + return '<%s = %s(%s)>' % (self.result, self.opname, + ', '.join([repr(a) for a in self.args])) class Branch: def __init__(self, args=None, target=None): From arigo at codespeak.net Tue Sep 30 17:42:30 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 17:42:30 +0200 (MEST) Subject: [pypy-svn] rev 1474 - pypy/trunk/src/pypy/objspace/flow/test Message-ID: <20030930154230.91FE75A63A@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 17:42:30 2003 New Revision: 1474 Modified: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Log: sorry, removed call to pdb Modified: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Tue Sep 30 17:42:30 2003 @@ -27,7 +27,6 @@ " return i\n" " return j\n", 'f') - import pdb; pdb.set_trace() def test_ifthenelse(self): x = self.codetest("def g(i):\n" From arigo at codespeak.net Tue Sep 30 17:45:48 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 17:45:48 +0200 (MEST) Subject: [pypy-svn] rev 1475 - pypy/trunk/src/pypy/objspace/flow Message-ID: <20030930154548.3A05D5A271@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 17:45:47 2003 New Revision: 1475 Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py Log: 3rd test passes Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/objspace.py Tue Sep 30 17:45:47 2003 @@ -105,7 +105,8 @@ op = getattr(operator, name, None) if not op: - return # Can't do it + if name == 'call': + op = apply def generic_operator(self, *args_w): assert len(args_w) == arity, name+" got the wrong number of arguments" @@ -118,14 +119,15 @@ else: args.append(arg) else: - # All arguments are constants: call the operator now - #print >> sys.stderr, 'Constant operation:', op, args - try: - result = op(*args) - except: - self.reraise() - else: - return self.wrap(result) + if op: + # All arguments are constants: call the operator now + #print >> sys.stderr, 'Constant operation:', op, args + try: + result = op(*args) + except: + self.reraise() + else: + return self.wrap(result) spaceop = SpaceOperation(name, args_w, W_Variable()) self.executioncontext.crnt_ops.append(spaceop) From dinu at codespeak.net Tue Sep 30 17:47:06 2003 From: dinu at codespeak.net (dinu at codespeak.net) Date: Tue, 30 Sep 2003 17:47:06 +0200 (MEST) Subject: [pypy-svn] rev 1476 - pypy/trunk/doc/funding Message-ID: <20030930154706.C45D35A271@thoth.codespeak.net> Author: dinu Date: Tue Sep 30 17:47:06 2003 New Revision: 1476 Modified: pypy/trunk/doc/funding/B3.0_impact.txt Log: fixed wording, adding comment and filter tags Modified: pypy/trunk/doc/funding/B3.0_impact.txt ============================================================================== --- pypy/trunk/doc/funding/B3.0_impact.txt (original) +++ pypy/trunk/doc/funding/B3.0_impact.txt Tue Sep 30 17:47:06 2003 @@ -1,5 +1,6 @@ B.3 Potential impact +:DELETE:BEGIN Describe the strategic impact of the proposed project, for example in reinforcing competitiveness or on solving societal problems. Describe the innovation-related activities. Describe the exploitation and/or @@ -9,6 +10,7 @@ international research activities. (Recommended length ? three pages) +:DELETE:END This project has direct relevance to European competitiveness. Europe is the acknowledged world leader in handheld, mobile, and embedded @@ -25,7 +27,7 @@ Moreover, a great many companies, in deciding what language to develop in, reject Very High Level Languages, despite their known advantages -for programmer productivity and code-reuse and maintainability because +for programmer productivity, code-reuse and maintainability because the code produced simply does not run fast enough. If we give them a *fast* VHLL, they will switch. @@ -58,7 +60,7 @@ more insidious, and more long term. A good workman knows his tools. This is much more than the theoretical knowledge of how his tools ought to work, according to principles learned in school. The way car -mechanics know how cars work is distinctly different from the what you +mechanics know how cars work is distinctly different from what you would know if you had attended classes on 'the principles of the internal combustion engine', let alone what you need to know to just drive the thing. @@ -93,7 +95,7 @@ >Since I presume the goal of PyPy is to implement *Python* in Python, >wouldn't the implementation language be rather insignificant to an ->end-user such as an educator? Why would it be "better" than >CPython? +>end-user such as an educator? Why would it be "better" than CPython? For whatever reason, the complex built_in and the cmath module, implemented in Python, are part of the early pypy codebase. As I had been spending some @@ -149,6 +151,10 @@ Conference) the International Python Conference, and others. ADD SOME GOOD ONES THAT ARE FOR CSC ACADEMICS + How about OOPSLA (Conference on Object-Oriented Programming, +Systems, Languages, and Applications) and ECOOP (European Conference +on Object-Oriented Programming) and the like as the next target? + ASK STOCKHOLM -- will the EU pay for us to go to conferences? even ones we were going to attend anyway? From tismer at codespeak.net Tue Sep 30 18:06:38 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 30 Sep 2003 18:06:38 +0200 (MEST) Subject: [pypy-svn] rev 1477 - in pypy/trunk/doc/funding: . makedoc Message-ID: <20030930160638.F003F5A271@thoth.codespeak.net> Author: tismer Date: Tue Sep 30 18:06:37 2003 New Revision: 1477 Modified: pypy/trunk/doc/funding/B0.0_preamble.txt pypy/trunk/doc/funding/B1.0_objectives.txt pypy/trunk/doc/funding/B2.0.0_new_relevance.txt pypy/trunk/doc/funding/B3.0_impact.txt pypy/trunk/doc/funding/B3.1_standards.txt pypy/trunk/doc/funding/B4.0_resources.txt pypy/trunk/doc/funding/B4.1_subcontracting.txt pypy/trunk/doc/funding/B4.2_other_countries.txt pypy/trunk/doc/funding/B5.0_management.txt pypy/trunk/doc/funding/B6.0_detailed_implementation.txt pypy/trunk/doc/funding/B6.1_plan_introduction.txt pypy/trunk/doc/funding/makedoc/mkdoclist.py pypy/trunk/doc/funding/makedoc/part_b.sxw pypy/trunk/doc/funding/makedoc/wp-toptable.asc Log: more formatting changes of all the .txt files Modified: pypy/trunk/doc/funding/B0.0_preamble.txt ============================================================================== --- pypy/trunk/doc/funding/B0.0_preamble.txt (original) +++ pypy/trunk/doc/funding/B0.0_preamble.txt Tue Sep 30 18:06:37 2003 @@ -1,3 +1,5 @@ +:DELETE:BEGIN + Annex 2 - Proposal Part B IST Priority - ? 2nd call - Guide for Proposers ? Specific Targeted Research Projects ? June 2003 @@ -36,6 +38,8 @@ Part B. +:DELETE:END + *** Front page -- PAGE 1 *** PYPY @@ -66,25 +70,26 @@ Coordinator email Coordinator fax -*** END OF PAGE 1 *** -*** Contents page Page 2 *** -*show contents list* -*** END OF PAGE 2 *** +Contents +========= -*** Proposal Summary Page Page 3 *** +:CONTENTS: -Proposal full title +Proposal Summary Page +======================= + +**Proposal full title:** The full title of the Proposal is 'Pypy: A flexible, modular, self-hosting, next-generation specialising compiler for the Open Source Programming Language Python'. -Proposal acronym +**Proposal acronym:** The proposal acronym is PYPY. -Strategic Objectives Addressed: +**Strategic Objectives:** This proposal directly addresses the Strategic Objective IST-2002-2.3.2.3 - 'Open development platforms for software and @@ -98,10 +103,16 @@ and enhancing openness and security in the public and private sector. -Proposal abstract copied from Part A +**Proposal abstract:** + +:DELETE:BEGIN + +copied from Part A (Actually, we will paste this one INTO Part A) +:DELETE:END + The PyPy project will build a flexible, modular, context-aware implementation of the Open Source programming language Python written in itself. Python is already a very popular Object-Oriented @@ -135,5 +146,3 @@ needs. Finally in Phase four, we will Validate our design choices by .... - -*** END OF PAGE 3 *** \ No newline at end of file Modified: pypy/trunk/doc/funding/B1.0_objectives.txt ============================================================================== --- pypy/trunk/doc/funding/B1.0_objectives.txt (original) +++ pypy/trunk/doc/funding/B1.0_objectives.txt Tue Sep 30 18:06:37 2003 @@ -1,3 +1,5 @@ +:DELETE:BEGIN + B.1 Scientific and technological objectives of the project and state of the art =============================================================================== @@ -16,6 +18,10 @@ (Recommended length ? three pages) """ +:DELETE:END + +Scientific and technological objectives of the project & state of the art +=============================================================================== Introduction ------------ @@ -261,3 +267,30 @@ becomes in fact translated... In summary, to translate PyPy itself to C, we will be using PyPy! + +:DELETE:BEGIN + +XXX this chapter shoud **have the following structure** + +B.1.1 Problem to be solved + +It is also a good idea to clearly identify the problem to be solved + +B.1.2 Quantified specific objectives + +Try to quantify statements whereever made, especially technical targets. + +B.1.3 Current State of the Art + +Demonstrate awareness of the current state-of-the-art and differences between +this proposal and any similar existing or previous projects. + +B.1.4 Beyond the State of the Art + +Show the degree of originality, innovation and promise of progress beyond it. +Strike an appropriate balance in the level of risk associated with the project +compared to its potential benefits - high risk may be acceptable in return for +high benefits. Avoid very large or unacceptable levels of risk. + +:DELETE:END + Modified: pypy/trunk/doc/funding/B2.0.0_new_relevance.txt ============================================================================== --- pypy/trunk/doc/funding/B2.0.0_new_relevance.txt (original) +++ pypy/trunk/doc/funding/B2.0.0_new_relevance.txt Tue Sep 30 18:06:37 2003 @@ -1,3 +1,5 @@ +:DELETE:BEGIN + B.2 Relevance to the objectives of the IST Priority Describe the manner in which the proposed project's goals address the @@ -32,7 +34,9 @@ -------------------------------------- -B2 Relevance to the Objectives of the IST Priority +:DELETE:END + +Relevance to the Objectives of the IST Priority -------------------------------------------------- The PyPy project will build a flexible, modular, context-aware @@ -73,8 +77,9 @@ European businesses and industry and to contributing to the greater well-being of all European citizens. +:DELETE:BEGIN + 2.1 Relevance to the IST Workprogramme -++++++++++++++++++++++++++++++++++++++ The IST thematic priority will contribute directly to realising European policies for the knowledge society as agreed at the Lisbon @@ -85,6 +90,11 @@ - solving 'trust and confidence' problems so as to improve dependability of technologies, infrastructures and applications. + +:DELETE:END + +Relevance to the IST Workprogramme +++++++++++++++++++++++++++++++++++++++ PyPy can contribute to this goal because it is an Open Source language. Open Source programs are more widely trusted than @@ -178,18 +188,37 @@ Also, I have missed some parts, and Nicolas needs to write a bit about constraint programming. +:DELETE:BEGIN + 2.2 European Dimension + +:DELETE:END + +European Dimension ++++++++++++++++++++++ I don't know what to say here. +:DELETE:BEGIN + 2.3 Links and Complimentary with Other Programmes + +:DELETE:END + +Links and Complimentary with Other Programmes +++++++++++++++++++++++++++++++++++++++++++++++++ Direct Info/Semantic Web, correct? +:DELETE:BEGIN + 2.4 Contribution to Community Social Objectives + +:DELETE:END + +Contribution to Community Social Objectives +++++++++++++++++++++++++++++++++++++++++++++++ + I don't know what to say here, either. Have I said it already? 2.5 Contributiuon to EC Policies Modified: pypy/trunk/doc/funding/B3.0_impact.txt ============================================================================== --- pypy/trunk/doc/funding/B3.0_impact.txt (original) +++ pypy/trunk/doc/funding/B3.0_impact.txt Tue Sep 30 18:06:37 2003 @@ -1,6 +1,7 @@ +:DELETE:BEGIN + B.3 Potential impact -:DELETE:BEGIN Describe the strategic impact of the proposed project, for example in reinforcing competitiveness or on solving societal problems. Describe the innovation-related activities. Describe the exploitation and/or @@ -10,8 +11,12 @@ international research activities. (Recommended length ? three pages) + :DELETE:END +Potential impact +=================== + This project has direct relevance to European competitiveness. Europe is the acknowledged world leader in handheld, mobile, and embedded devices. But people working in such industries have long desired a @@ -82,60 +87,59 @@ But let me quote from an article posted to the Python-in-Education mailing list. -FIXME -- I promised Arthur I would fix any typos. - -Date: Sun, 14 Sep 2003 11:52:05 -0400 -From: Arthur -To: edu-sig at python.org -Subject: [Edu-sig] re : If the PyPy Project ... - -List-Id: Python in education - -Terry - - ->Since I presume the goal of PyPy is to implement *Python* in Python, ->wouldn't the implementation language be rather insignificant to an ->end-user such as an educator? Why would it be "better" than CPython? - -For whatever reason, the complex built_in and the cmath module, implemented -in Python, are part of the early pypy codebase. As I had been spending some -time in the complex realm with PyGeo - a simple version of the complex -realm, as these things go - Laura's post gave me the impetus to try to -plugin the pypy implementations. - -Only got stuck on the typing issue. My code tests for -instance(object,complex). The pypy complexobject, unadorned, is a class - -and fails the test. But that leads me into a deeper look at some of the -pypy codebase, trying to understand a little bit of how this kind of issue -are to be dealt with. Not that I got there, yet - but I did seem to have an -avenue to explore I would not have with CPython - as someone who doesn't C, -and has no intention of trying, seriously, to do so. - -As someone living within the limits of having Python as my only real -language, I think that pypy should open things up for me considerably. It -will make Python, I believe, a more attractive educational language, because -it will make someone with a strong foundation in Python - as the language of -choice - a more self-sufficient programmer. - -Presumably - the point is - there will be less cases where the right -approach would be an extension module in C or C++, and a sense of -fundamental compromise should one not be equipped to go there. Many -thousands of folks - using VB and the like - already do involved, -highly performing real world applications and make nice livings doing -so, without being equipped to do C. I am thinking that pypy would put -Python more squarely in that "space". +FIXME -- I promised Arthur I would fix any typos:: -Is any of this so, or just hope? - -Art - -:: - - _______________________________________________ - Edu-sig mailing list - Edu-sig at python.org - http://mail.python.org/mailman/listinfo/edu-sig - ------------------------------------------------------ + Date: Sun, 14 Sep 2003 11:52:05 -0400 + From: Arthur + To: edu-sig at python.org + Subject: [Edu-sig] re : If the PyPy Project ... + + List-Id: Python in education + + Terry - + + >Since I presume the goal of PyPy is to implement *Python* in Python, + >wouldn't the implementation language be rather insignificant to an + >end-user such as an educator? Why would it be "better" than CPython? + + For whatever reason, the complex built_in and the cmath module, implemented + in Python, are part of the early pypy codebase. As I had been spending some + time in the complex realm with PyGeo - a simple version of the complex + realm, as these things go - Laura's post gave me the impetus to try to + plugin the pypy implementations. + + Only got stuck on the typing issue. My code tests for + instance(object,complex). The pypy complexobject, unadorned, is a class - + and fails the test. But that leads me into a deeper look at some of the + pypy codebase, trying to understand a little bit of how this kind of issue + are to be dealt with. Not that I got there, yet - but I did seem to have an + avenue to explore I would not have with CPython - as someone who doesn't C, + and has no intention of trying, seriously, to do so. + + As someone living within the limits of having Python as my only real + language, I think that pypy should open things up for me considerably. It + will make Python, I believe, a more attractive educational language, because + it will make someone with a strong foundation in Python - as the language of + choice - a more self-sufficient programmer. + + Presumably - the point is - there will be less cases where the right + approach would be an extension module in C or C++, and a sense of + fundamental compromise should one not be equipped to go there. Many + thousands of folks - using VB and the like - already do involved, + highly performing real world applications and make nice livings doing + so, without being equipped to do C. I am thinking that pypy would put + Python more squarely in that "space". + + Is any of this so, or just hope? + + Art + + + _______________________________________________ + Edu-sig mailing list + Edu-sig at python.org + http://mail.python.org/mailman/listinfo/edu-sig + ------------------------------------------------------ Here is somebody who is hoping we can give him a language he can understand. Python already is an excellent teaching language. PyPy @@ -168,12 +172,14 @@ communicate with each other while we are apart. Beyond that -- we are willing to take any actions the EU would like to fund. - FIXME - (*) emphasis on this one? If this really happens we'll arguably - become the software European Project with the largest distribution - and user base ever (XXX rephrase :-) - Also mention that the issue (and the word 'PEP') is discussed - later in B3.1_standards.txt? +:: + + FIXME + (*) emphasis on this one? If this really happens we'll arguably + become the software European Project with the largest distribution + and user base ever (XXX rephrase :-) + Also mention that the issue (and the word 'PEP') is discussed + later in B3.1_standards.txt? Ask Stockholm We think you get no money for this in a STREP. True? Modified: pypy/trunk/doc/funding/B3.1_standards.txt ============================================================================== --- pypy/trunk/doc/funding/B3.1_standards.txt (original) +++ pypy/trunk/doc/funding/B3.1_standards.txt Tue Sep 30 18:06:37 2003 @@ -1,11 +1,17 @@ -.3.1 Contributions to standards ------------------------------------ +:DELETE:BEGIN + +3.1 Contributions to Standards Describe contributions to national or international standards which may be made by the project, if any. (Recommended length ? one page) +:DELETE:END + +Contributions to Standards +============================== + There are currently two implementations of Python in common use. The first one, which we will call CPython, but which is what the world knows as Python, is a C implementation of the Python Programming @@ -71,10 +77,12 @@ likely, though not certain, that PyPy will become the standard reference implementation of the Python language. - FIXME - either say 'very likely if all the goals described in the project - are met' or give some more precise conditions, e.g. existence of a - PyPy version largely compatible with existing C extension modules +:: + + FIXME + either say 'very likely if all the goals described in the project + are met' or give some more precise conditions, e.g. existence of a + PyPy version largely compatible with existing C extension modules The complete details of how to write a PEP are themselves an Informational PEP -- PEP #1 in fact. Complete details are to be found Modified: pypy/trunk/doc/funding/B4.0_resources.txt ============================================================================== --- pypy/trunk/doc/funding/B4.0_resources.txt (original) +++ pypy/trunk/doc/funding/B4.0_resources.txt Tue Sep 30 18:06:37 2003 @@ -1,3 +1,5 @@ +:DELETE:BEGIN + B.4 The consortium and project resources Describe the role of the participants and the specific skills of each @@ -14,3 +16,9 @@ (Recommended length ? five pages) +:DELETE:END + +The consortium and project resources +===================================== + +add something! \ No newline at end of file Modified: pypy/trunk/doc/funding/B4.1_subcontracting.txt ============================================================================== --- pypy/trunk/doc/funding/B4.1_subcontracting.txt (original) +++ pypy/trunk/doc/funding/B4.1_subcontracting.txt Tue Sep 30 18:06:37 2003 @@ -1,3 +1,5 @@ +:DELETE:BEGIN + B.4.1 Sub-contracting: If any part of the work is foreseen to be sub-contracted by the @@ -6,6 +8,11 @@ (Recommended length ? one page) +:DELETE:END + +Sub-contracting +================= + I am going to run my accounting figures though the AB Strakt bookkeeper, as part of my management duties. She is a contractor who consults regularly to AB Strakt. Aside from that I don't @@ -14,10 +21,12 @@ firm. This is the sort of thing that AB Strakt does all the time. ASK_STOCKHOLM Do I have to discuss this? + +:: - FIXME - should we mention that other people out of the project - are likely to contribute patches for specific parts, - as is always the case in this kind of open source project? - maybe these contribution would not be significant to the - primary WP goals. + FIXME + should we mention that other people out of the project + are likely to contribute patches for specific parts, + as is always the case in this kind of open source project? + maybe these contribution would not be significant to the + primary WP goals. Modified: pypy/trunk/doc/funding/B4.2_other_countries.txt ============================================================================== --- pypy/trunk/doc/funding/B4.2_other_countries.txt (original) +++ pypy/trunk/doc/funding/B4.2_other_countries.txt Tue Sep 30 18:06:37 2003 @@ -1,3 +1,5 @@ +:DELETE:BEGIN + B.4.2 Other countries : If one or more of the participants is based outside of the EU Member @@ -7,6 +9,11 @@ (Recommended length ? one page). +:DELETE:END + +Other countries +================= + ASK_STOCKHOLM Samuele Pedroni is Swiss. [*]_ But he says that he will move to do this Modified: pypy/trunk/doc/funding/B5.0_management.txt ============================================================================== --- pypy/trunk/doc/funding/B5.0_management.txt (original) +++ pypy/trunk/doc/funding/B5.0_management.txt Tue Sep 30 18:06:37 2003 @@ -1,3 +1,5 @@ +:DELETE:BEGIN + B.5 Project management Describe the organisation, management and decision making structures @@ -7,4 +9,9 @@ (Recommended length ? three pages) +:DELETE:END + +Project Management +===================== +add text! Modified: pypy/trunk/doc/funding/B6.0_detailed_implementation.txt ============================================================================== --- pypy/trunk/doc/funding/B6.0_detailed_implementation.txt (original) +++ pypy/trunk/doc/funding/B6.0_detailed_implementation.txt Tue Sep 30 18:06:37 2003 @@ -1,4 +1,6 @@ -B.6 Detailed Implementation plan +:DELETE:BEGIN + +B.6 Detailed Implementation Plan This section describes in detail the work planned to achieve the objectives for the full duration of the of the proposed project. The @@ -42,3 +44,11 @@ more detailed plan. (recommended length, excluding the forms specified above,- up to 15 pages) + +:DELETE:END + +Detailed Implementation Plan +============================= + +In this section, we describe in detail the work planned to achieve the +objectives for the full duration of the proposed project. Modified: pypy/trunk/doc/funding/B6.1_plan_introduction.txt ============================================================================== --- pypy/trunk/doc/funding/B6.1_plan_introduction.txt (original) +++ pypy/trunk/doc/funding/B6.1_plan_introduction.txt Tue Sep 30 18:06:37 2003 @@ -1,6 +1,6 @@ .. include:: crossreferences.asc -B 6.1 Workplan Introduction +Workplan Introduction =============================== The PyPy project addresses a number of topics, which Modified: pypy/trunk/doc/funding/makedoc/mkdoclist.py ============================================================================== --- pypy/trunk/doc/funding/makedoc/mkdoclist.py (original) +++ pypy/trunk/doc/funding/makedoc/mkdoclist.py Tue Sep 30 18:06:37 2003 @@ -12,13 +12,43 @@ lis.sort() return lis -def mangle_wp_file(txt): +HEAD1 = [ + ":DELETE:BEGIN", + "", + "fake toplevel", + ".............", + "", + ":DELETE:END", + "", + ] + +HEAD2 = [ + ":DELETE:BEGIN", + "", + "fake toplevel", + ".............", + "fake sublevel", + "'''''''''''''", + "", + ":DELETE:END", + "", + ] + + +def mangle_wp_file(txt, name): lines = txt.split("\n") for line in lines: if line.startswith(".. |wp|"): break else: # no WP file + pat = "[bB][0-9].0" + if re.match(pat, name): + head = HEAD1 + else: + head = HEAD2 + lines[:0] = head + txt = "\n".join(lines) return txt #.. |wp| replace:: WP21 wp = line.split()[-1] @@ -26,6 +56,11 @@ lines[:0] =[ ":DELETE:BEGIN", "", + "fake toplevel", + ".............", + "fake sublevel", + "'''''''''''''", + "", "This is the anchor %s_" % wp, "", ":DELETE:END", @@ -52,7 +87,7 @@ def make_new_files(names): for name in names: txt = file(os.path.join("..", name+".txt")).read() - txt = mangle_wp_file(txt) + txt = mangle_wp_file(txt, name) file(name+".txt", "w").write(txt) def make_doc_list(names): Modified: pypy/trunk/doc/funding/makedoc/part_b.sxw ============================================================================== Binary files. No diff available. Modified: pypy/trunk/doc/funding/makedoc/wp-toptable.asc ============================================================================== --- pypy/trunk/doc/funding/makedoc/wp-toptable.asc (original) +++ pypy/trunk/doc/funding/makedoc/wp-toptable.asc Tue Sep 30 18:06:37 2003 @@ -8,19 +8,6 @@

-:DELETE:BEGIN - -toplevel -+++++++++++ - -sublevel1 -.................. - -sublevel2 -########## - -:DELETE:END - |title| ============== From arigo at codespeak.net Tue Sep 30 18:08:52 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 18:08:52 +0200 (MEST) Subject: [pypy-svn] rev 1478 - in pypy/trunk/src/pypy: interpreter objspace/flow/test Message-ID: <20030930160852.AAD545A271@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 18:08:52 2003 New Revision: 1478 Modified: pypy/trunk/src/pypy/interpreter/function.py pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Log: 7 tests pass and 1 fail Modified: pypy/trunk/src/pypy/interpreter/function.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/function.py (original) +++ pypy/trunk/src/pypy/interpreter/function.py Tue Sep 30 18:08:52 2003 @@ -176,7 +176,9 @@ w_args = self.space.newtuple(args_w) w_kwds = self.space.newdict([(wrap(key), w_value) for key, w_value in kwds_w.items()]) - return self.call(w_args, w_kwds) + # go through the object space, don't call directly here + # (for FlowObjSpace) + return self.space.call(wrap(self), w_args, w_kwds) class Method(object): Modified: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Tue Sep 30 18:08:52 2003 @@ -36,7 +36,43 @@ " i = j\n" " return g(i) + 1\n", 'f') - + + def test_print(self): + x = self.codetest("def f(i):\n" + " print i\n", + 'f') + + def test_while(self): + x = self.codetest("def f(i):\n" + " while i > 0:\n" + " i = i - 1\n" + " print i\n", + 'f') + + def test_union_easy(self): + x = self.codetest("def f(i):\n" + " if i:\n" + " pass\n" + " else:\n" + " i = 5\n" + " return i\n", + 'f') + + def test_union_hard(self): + x = self.codetest("def f(i):\n" + " if i:\n" + " i = 5\n" + " return i\n", + 'f') + + def dont_test_while_union(self): + x = self.codetest("def f(i):\n" + " total = 0\n" + " while i > 0:\n" + " total += i\n" + " i = i - 1\n" + " return total\n", + 'f') if __name__ == '__main__': test.main() From arigo at codespeak.net Tue Sep 30 19:02:11 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 19:02:11 +0200 (MEST) Subject: [pypy-svn] rev 1479 - in pypy/trunk/src/pypy: interpreter objspace/flowobjspace/flow/test Message-ID: <20030930170211.7F2E45A271@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 19:02:07 2003 New Revision: 1479 Modified: pypy/trunk/src/pypy/interpreter/pyframe.py pypy/trunk/src/pypy/objspace/flow/flowcontext.py pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Log: 8 tests out of 8! Modified: pypy/trunk/src/pypy/interpreter/pyframe.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/pyframe.py (original) +++ pypy/trunk/src/pypy/interpreter/pyframe.py Tue Sep 30 19:02:07 2003 @@ -107,9 +107,9 @@ def setflowstate(self, (mergeablestate, nonmergeablestate)): self.setfastscope(mergeablestate[:len(self.fastlocals_w)]) - self.valuestack.items = mergeablestate[len(self.fastlocals_w):] + self.valuestack.items[:] = mergeablestate[len(self.fastlocals_w):] ( - self.blockstack.items, + self.blockstack.items[:], self.last_exception, self.next_instr, ) = nonmergeablestate Modified: pypy/trunk/src/pypy/objspace/flow/flowcontext.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/flowcontext.py (original) +++ pypy/trunk/src/pypy/objspace/flow/flowcontext.py Tue Sep 30 19:02:07 2003 @@ -17,6 +17,7 @@ inputargs.append(w) # collects all variables newstate.append(w) self.framestate = newstate, unmergeablestate + #import sys; print >> sys.stderr, "** creating SpamBlock", self.framestate BasicBlock.__init__(self, inputargs, inputargs, [], None) def patchframe(self, frame, executioncontext): Modified: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Tue Sep 30 19:02:07 2003 @@ -43,11 +43,13 @@ 'f') def test_while(self): + #import sys; print >> sys.stderr, "--- starting! ---" x = self.codetest("def f(i):\n" " while i > 0:\n" " i = i - 1\n" " print i\n", 'f') + #import sys; print >> sys.stderr, "--- done! ---" def test_union_easy(self): x = self.codetest("def f(i):\n" @@ -65,7 +67,7 @@ " return i\n", 'f') - def dont_test_while_union(self): + def test_while_union(self): x = self.codetest("def f(i):\n" " total = 0\n" " while i > 0:\n" From arigo at codespeak.net Tue Sep 30 19:13:17 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 19:13:17 +0200 (MEST) Subject: [pypy-svn] rev 1480 - in pypy/trunk/src/pypy/objspace/flow: . test Message-ID: <20030930171317.735F75A271@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 19:12:58 2003 New Revision: 1480 Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Log: for loops seem to work Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/objspace.py Tue Sep 30 19:12:58 2003 @@ -1,7 +1,7 @@ # ______________________________________________________________________ import sys, operator import pypy -from pypy.interpreter.baseobjspace import ObjSpace +from pypy.interpreter.baseobjspace import ObjSpace, NoValue from pypy.interpreter.pycode import PyCode from pypy.interpreter.error import OperationError from pypy.objspace.flow.wrapper import * @@ -87,6 +87,11 @@ return g # ____________________________________________________________ + def do_operation(self, name, *args_w): + spaceop = SpaceOperation(name, args_w, W_Variable()) + self.executioncontext.crnt_ops.append(spaceop) + return spaceop.result + def is_true(self, w_obj): try: obj = self.unwrap(w_obj) @@ -97,6 +102,15 @@ context = self.getexecutioncontext() return context.guessbool(w_obj) + def next(self, w_iter): + w_tuple = self.do_operation("next_and_flag", w_iter) + w_flag = self.do_operation("getitem", w_tuple, W_Constant(1)) + context = self.getexecutioncontext() + if context.guessbool(w_flag): + return self.do_operation("getitem", w_tuple, W_Constant(0)) + else: + raise NoValue + # ______________________________________________________________________ def make_op(name, symbol, arity, specialnames): @@ -129,9 +143,7 @@ else: return self.wrap(result) - spaceop = SpaceOperation(name, args_w, W_Variable()) - self.executioncontext.crnt_ops.append(spaceop) - return spaceop.result + return self.do_operation(name, *args_w) setattr(FlowObjSpace, name, generic_operator) Modified: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Tue Sep 30 19:12:58 2003 @@ -76,5 +76,13 @@ " return total\n", 'f') + def test_simple_for(self): + x = self.codetest("def f(lst):\n" + " total = 0\n" + " for i in lst:\n" + " total += i\n" + " return total\n", + 'f') + if __name__ == '__main__': test.main() From hpk at codespeak.net Tue Sep 30 19:15:24 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 19:15:24 +0200 (MEST) Subject: [pypy-svn] rev 1481 - in pypy/trunk/src/pypy/translator: . test Message-ID: <20030930171524.19C755A271@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 19:15:22 2003 New Revision: 1481 Modified: pypy/trunk/src/pypy/translator/controlflow.py pypy/trunk/src/pypy/translator/test/make_dot.py Log: rehacked the viewer to work more generic Modified: pypy/trunk/src/pypy/translator/controlflow.py ============================================================================== --- pypy/trunk/src/pypy/translator/controlflow.py (original) +++ pypy/trunk/src/pypy/translator/controlflow.py Tue Sep 30 19:15:22 2003 @@ -15,10 +15,15 @@ class Variable: def __init__(self, pseudoname): self.pseudoname = pseudoname + def __repr__(self): + return "<%s>" % self.pseudoname class Constant: def __init__(self, value): self.value = value + + def __repr__(self): + return str(self.value) class SpaceOperation: def __init__(self, opname, args, result): @@ -30,13 +35,14 @@ self.opname == other.opname and self.args == other.args and self.result == other.result) + def __repr__(self): - return '<%s = %s(%s)>' % (self.result, self.opname, - ', '.join([repr(a) for a in self.args])) + return "%s(%s) -> %s" % (self.opname, ", ".join(map(str, self.args)), self.result) class Branch: def __init__(self, args=None, target=None): self.set(args, target) + def set(self, args, target): self.args = args # list of variables self.target = target # basic block instance Modified: pypy/trunk/src/pypy/translator/test/make_dot.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/make_dot.py (original) +++ pypy/trunk/src/pypy/translator/test/make_dot.py Tue Sep 30 19:15:22 2003 @@ -1,4 +1,4 @@ -# +from __future__ import generators """ """ @@ -7,114 +7,195 @@ from pypy.translator.controlflow import * import os -class Source: - def __init__(self): - self.lines = [] - def getsource(self): - content = "\n".join(self.lines) +counters = {} +class Node: + allnodes = [] + def __init__(self, obj): + self.obj = obj + self.allnodes.append(self) + self.edges = [] + self.make_name() + self.data = [] + + def make_name(self): + cls = type(self) + num = counters.setdefault(cls, 0) + counters[cls] += 1 + self.name = '%s%d' % (self.__class__.__name__, num) + + def addedge(self, othernode, name=None): + """ append edge (if not already in the list) + return true if it really was added + """ + if othernode not in self.edges: + self.edges.append((othernode, name)) + return 1 + + def descr_edges(self): + l = [] + for node,name in self.edges: + if node in self.allnodes: + style = node.linestyle() + l.append('edge [style=%s, label="%s"]; %s -> %s;' % (style, name, self, node)) + else: + pass + return "\n".join(l) + + def linestyle(self): + return "solid" + + def get_data(self): + return "\\n".join(self.data) + + def __str__(self): + return self.name + +class NodeFunctionGraph(Node): + def descr_node(self): + """ obj attrs startblock, functionname """ + name = self.name + data = self.get_data() + return '%(name)s [shape=box, label="%(data)s"];' % locals() + + def make_name(self): + self.name = self.obj.functionname + +class NodeBasicBlock(Node): + def descr_node(self): + """ condition, ifbranch, elsebranch """ + name = self.name + data = self.get_data() + #content = "\\n".join([name] + map(repr,self.obj.operations)) + "\\n"+data + return '%(name)s [shape=box, label="%(data)s"];' % locals() + +class NodeBranch(Node): + def descr_node(self): + """ args, target """ + name = self.name + data = self.get_data() + return '%(name)s [shape=box, label="%(data)s"];' % locals() + +class NodeConditionalBranch(Node): + def descr_node(self): + """ args, target """ + name = self.name + data = self.get_data() + return '%(name)s [shape=box, label="%(data)s"];' % locals() + +class NodeEndBranch(Node): + def descr_node(self): + """ args, target """ + name = self.name + data = self.get_data() + return '%(name)s [shape=box, label="%(data)s"];' % locals() - return """digraph test { - node [fontname=Times]; - edge [fontname=Times]; - %(content)s - } - """ % locals() - - def putnode(self, name, node): - self.lines.append('%(name)s [shape=box, label=%(name)s];' % locals()) - - def putedge(self, name, othername): - self.lines.append('%(name)s -> %(othername)s;' % locals()) - -#class Node: -# def __init__(self, **kwargs): -# self.__dict__.update(kwargs) -# -#class NodeFunctionGraph(Node): -# def dotsource(self): +def flatten(*args): + for arg in args: + try: + for atom in apply(flatten,arg): + yield atom + except: yield arg + class DotGen: def __init__(self): - self.graph = {} # node1 : [node1,node2,node3] } - self.names = {} # obj : string - self.blocknum = 0 - self.branchnum = 0 - - def get_source(self, obj): - self.dispatch(obj) - source = Source() - for obj,edges in self.graph.items(): - objname = self.names[obj] - source.putnode(objname, obj) - for edge in edges: - source.putedge(objname, edge) - return source.getsource() + self.nodes = {} + self.counters = {} + + def get_source(self, fun): + self.traverse(fun) + l = [] + for node in self.nodes.values(): + l.append(node.descr_node()) + l.append(node.descr_edges()) + + content = "\n".join(l) + + return """ +digraph test { +node [fontname=Times]; +edge [fontname=Times]; +%(content)s +}""" % locals() + + def makenode(self, obj): + try: + return self.nodes[obj] + except KeyError: + pass + try: + name = obj.__class__.__name__ + except AttributeError: + return + g = globals() + cls = g.get('Node%s' % name, None) + if cls is not None: + node = cls(obj) + self.nodes[obj] = node + return node - def dispatch(self, obj): + def traverse(self, obj, objname=None): try: - return self.names[obj] + return self.nodes[obj] except KeyError: pass - typename = obj.__class__.__name__ - method = getattr(self, "gen_%s" % typename, None) - if method is not None: - return method(obj) - - raise ValueError, "unknown type for %r" % typename - - def append_edge(self, obj, otherobj): - points_to = self.graph.setdefault(obj, []) - points_to.append(otherobj) - - def gen_FunctionGraph(self, obj): - name = obj.functionname - self.names[obj] = name - self.append_edge(obj, self.dispatch(obj.startblock)) - return name - - def gen_BasicBlock(self, obj): - name = "block%s" % self.blocknum - self.blocknum += 1 - self.names[obj] = name - self.append_edge(obj, self.dispatch(obj.branch)) - return name - - def gen_Branch(self, obj): - name = "branch%s" % self.branchnum - self.branchnum += 1 - self.names[obj] = name - self.append_edge(obj, self.dispatch(obj.target)) - return name - - def gen_ConditionalBranch(self, obj): - name = "Condbranch%s" % self.branchnum - self.branchnum += 1 - self.names[obj] = name - self.append_edge(obj, self.dispatch(obj.ifbranch)) - self.append_edge(obj, self.dispatch(obj.elsebranch)) - return name - - def gen_EndBranch(self, obj): - name = "endbranch%s" % self.branchnum - self.branchnum += 1 - self.names[obj] = name - self.graph[obj] = [] - return name - #self.append_edge(self.dispatch(obj.target)) + except TypeError: + return + node = self.makenode(obj) + if node: + self.nodes[obj] = node + for name, attr in obj.__dict__.items(): + trynode = self.traverse(attr, name) + if trynode: + node.addedge(trynode, name) + else: + node.data.append("%s=%r" % (name, attr)) + #print "unknown attribute", name, item + return node + else: + print "unknown obj", obj -def make_ps(fun): +def make_png(fun): dotgen = DotGen() from vpath.local import Path from vpath.adapter.process import exec_cmd dest = Path('/tmp/testgraph.dot') + #print dotgen.get_source(fun) dest.write(dotgen.get_source(fun)) - psdest = dest.newsuffix('.ps') - out = exec_cmd('dot -Tps %s' % str(dest)) + psdest = dest.newsuffix('.png') + out = exec_cmd('dot -Tpng %s' % str(dest)) psdest.write(out) print "wrote", psdest + raise SystemExit, 0 + +if __name__ == '__main__': + i = Variable("i") + sum = Variable("sum") + + conditionres = Variable("conditionres") + conditionop = SpaceOperation("gt", [i, Constant(0)], conditionres) + decop = SpaceOperation("add", [i, Constant(-1)], i) + addop = SpaceOperation("add", [i, sum], sum) + + conditionbranch = ConditionalBranch() + headerbranch = Branch() + headerbranch2 = Branch() + whileblock = BasicBlock([i, sum], [i, sum], [addop, decop], headerbranch2) + whilebranch = Branch([i, sum], whileblock) + + endbranch = EndBranch(sum) + conditionbranch.set(conditionres, whilebranch, endbranch) + + headerblock = BasicBlock([i, sum], [i, conditionres], + [conditionop], conditionbranch) + + headerbranch.set([i, Constant(0)], headerblock) + headerbranch2.set([i, sum], headerblock) + startblock = BasicBlock([i], [i, sum], + [], headerbranch) + + fun = FunctionGraph(startblock, "f") -#if __name__ == '__main__': - - + make_png(fun) From hpk at codespeak.net Tue Sep 30 19:30:51 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 19:30:51 +0200 (MEST) Subject: [pypy-svn] rev 1482 - pypy/trunk/src/pypy/translator/test Message-ID: <20030930173051.C272E5A271@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 19:30:48 2003 New Revision: 1482 Modified: pypy/trunk/src/pypy/translator/test/make_dot.py Log: make it recognize subclasses of known node classes Modified: pypy/trunk/src/pypy/translator/test/make_dot.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/make_dot.py (original) +++ pypy/trunk/src/pypy/translator/test/make_dot.py Tue Sep 30 19:30:48 2003 @@ -119,17 +119,26 @@ %(content)s }""" % locals() + def get_node_class(self, cls): + g = globals() + #print cls.__name__ + nodeclass = g.get('Node%s' % cls.__name__, None) + if nodeclass: + return nodeclass + + for base in cls.__bases__: + nodeclass = self.get_node_class(base) + if nodeclass: + return nodeclass + def makenode(self, obj): try: return self.nodes[obj] except KeyError: pass - try: - name = obj.__class__.__name__ - except AttributeError: + if not hasattr(obj, '__class__'): return - g = globals() - cls = g.get('Node%s' % name, None) + cls = self.get_node_class(obj.__class__) if cls is not None: node = cls(obj) self.nodes[obj] = node @@ -168,7 +177,6 @@ out = exec_cmd('dot -Tpng %s' % str(dest)) psdest.write(out) print "wrote", psdest - raise SystemExit, 0 if __name__ == '__main__': i = Variable("i") From hpk at codespeak.net Tue Sep 30 19:40:14 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 19:40:14 +0200 (MEST) Subject: [pypy-svn] rev 1483 - pypy/trunk/src/pypy/translator/test Message-ID: <20030930174014.25C495A271@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 19:40:12 2003 New Revision: 1483 Modified: pypy/trunk/src/pypy/translator/test/make_dot.py Log: added class name to boxes Modified: pypy/trunk/src/pypy/translator/test/make_dot.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/make_dot.py (original) +++ pypy/trunk/src/pypy/translator/test/make_dot.py Tue Sep 30 19:40:12 2003 @@ -34,9 +34,9 @@ def descr_edges(self): l = [] + style = self.linestyle() for node,name in self.edges: if node in self.allnodes: - style = node.linestyle() l.append('edge [style=%s, label="%s"]; %s -> %s;' % (style, name, self, node)) else: pass @@ -46,7 +46,9 @@ return "solid" def get_data(self): - return "\\n".join(self.data) + l = self.data[:] + l.insert(0, "%s" % self.obj.__class__.__name__) + return "\\n".join(l) def __str__(self): return self.name @@ -83,6 +85,9 @@ data = self.get_data() return '%(name)s [shape=box, label="%(data)s"];' % locals() + def linestyle(self): + return 'dashed' + class NodeEndBranch(Node): def descr_node(self): """ args, target """ From hpk at codespeak.net Tue Sep 30 20:10:41 2003 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 30 Sep 2003 20:10:41 +0200 (MEST) Subject: [pypy-svn] rev 1484 - pypy/trunk/src/pypy/translator/test Message-ID: <20030930181041.E60265A271@thoth.codespeak.net> Author: hpk Date: Tue Sep 30 20:10:41 2003 New Revision: 1484 Modified: pypy/trunk/src/pypy/translator/test/make_dot.py Log: some strange style hacks to satisfy armin's wishes Modified: pypy/trunk/src/pypy/translator/test/make_dot.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/make_dot.py (original) +++ pypy/trunk/src/pypy/translator/test/make_dot.py Tue Sep 30 20:10:41 2003 @@ -37,7 +37,10 @@ style = self.linestyle() for node,name in self.edges: if node in self.allnodes: - l.append('edge [style=%s, label="%s"]; %s -> %s;' % (style, name, self, node)) + if name == 'prevblock': + l.append('edge [style=dotted, weight=0, label="%s"]; %s -> %s;' % (name, self, node)) + else: + l.append('edge [style=%s, label="%s"]; %s -> %s;' % (style, name, self, node)) else: pass return "\n".join(l) @@ -76,14 +79,14 @@ """ args, target """ name = self.name data = self.get_data() - return '%(name)s [shape=box, label="%(data)s"];' % locals() + return '%(name)s [shape=circle, label="%(data)s"];' % locals() class NodeConditionalBranch(Node): def descr_node(self): """ args, target """ name = self.name data = self.get_data() - return '%(name)s [shape=box, label="%(data)s"];' % locals() + return '%(name)s [shape=diamond, label="%(data)s"];' % locals() def linestyle(self): return 'dashed' @@ -93,7 +96,7 @@ """ args, target """ name = self.name data = self.get_data() - return '%(name)s [shape=box, label="%(data)s"];' % locals() + return '%(name)s [shape=circle, label="%(data)s"];' % locals() def flatten(*args): for arg in args: @@ -209,6 +212,8 @@ startblock = BasicBlock([i], [i, sum], [], headerbranch) + startblock.prevblock = headerblock + fun = FunctionGraph(startblock, "f") make_png(fun) From stephan at codespeak.net Tue Sep 30 20:11:10 2003 From: stephan at codespeak.net (stephan at codespeak.net) Date: Tue, 30 Sep 2003 20:11:10 +0200 (MEST) Subject: [pypy-svn] rev 1485 - pypy/trunk/doc/funding/tools Message-ID: <20030930181110.B49A05A63A@thoth.codespeak.net> Author: stephan Date: Tue Sep 30 20:11:10 2003 New Revision: 1485 Modified: pypy/trunk/doc/funding/tools/format.py pypy/trunk/doc/funding/tools/zipmod.py Log: some changes. format doesn't work yet Modified: pypy/trunk/doc/funding/tools/format.py ============================================================================== --- pypy/trunk/doc/funding/tools/format.py (original) +++ pypy/trunk/doc/funding/tools/format.py Tue Sep 30 20:11:10 2003 @@ -15,6 +15,21 @@ self.text = text self.index = 0 + def extractTagName(self,txt): + txt = txt.strip() + if not txt.startswith('<'): + return '' + if txt.find('/') != -1: + return '' + i = txt.find(' ') + if i != -1: + tagname = txt[1:i] + else: + tagname = txt[1:-1] + + #print >> sys.stderr,tagname + return tagname + def start(self,txt):pass def stop(self,txt):pass def empty(self,txt):pass @@ -62,8 +77,7 @@ super(format,self).__init__(*argl,**argd) self.level = 0 self.outbuf = [] - self.stack = [] - + def __str__(self): self.parse() return '\n'.join(self.outbuf) @@ -132,7 +146,6 @@ elem = txt[1:i] else: elem = txt[1:-1] - self.stack.append(elem) txt = txt.strip() if txt.startswith('')) - self.outbuf.append('' % self.stack.pop()) + self.outbuf.append(txt) def other(self,txt): txt = txt.strip() @@ -166,14 +177,78 @@ if txt: self.outbuf.append(txt) +class cut(format): + def __init__(self,*argl,**argd): + super(cut,self).__init__(*argl,**argd) + self.level = 0 + self.write = True + self.stack = [] + + def __str__(self): + self.parse() + return ''.join(self.outbuf) + + def start(self,txt): + if self.write: + self.outbuf.append(txt) + else: + self.stack.append(self.extractTagName(txt)) + + def other(self,txt): + if self.write: + self.outbuf.append(txt) + + def stop(self,txt): + if self.write: + if self.stack and txt.find(self.stack[-1]) != -1: + self.stack.pop() + else: + self.outbuf.append(txt) + + def empty(self,txt): + if self.write: + self.outbuf.append(txt) + + def any(self,txt): + if txt.find('DELETE:BEGIN') != -1: + self.initStartCut() + elif txt.find('DELETE:END') != -1: + self.initStopCut() + else: + if self.write: + self.outbuf.append(txt) + + def initStartCut(self): + self.write = False + self.level = 0 + while 1: + last = self.outbuf.pop() + last = last.strip() + if last.startswith(''): + continue + tag = self.extractTagName(last) + print >> sys.stderr,'startCut',last,self.level,tag + if tag and not self.level: + break + elif tag: + self.level -= 1 + + def initStopCut(self): + self.write = True + def changeStyle(txt): - f = format(str(replaceMargin(txt))) - return str(f) - #return str(replaceMargin(txt)) + txt = str(cut(txt)) + txt = str(replaceMargin(txt)) + txt = str(format(txt)) + return txt if __name__ == '__main__': import sys fn = sys.argv[1] content = open(fn).read() - f = format(str(replaceMargin(content))) + #f = format(str(replaceMargin(content))) + f = str(format(content)) print f Modified: pypy/trunk/doc/funding/tools/zipmod.py ============================================================================== --- pypy/trunk/doc/funding/tools/zipmod.py (original) +++ pypy/trunk/doc/funding/tools/zipmod.py Tue Sep 30 20:11:10 2003 @@ -67,7 +67,9 @@ fs.write(filemap[fn]) # rebuild a copy of the original ZIP file from tmp. dir. - zip = zipfile.ZipFile(docPath, 'w', zipfile.ZIP_DEFLATED) + name,end = os.path.splitext(docPath) + newpath = name+'_cp'+end + zip = zipfile.ZipFile(newpath, 'w', zipfile.ZIP_DEFLATED) for fn in filenames: destPath = os.path.join(tmp, fn.replace('/',os.sep)) zip.write(destPath, fn) @@ -75,7 +77,7 @@ def filterAll(txt): txt = changeStyle(txt) - txt = filter(txt) + #txt = filter(txt) return txt From stephan at codespeak.net Tue Sep 30 20:32:15 2003 From: stephan at codespeak.net (stephan at codespeak.net) Date: Tue, 30 Sep 2003 20:32:15 +0200 (MEST) Subject: [pypy-svn] rev 1486 - pypy/trunk/doc/funding/tools Message-ID: <20030930183215.BE7C05A271@thoth.codespeak.net> Author: stephan Date: Tue Sep 30 20:32:15 2003 New Revision: 1486 Modified: pypy/trunk/doc/funding/tools/format.py Log: bugfixed format Modified: pypy/trunk/doc/funding/tools/format.py ============================================================================== --- pypy/trunk/doc/funding/tools/format.py (original) +++ pypy/trunk/doc/funding/tools/format.py Tue Sep 30 20:32:15 2003 @@ -211,6 +211,9 @@ def any(self,txt): if txt.find('DELETE:BEGIN') != -1: + if not self.write: + print >> sys.stderr,self.outbuf[-20:] + raise Exception('No closing DELETE:END found') self.initStartCut() elif txt.find('DELETE:END') != -1: self.initStopCut() @@ -222,19 +225,24 @@ self.write = False self.level = 0 while 1: + if self.level > 2: + print >> sys.stderr,'LEN',len(self.outbuf) + print >> sys.stderr, self.outbuf[-10:] + raise Exception last = self.outbuf.pop() last = last.strip() + print >> sys.stderr,'startCut',last,self.level if last.startswith(''): continue tag = self.extractTagName(last) - print >> sys.stderr,'startCut',last,self.level,tag if tag and not self.level: break elif tag: self.level -= 1 + print >> sys.stderr,'DONE' def initStopCut(self): self.write = True From arigo at codespeak.net Tue Sep 30 20:35:13 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 20:35:13 +0200 (MEST) Subject: [pypy-svn] rev 1487 - pypy/trunk/src/pypy/translator/test Message-ID: <20030930183513.9AF675A271@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 20:35:13 2003 New Revision: 1487 Modified: pypy/trunk/src/pypy/translator/test/make_dot.py Log: using diamonds instead of circles for Branch Modified: pypy/trunk/src/pypy/translator/test/make_dot.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/make_dot.py (original) +++ pypy/trunk/src/pypy/translator/test/make_dot.py Tue Sep 30 20:35:13 2003 @@ -79,7 +79,7 @@ """ args, target """ name = self.name data = self.get_data() - return '%(name)s [shape=circle, label="%(data)s"];' % locals() + return '%(name)s [shape=diamond, label="%(data)s"];' % locals() class NodeConditionalBranch(Node): def descr_node(self): From arigo at codespeak.net Tue Sep 30 21:34:36 2003 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Sep 2003 21:34:36 +0200 (MEST) Subject: [pypy-svn] rev 1488 - in pypy/trunk/src/pypy: interpreter objspace/flowobjspace/flow/test translator/test Message-ID: <20030930193436.9F5255A271@thoth.codespeak.net> Author: arigo Date: Tue Sep 30 21:34:35 2003 New Revision: 1488 Added: pypy/trunk/src/pypy/translator/test/test_pyrextrans.py (contents, props changed) Modified: pypy/trunk/src/pypy/interpreter/eval.py pypy/trunk/src/pypy/objspace/flow/flowcontext.py pypy/trunk/src/pypy/objspace/flow/objspace.py pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py pypy/trunk/src/pypy/translator/test/buildpyxmodule.py pypy/trunk/src/pypy/translator/test/make_dot.py (contents, props changed) Log: We already have a milestone here: try pypy/translator/test/test_pyrextrans.py Modified: pypy/trunk/src/pypy/interpreter/eval.py ============================================================================== --- pypy/trunk/src/pypy/interpreter/eval.py (original) +++ pypy/trunk/src/pypy/interpreter/eval.py Tue Sep 30 21:34:35 2003 @@ -35,6 +35,15 @@ argnames = argnames + [kwargname] return argnames + def getformalargcount(self): + argnames, varargname, kwargname = self.signature() + argcount = len(argnames) + if varargname is not None: + argcount += 1 + if kwargname is not None: + argcount += 1 + return argcount + UNDEFINED = object() # marker for undefined local variables Modified: pypy/trunk/src/pypy/objspace/flow/flowcontext.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/flowcontext.py (original) +++ pypy/trunk/src/pypy/objspace/flow/flowcontext.py Tue Sep 30 21:34:35 2003 @@ -6,6 +6,7 @@ from pypy.translator.controlflow import * class SpamBlock(BasicBlock): + dead = False def __init__(self, framestate): mergeablestate, unmergeablestate = framestate @@ -21,6 +22,8 @@ BasicBlock.__init__(self, inputargs, inputargs, [], None) def patchframe(self, frame, executioncontext): + if self.dead: + raise ExitFrame(None) frame.setflowstate(self.framestate) executioncontext.crnt_block = self executioncontext.crnt_ops = self.operations @@ -28,9 +31,10 @@ def union(self, other): mergeablestate1, unmergeablestate1 = self.framestate mergeablestate2, unmergeablestate2 = other.framestate - assert unmergeablestate1 == unmergeablestate2, ( - "non mergeable states reached:\n%r\n%r" % ( - unmergeablestate1, unmergeablestate2)) +## XXX reintroduce me +## assert unmergeablestate1 == unmergeablestate2, ( +## "non mergeable states reached:\n%r\n%r" % ( +## unmergeablestate1, unmergeablestate2)) assert len(mergeablestate1) == len(mergeablestate2), ( "non mergeable states (different value stack depth)") @@ -96,12 +100,17 @@ self.code = code self.w_globals = w_globals = space.wrap(globals) frame = code.create_frame(space, w_globals) - arg_list = [W_Variable() for w in frame.fastlocals_w] + formalargcount = code.getformalargcount() + arg_list = ([W_Variable() for i in range(formalargcount)] + + [W_Constant(None)] * (len(frame.fastlocals_w) - + formalargcount)) frame.setfastscope(arg_list) self.joinpoints = {} for joinpoint in code.getjoinpoints(): self.joinpoints[joinpoint] = None initialblock = SpamBlock(frame.getflowstate()) + # only keep arguments of the function in initialblock.input_args + del initialblock.input_args[formalargcount:] self.pendingblocks = [initialblock] self.graph = FunctionGraph(initialblock, code.co_name) @@ -125,6 +134,16 @@ # phew if finished: raise ExitFrame(None) + if block is not None and isinstance(block.operations, tuple): + # patch the old block to point directly at the new block + block.dead = True + block.operations = () + outputargs = [] + for w_output, w_target in zip(block.framestate[0], + newblock.framestate[0]): + if isinstance(w_target, W_Variable): + outputargs.append(w_output) + block.branch = Branch(outputargs, newblock) newblock.patchframe(frame, self) self.joinpoints[next_instr] = newblock @@ -150,7 +169,10 @@ while self.pendingblocks: block = self.pendingblocks.pop(0) frame = self.code.create_frame(self.space, self.w_globals) - block.patchframe(frame, self) + try: + block.patchframe(frame, self) + except ExitFrame: + continue # restarting a dead SpamBlock w_result = frame.eval(self) if w_result is not None: self.crnt_block.closeblock(EndBranch(w_result)) Modified: pypy/trunk/src/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/objspace.py Tue Sep 30 21:34:35 2003 @@ -16,16 +16,17 @@ #self.make_sys() def newdict(self, items_w): - # XXX Issue a delayed command to create a dictionary - return W_Variable() + flatlist_w = [] + for w_key, w_value in items_w: + flatlist_w.append(w_key) + flatlist_w.append(w_value) + return self.do_operation('newdict', *flatlist_w) def newtuple(self, args_w): - # XXX Issue a delayed command to create a tuple and assign to a new W_Variable - return W_Variable() + return self.do_operation('newtuple', *args_w) - #def getattr(self, w_obj, w_key): - # # XXX Issue a delayed command - # return W_Variable() + def newlist(self, args_w): + return self.do_operation('newlist', *args_w) def wrap(self, obj): if isinstance(obj, W_Object): Modified: pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/src/pypy/objspace/flow/test/test_objspace.py Tue Sep 30 21:34:35 2003 @@ -14,75 +14,111 @@ func = glob[functionname] return self.space.build_flow(func) + def reallyshow(self, x): + import os + from pypy.translator.test.make_dot import make_png + make_png(x) + #os.system('xv -nolimits /tmp/testgraph.png') + os.system('gv /tmp/testgraph.ps') + + def show(self, x): + pass # or self.reallyshow(x) + def test_nothing(self): - x = self.codetest("def f():\n" + x = self.codetest("def nothing():\n" " pass\n", - 'f') - self.assertEquals(x.functionname, 'f') + 'nothing') + #self.assertEquals(x.functionname, 'f') self.assertEquals(x.startblock.branch.__class__, EndBranch) + self.show(x) def test_simplebranch(self): - x = self.codetest("def f(i, j):\n" + x = self.codetest("def simplebranch(i, j):\n" " if i < 0:\n" " return i\n" " return j\n", - 'f') + 'simplebranch') + self.show(x) def test_ifthenelse(self): x = self.codetest("def g(i):\n" " pass\n" - "def f(i, j):\n" + "def ifthenelse(i, j):\n" " if i < 0:\n" " i = j\n" " return g(i) + 1\n", - 'f') + 'ifthenelse') + self.show(x) def test_print(self): - x = self.codetest("def f(i):\n" + x = self.codetest("def test_print(i):\n" " print i\n", - 'f') + 'test_print') + self.show(x) def test_while(self): #import sys; print >> sys.stderr, "--- starting! ---" - x = self.codetest("def f(i):\n" + x = self.codetest("def test_while(i):\n" " while i > 0:\n" " i = i - 1\n" - " print i\n", - 'f') + " #print i\n", + 'test_while') #import sys; print >> sys.stderr, "--- done! ---" + self.show(x) def test_union_easy(self): - x = self.codetest("def f(i):\n" + x = self.codetest("def union_easy(i):\n" " if i:\n" " pass\n" " else:\n" " i = 5\n" " return i\n", - 'f') + 'union_easy') + self.show(x) def test_union_hard(self): - x = self.codetest("def f(i):\n" + x = self.codetest("def union_hard(i):\n" " if i:\n" " i = 5\n" " return i\n", - 'f') + 'union_hard') + self.show(x) def test_while_union(self): - x = self.codetest("def f(i):\n" + x = self.codetest("def while_union(i):\n" " total = 0\n" " while i > 0:\n" " total += i\n" " i = i - 1\n" " return total\n", - 'f') + 'while_union') + self.show(x) def test_simple_for(self): - x = self.codetest("def f(lst):\n" + x = self.codetest("def simple_for(lst):\n" " total = 0\n" " for i in lst:\n" " total += i\n" " return total\n", - 'f') + 'simple_for') + self.show(x) + + def test_nested_whiles(self): + src = """ +def nested_whiles(i, j): + s = '' + z = 5 + while z > 0: + z = z - 1 + u = i + while u < j: + u = u + 1 + s = s + '.' + s = s + '!' + return s +""" + x = self.codetest(src, 'nested_whiles') + self.reallyshow(x) if __name__ == '__main__': test.main() Modified: pypy/trunk/src/pypy/translator/test/buildpyxmodule.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/buildpyxmodule.py (original) +++ pypy/trunk/src/pypy/translator/test/buildpyxmodule.py Tue Sep 30 21:34:35 2003 @@ -5,7 +5,7 @@ from vpath.local import Path, mkdtemp import os, sys -debug = 0 +debug = 1 def make_module_from_pyxstring(string, num=[0]): tmpdir = mkdtemp() Modified: pypy/trunk/src/pypy/translator/test/make_dot.py ============================================================================== --- pypy/trunk/src/pypy/translator/test/make_dot.py (original) +++ pypy/trunk/src/pypy/translator/test/make_dot.py Tue Sep 30 21:34:35 2003 @@ -181,8 +181,8 @@ dest = Path('/tmp/testgraph.dot') #print dotgen.get_source(fun) dest.write(dotgen.get_source(fun)) - psdest = dest.newsuffix('.png') - out = exec_cmd('dot -Tpng %s' % str(dest)) + psdest = dest.newsuffix('.ps') + out = exec_cmd('dot -Tps %s' % str(dest)) psdest.write(out) print "wrote", psdest Added: pypy/trunk/src/pypy/translator/test/test_pyrextrans.py ============================================================================== --- (empty file) +++ pypy/trunk/src/pypy/translator/test/test_pyrextrans.py Tue Sep 30 21:34:35 2003 @@ -0,0 +1,79 @@ + +import autopath +from pypy.tool import test + +from pypy.translator.genpyrex import GenPyrex +from pypy.translator.controlflow import * + +from pypy.translator.test.buildpyxmodule import make_module_from_pyxstring +#from pypy.translator.test.make_dot import make_ps + +class TestCase(test.IntTestCase): + def setUp(self): + self.space = test.objspace('flow') + + def makemod(self, source, fname): + fun = self.codetest(source, fname) + fun.source = source + result = GenPyrex(fun).emitcode() + return make_module_from_pyxstring(result) + + def codetest(self, source, functionname): + glob = {} + exec source in glob + func = glob[functionname] + return self.space.build_flow(func) + + def test_simple_func(self): + src = """ +def simple_func(i): + return i+1 +""" + mod = self.makemod(src, 'simple_func') + self.assertEquals(mod.simple_func(1), 2) + + def test_while(self): + src = """ +def while_func(i): + total = 0 + while i > 0: + total = total + i + i = i - 1 + return total +""" + mod = self.makemod(src, 'while_func') + self.assertEquals(mod.while_func(10), 55) + + def test_nested_whiles(self): + src = """ +def nested_whiles(i, j): + s = '' + z = 5 + while z > 0: + z = z - 1 + u = i + while u < j: + u = u + 1 + s = s + '.' + s = s + '!' + return s +""" + mod = self.makemod(src, 'nested_whiles') + self.assertEquals(mod.nested_whiles(111, 114), + '...!...!...!...!...!') + + def dont_yet_test_poor_man_range(self): + src = """ +def poor_man_range(i): + lst = [] + while i > 0: + i = i - 1 + lst.append(i) + lst.reverse() + return lst +""" + mod = self.makemod(src, 'poor_man_range') + self.assertEquals(mod.poor_man_range(10), range(10)) + +if __name__ == '__main__': + test.main() From tismer at codespeak.net Tue Sep 30 21:46:34 2003 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 30 Sep 2003 21:46:34 +0200 (MEST) Subject: [pypy-svn] rev 1489 - in pypy/trunk/doc/funding: . makedoc tools Message-ID: <20030930194634.8626D5A271@thoth.codespeak.net> Author: tismer Date: Tue Sep 30 21:46:33 2003 New Revision: 1489 Modified: pypy/trunk/doc/funding/B6.7.wp42_several_obj_impl.txt pypy/trunk/doc/funding/makedoc/part_b.sxw pypy/trunk/doc/funding/tools/format.py Log: much progress on the formatting. fomatter now works. Modified: pypy/trunk/doc/funding/B6.7.wp42_several_obj_impl.txt ============================================================================== --- pypy/trunk/doc/funding/B6.7.wp42_several_obj_impl.txt (original) +++ pypy/trunk/doc/funding/B6.7.wp42_several_obj_impl.txt Tue Sep 30 21:46:33 2003 @@ -23,7 +23,9 @@ **Objectives** - Providing several object implementations for + - general optimisation + - application specific alternatives .. include:: wp-tableend.asc Modified: pypy/trunk/doc/funding/makedoc/part_b.sxw ============================================================================== Binary files. No diff available. Modified: pypy/trunk/doc/funding/tools/format.py ============================================================================== --- pypy/trunk/doc/funding/tools/format.py (original) +++ pypy/trunk/doc/funding/tools/format.py Tue Sep 30 21:46:33 2003 @@ -191,19 +191,16 @@ def start(self,txt): if self.write: self.outbuf.append(txt) - else: - self.stack.append(self.extractTagName(txt)) + self.stack.append(self.write) def other(self,txt): if self.write: self.outbuf.append(txt) - def stop(self,txt): - if self.write: - if self.stack and txt.find(self.stack[-1]) != -1: - self.stack.pop() - else: - self.outbuf.append(txt) + def stop(self,txt): + write = self.stack.pop() + if write: + self.outbuf.append(txt) def empty(self,txt): if self.write: @@ -223,7 +220,10 @@ def initStartCut(self): self.write = False - self.level = 0 + self.level = 0 + #CT adjust stack top for enclosing tag + self.stack.pop() + self.stack.append(self.write) while 1: if self.level > 2: print >> sys.stderr,'LEN',len(self.outbuf) @@ -249,7 +249,8 @@ def changeStyle(txt): txt = str(cut(txt)) - txt = str(replaceMargin(txt)) + #txt = str(replaceMargin(txt)) + # not correct yet txt = str(format(txt)) return txt